...
Follow principles/patterns of RESTCONF RFC-8040 https://datatracker.ietf.org/doc/html/rfc8040 Follow principles/patterns of yang-patch RFC-8072 https://datatracker.ietf.org/doc/html/rfc8040 Follow principles/patterns of RESTCONF NMDA RFC-8527 https://datatracker.ietf.org/doc/html/rfc8527
Requirements NotePlease note this section was added long after the implementation and focusses on characteristic only.
Characteristics Synchronous single cm-handle pass-trough (read) requests Issues & Decisions Issues Notes Decisions 1 KPI for De-registration of 100 CM-handles This was mentioned. Was this ever agreed, is this a valid use case that needs to be covered together with the Registration ? Not priority for now, but acceptable if we match the registration req. #2 for de-reg 21 May 2024 kieran mccarthy Kolawole Adebisi-Adeolokun
2 DMI delay Could we get some feedback on DMI-delays for other use cases as not mentioned in FS document Awaiting for ETH feedback 21 May 2024 AP On Kolawole Adebisi-Adeolokun and Csaba Kocsis
02 Jul 2024 Provided
3 Number of instances
In some cases, ETH have used 2 instances, can we verify the number of instances for each use case.
Some of the req were defined per instance and resources used : Identify which of these ?
Agreed to;
CPS use 1 instance currently, but should focus on aligning performance with 2 instances for all use case
4 Input Load Distribution the CM-handle search and ID search Currently has 5 parallel request between them distributed at 2.5 each. This fractional distribution isn't feasible for parallel processing; the load should be allocated as whole numbers. Load needs to be distributed at. Would it be acceptable to adjust this distribution to either 2 or 3 parallel requests each (and vice versa ) without any negative repercussions?
Agreed to do 6 parallel request combined total and divide the load to 3 parallel request each
5 Regarding CM-handle search and ID search FS only identified Module performance, are there any testing done towards a combined search of properties and modules in a single query
Confirmed no other testing was previously done on this.....
CPS have the capabilities to do mixed testing. ETH tbc on if they want to consider this ( Csaba Kocsis )
Requirements NotePlease note this section was added long after the implementation and focuses on characteristic and enhancements after this study only.
Excerpt IncludeDW:CPS-2249: NCMP To support Conflict Handling (Policy Execution) using External Service DW:CPS-2249: NCMP To support Conflict Handling (Policy Execution) using External Service
Characteristics It is proposed that reported characteristics will be used as a baseline for NCMP when agreed and sign-off.
Operation Concurrent requests/parallel DMI Delay Response size Performance Requirement (Blue Stone tablet KPI) Notes Sign-Off 1 Registration of 20,000 CM-handles (in batches of 100) 1 (requests are sequential) 100 ms to get module references 1,000 ms to get module resources
N/A
11 CM-Handles/second as per Stone Tablet E2E Wich include module conversion warm-up NCMP Budget: 22 Cm Handles/second Batch Size: 100 (per request) Not using Module Set Tags Time measured start of first rest-call until all cm handle states READY 1,000 unique module references. Five different types of Nodes. So 5 requests for Module Resources. Avg 200 modules each. 2 De-registration of 100 CM-handles 1 (requests are sequential) No Module delays
N/A
11 NEs/second NCMP should target 22 NEs/Second De-registration is currently not mentioned in Stone Tablet KPI or FS, however we have agreed to match the performance of registration for now as de-reg is also not a priority at this point in time
3 CM-handle ID search with Module filter N/A 20,000 CM Handles i.e. 100*20.000 = 2MB As provided by Csaba Kocsis 0.625 seconds/Operation
FS stated 5 parallel request for both ID search and search. CPS to run with 3 parallel each for both ID search and Search meaning a combine total of 6 parallel request 4 CM-handle search with Module filter 3 Run in parallel with #3 N/A 20,000 CM Handles i.e. 500*20.000 = 10MB As provided by Csaba Kocsis 13 seconds/Operation
FS stated 5 parallel request for both ID search and search. CPS to test with 3 parallel each for both ID search and Search meaning a combine total of 6 parallel request 5 Synchronous single CM-handle pass-through read 10 Run in parallel with #6
25 (parallel) request/sec
Read are done in parallel with Write
6 Synchronous single CM-handle pass-through write (CUD) 10 Run in parallel with #5
13 (parallel) request/sec
No response is expected 7 Batch/Bulk Read 1 read request with 200 cmHandle per second
Notes This is for mixed TCs Single KPIs will be monitored in NCMP owned pipeline with our performance every day(2 hrs interval) - Performance Synchronous single cm-handle pass-through (read) requests Parameter
Expectation
Notes
Sign-Off
Parameter
Expectation
Notes
Sign-Off
Request Frequency 250 request per second / 12 threads (see also
Jiraserver ONAP Jira serverId 425b2b0a-557c-3c0c-b515-579789cceedb key CPS-1968
) Single instance of NCMP/CPS For maximum throughput client load should be spread over at least 12 separate sessions (threads/clients) 06 Dec 2023 Kolawole Adebisi-Adeolokun
Average Response Size 5KB
Shall not exceed 5KB Concurrent request 12 clients requests toward 1 NCMP simultaneously DMI also support 12 simultaneous requests 40ms of overhead on top of DMI latency for each requests, at most for NCMP request. This shall remain within 40ms for 12 parallel requests .
Given the DMI delay below; this means up to 240 request per second
DMI Delay 10ms This is not in control of CPS. So for performance testing our stub should simulate a 10ms delay
Assume DMI is 1.25 seconds average DMI response time for high latency, low latency =10 ms, this should also work for DMI Plugin. I.e 40ms ontop of the DMI. 1.25seconds+40ms= 1.29seconds
Test Environment
ExpandCPS and NCMP requests: cpu: 2000m memory: 2Gi limits: memory: 3Gi cpu: 3000m
2. Postgres
requests:
cpu: 4000m
memory: 1Gi
limits:
memory: 3Gi
cpu: 6000m
Security Disable Basic Authentication in Springboot
If configurable from application yaml, then it’s acceptable.
...
ExcerptState Input Behavior Data Notes # Data-Sync Datastore parameter Expected resourcePath
format
Accept-Header Fields
(filter)
Data Source Included DataNodes (config)
1 On Not Specified cpsPath application/yang-data+json N/A Not supported N/A N/A 2 On Not Specified cpsPath application/json N/A Not supported N/A N/A 3 Off Not Specified
cpsPath application/yang-data+json N/A
Not supported
N/A N/A
4 Off Not Specified cpsPath N/A N/A Not supported N/A N/A there are NO DataNode objects in CPS to output as JSON) 5 Off Not Specified other then cpsPath N/A N/A Not supported N/A N/A Not supported Since NCMP can only convert cpsPaths 6 On | Off ncmp/passthrough-operational NCMP does not parse
NCMP does not parse depends on DMI-Plugin (supported in ONAP) Resolve DMI plugin
Forward request to plugin
Output received responseDMI-Plugin config + non-config The DMI plugin may error if the RP or accept header are not supported.
The DMI plugin may forward the request without processing too.7 On | Off ncmp/passthrough-running NCMP does not parse NCMP does not parse depends on DMI-Plugin (supported in ONAP) Resolve DMI plugin
Forward request to plugin
Output received responseDMI-Plugin config-only 8 On ncmp/operational cpsPath application/yang-data+json Not supported in Istanbul releases. Considered for Kohn Release Read from cache
output: application/yang-data+json
CPS-Core config + non-config NCMP/CPS-Core needs to remove DataNode wrapping 9 On ncmp/operational cpsPath application/json Not supported in Istanbul releases. Planned for Kohn Release Read from cache
output: application/json
CPS-Core config + non-config Output will use DataNode wrapping (as is from CPS-Core)
For forwarding (cached config off) dmi-reposne need to be wrapped explicitly in 'DataNode'
10 Off ncmp/operational cpsPath application/yang-data+json to be determined in spike, see issue #28
Resolve DMI plugin
Convert cpsPath to RESTConfPath*
Forward request to plugin | Read from DMI plugin
Output application/yang-data+json
DMI-Plugin config + non-config
11 On | Off ncmp/running cpsPath application/yang-data+json to be determined in spike, see issue #28 Resolve DMI plugin
Convert cpsPath to RESTConfPath*
Forward request to plugin | Read from DMI plugin
Output application/yang-data+json
DMI-Plugin config-only
*Note Convert cpsPath to RESTConfPath wil only support 'absolute' cpsPath for conversion no query-type paths
Read Example Code Blocklanguage xml title Read with fields collapse true
{ncmpRoot}/ncmp/v1/ch/<cmHandle>/data/ds/<datastore>/{dataResourceIdentifier}?fields={fieldsExpression}
URI :{ncmpRoot}/ncmp/v1/ch/node123/data/ds/ncmp-datastores:operational/TopElement[@id=1]/SomeFunction[@id=1]?fields=cell-model:Cell/attributes(attr1;attr2)
Header :
Accept : application/yang-data+json
Response :
200 OK
{
"function-model:SomeFunction": [
{
"id": "1",
"cell-model:Cell": [
{
"id": "Cell-001",
"attributes": {
"attr1": "value1",
"attr2": "value2"
}
},
{
"id": "Cell-002",
"attributes": {
"attr3": "value3",
"attr4": "value4"
}
}
]
}
]
}
Works Items for above.
# Description Component Enables 1 Forward request from NCMP to CPS-Core NCMP 8,9 2 Forward request from NCMP to DMI-Plugin NCMP 6,7 3 Convert json (dataNode) to yang-data+json CPS-Core/NCMP 8 4 Convert cpsPath to RESTConf Path NCMP 10,11 5 Enhance &fields parameter where needed NCMP 10,11+fields option 6 NOT Supported N/A 1,2,3,4,5
Write operations are only supported on the ncmp-datastores:running and ncmp-datastores:passthrough-running datastores The Data Target for all write operation is DMI-Plugin Write operations are only supported for config=true data Fields and similar parameters are not supported for write operations State Input Behavior Notes # Data-Sync Operation Datastore parameter Expected resourcePath
format
Content-Type 1 On | Off Create ncmp/passthrough-running NCMP does not parse
NCMP only checks it is valid JSON, then embeds the data in a larger JSON structure (see CPS-390 page) Resolve DMI plugin
Forward request to plugin
Output received response (success or failure)The DMI plugin may error if the RP or content type are not supported.
The DMI plugin may forward the request without processing too.2 On | Off Replace ncmp/passthrough-running NCMP does not parse
NCMP only checks it is valid JSON, then embeds the data in a larger JSON structure (see CPS-390 page) Resolve DMI plugin
Forward request to plugin
Output received response (success or failure)The DMI plugin may error if the RP or content type are not supported.
The DMI plugin may forward the request without processing too.3 On | Off Delete ncmp/passthrough-running NCMP does not parse
NCMP doesn't expect any input data from application, will create request body to DMI plugin without embedded data. Resolve DMI plugin
Forward request to plugin
Output received response (success or failure)The DMI plugin may error if the RP or content type are not supported.
The DMI plugin may forward the request without processing too.4 On | Off Patch ncmp/passthrough-running NCMP does not parse
NCMP only checks it is valid JSON, then embeds the data in a larger JSON structure (see CPS-390 page) Resolve DMI plugin
Forward request to plugin
Output received response (success or failure)The DMI plugin may error if the RP or content type are not supported.
The DMI plugin may forward the request without processing too.5 On | Off Create ncmp/running cpsPath application/yang-data+json Resolve DMI plugin
Convert cpsPath to RESTConfPath
Forward request to plugin
Output received response (success or failure)
6 On | Off Update ncmp/running cpsPath application/yang-data+json Resolve DMI plugin
Convert cpsPath to RESTConfPath
Forward request to plugin
Output received response (success or failure)
7 On | Off Delete ncmp/running cpsPath N/A Resolve DMI plugin
Convert cpsPath to RESTConfPath
Forward request to plugin
Output received response (success or failure)
8 On | Off Patch ncmp/running cpsPath application/yang-data +json
(*plain patch)
Resolve DMI plugin
Convert cpsPath to RESTConfPath
Forward request to plugin
Output received response (success or failure)
9 On | Off Patch ncmp/running cpsPath application/yang-patch +json Resolve DMI plugin
Convert cpsPath to RESTConfPath
Forward request to plugin
Output received response (success or failure)
Write Example
...
Expand# Req/usecase REST
Method URI
Request/Response Example 1 DMI notifies NCMP of new , deleted or changed cmhandles DMI Plugin NCMP. Including initial registration POST {ncmpRoot}/ncmp/v1/ch/ Scenario : DMI notifies NCMP of new cmhandlesMethod : POSTURI : {ncmpRoot}/ncmp/v1/ch/Header : Content-Type : application/json Code Blocklanguage xml title Request Body
Request Body : {
"dmiPlugin" : "onap.dmi.plugin",
"createdCmHandles" : [ { "cmHandle" : "rf4er5454",
"cmHandleProperties" :
{ "subSystemId" : "system-001" }
}, {..} ],
"updatedCmHandles" : [ .. ],
"removedCmHandles" : [ "node-1", "node-2" , ... ]
}
json attributes:
"dmiPlugin" resolvable servicename "createdCmHandles" used for initial cm handle registrations or subsequent cmhandle creations "updatedCmHandles" Used for updates to cmhandles. Same structure as for create handles "removedCmHandles" array of cmhandles that have been deleted from the network (no additional properties 2 Get all cm handles that support all modules in a given list of modules POST {ncmpRoot}/ncmp/v1/ch/searches
URI : {ncmpRoot}/ncmp/v1/ch/searches
The minimal requirement is if we provide the AND query impl then for OR query the client can send multiple requests
Request Body
Content : application/json
Note: revision should be optional
{ "modules": [ { "moduleName": "", (Mandatory) "revision": "" (Optional) } ] }
Header :Accept : application/jsonResponse :
Should return an array of objects as we may add more data in the future { "cmHandles": [ { "cmHandleId": "xxx" } ] }
3 Request (trigger) Data Sync POST {ncmpRoot}/ncmp/v1/ch/<cmHandle>/syncData
Scenario : Client requests to sync a node
URI : {ncmpRoot}/ncmp/v1/ch/node123/syncData
Response : HTTP-Status code (only, no body)
4 Get model info for CMHandle GET {ncmpRoot}/ncmp/v1/ch/{cmHandle}/modules
Scenario : Get the model data for CMHandle
URI :{ncmpRoot}/ncmp/v1/ch/2334dedf/modules
Header : Accept: application/json
Response :
[
{
"moduleName" : "nc-notifications" ,
"revision" : "2008-07-14" ,
},
{
"moduleName" : "ietf-tls-server" ,
"revision" : "2016-11-02" ,
},
{
"moduleName" : "ietf-ssh-server" ,
"revision" : "2016-11-02" ,
}
] 5 Get all the registered cmhandles for a given plugin GET {ncmpRoot}/ncmp/v1/dmiPlugins/{pluginId}/ch Scenario : Get all cmhandles from NCMP for a given dmiPlugin. May be used for conciliationMethod : GETURI : {ncmpRoot}/ncmp/v1/dmiPlugins/{dmiPlugin}/chHeader :Content-Type : application/json
Code Blocklanguage xml title Response Body
Success Response :
HTTP/1.1 200 Ok
Date: Thu, 26 Jan 2021 20:56:30 GMT
Server: example-server
{ "cmHandles" : [ {
"cmHandle" : "node-1",
"cmHandleProperties " : { "subSystem" : "system-001" }
} ]
}
...