Create 'bulk' version for org.onap.cps.ncmp.rest.controller.NetworkCmProxyController#getResourceDataForCmHandle to support multiple cm-handles (~ multiple anchors in in CPS-Core)
Table of Contents
References
CPS-1515- Spike: Support multiple CM-Handles for NCMP Get Operation
Support batch read operation (new) using an asynchronous response on a client specified topic
payload includes list of cm handle ids
2
REST DMI-I-01
Support batch read operation (new) using an asynchronous response to an internal topic
payload includes list of (associated for this plugin) cm handles ids with their private (additional) properties
Error Handling
#
Error Scenario
Expected behavior
1
DMI Not respond to initial synchronous request with normal HTTP timeout
Special Error message (Kafka event) including affected, and reason for all the handles in the message cm handles send to client topic detailing cm-handles (per dmi plugin)
2
Topic not supplied on CPS-E-05
Return HTTP 501 (not implemented)
3
Client specided Topic is not configured
Log error message only
4
Non-existing cm-handle id
Similar message but different reason as specified #1 above
Capabilities
Excerpt
#
Parameter
Expectation
Notes
1
Response Time 1 Batch request
<2 seconds (average)
Async response available on client topic
No delay in DMI PLugin Plugin (tested/measured using stub DMI Plugin)
2
Batch-size
200 cm handles
No hardcoded limit
3
Response payload size
~2 KB per cm handles
Performance test for capability should be tested with this average response size
Should not affect performance, does not need to be tested
Out-of-scope
Support for multiple resource identifiers in one batch operation
Support cached data batch requests: only passthrough datastores will be supported (see decision #2)
NCMP does NOT keep track of request status to see if it is completed, amalgamate responses or anything like that. It wil simply forward responses from the internal topic to the client topic.
Access control
Assumptions
#
Assumption
Notes
1
Proprietary options or not in the scope of this analysis.
agreed with kieran mccarthy ; these are optional parameters (name-value pairs) not interpreted by NCMP but can be interpreted by proprietary plugins In the future "scope" might become standardized instead of proprietary but that wil be achieved through a separate requirement
2
same xpath (resourceIdentifierInQuery) for all cm handles or different for each cm handle
agreed with kieran mccarthy ; if different resources are required on the same cm-handle the client wil send another (batch) request
Issues & Decisions
3
options, resourceIdentifier is optional for bulk operations.
operation, datastore and cmhandleIds are mandatory fields
agreed with CPS team.
4
We are not merging any duplicate cm handles while sending request to dmi-plugin
agreed
5
Get batch operation does not support includeDescendants as it is impl. for
pass through datastore only
ncmp-datastore:passthrough-running and
ncmp-datastore:passthrough-operational
agreed on
Issues & Decisions
#
Issue
Notes
Decision
1
Which operation(s) need support for multiple cm handles?
Topic is optional but system will respond with 'Not implemented (yet when not specified or blank
7
We had an internal review with some of our rApp colleagues around some of the recent proposed NCMP batch interface and they came back with some valid comment. The proposal is that we should not distinguish batch from bulk or other flavors of read/write.
The aim is to only have a single flavor of interface for read or write for clients. Therefore the proposal is to drop “batch” from the interface URL and just act toward “data”
Additional error (messages) response with all cm-handles that cannot be resolved, also a separate error message wil be sent for each failed DMI
11
Overlap/clash with Deutsche Telekom user story:
Jira
server
ONAP Jira
serverId
425b2b0a-557c-3c0c-b515-579789cceedb
key
CPS-1377
?!
discussed in weekly ONAP meeting the DT user story is affect CPS-Core interface (not NCMP) and the requirement is to execute a query over ALL cm-handles (cached only?!) instead of a given list of cm-handles (~anchors)
1) Agreement required on the structure of the response. Please see response structure below. 2) Does 'eventTime' field which holds the timestamp of the bulk response event, required or can it be dropped?
Need to follow schema structure there in #15 under Issues & Decisions section and it is agreed on
13
Schema for Bulk Response event forwarding to client specified topic
1. Do we need to send only one response containing all the requested cm handles ? 2. Send single response for each cm handle? This could lead to to much data in the event for large batch requests. Csaba Kocsis , Sourabh Sourabh and Toine Siebelink proposed to us single event for each cm handle 3. Single response message per DMI plugin? Just is a list of IDS should be OK.
We have had some internal discussions including with some O-RAN standards representatives and one of the outcomes is that it would be good if we aligned with the some community standards for event header definitions. IT is proposed (initially from AT&T) that we should follow Cloud Native Computing Foundation (CNCF) specification as defined in their cloudevents incubator project. Some non-standard headers wil need to be implemented as 'extensions' and names to be confirmed
'eventType' value as below 'org.onap.cps.ncmp.events.BatchDataEvent'
proposed Operation like :
'org.onap.cps.ncmp.events.DataOperationEvent'
'batch' was the keyword use in the URL for this feature (now gone) but much code (classnames, variable names) etc uses this name still...
kieran mccarthy and Team agreed on and some cost (1-2 days) is acceptable
18
What would be the field name of response data?
data.responses[0].data
kieran mccarthyplease confirm : we are using 'responseContent' as name of this field as 'data' is already used for cloud event standard payload.
Now it is like : data.responses[0].responseContent
Sourabh Sourabh Suggest to use "result". This is after all where the actual result output. the other tags in responses[x] are status related to the request :
(see org.onap.cps.ncmp.api.impl.utils.EventDateTimeFormatter )
Toine Siebelink Although not clearly communicated before ISO8601 is the standard to be used and it seems the current implementation in NCMP conforms to this standard
To prevent problems with large data request multiple response event will be sent for a successful read for each cm handle (id)
Error messages can contain information about single cm handle ids
Field
Type
Description
Mandatory?
Notes
data
Event
The payload of an event
Mandatory
Cloud event standard (data)
data.responses[0, 1, 2, ...]
Array
contains an array or batch response that includes both success and failure.
Mandatory
data.responses[0].operationId
String
specified to distinguish multiple operations using same cmhandleId
Mandatory
data.responses[0].ids
String
cmhandle-ids
Mandatory
Example : ["0df4d39af6514d99b816758148389cfd"] Note: Ids array should contain only a single element in the array in case of success messages and In case of error it can have any number of elements.
data.responses[0].statusCode
String
Mandatory
Common NCMP defined error codes:
status-code 0-99 is reserved for any success response
status-code from 100 to 199 is reserved for any failed response.
data.responses[0].statusMessage
String
Mandatory
Examples for code & message :
statusCode
statusMessage
0
Successfully applied changes.
100
cm handle id(s) is(are) not found.
101
cm handle id(s) is(are) in non ready state.
data.responses[0].data
Object
Optional
In case of success :
Optional, for write operations then no need to return configurations application/yang-patch+json | application/yang-data+json
In case of failure :
Optional, any supplementary error data matching the error status-code
202 is non-committal, meaning that there is no way for the HTTP to later send an asynchronous response indicating the outcome of processing the request. It is intended for cases where another process or server handles the request, or for batch processing.
{
"$schema": "https://json-schema.org/draft/2019-09/schema",
"$id": "urn:cps:org.onap.cps.ncmp.events:bulk-response-event-schema:v1",
"$ref": "#/definitions/BulkResponseEvent",
"definitions": {
"BulkResponseEvent": {
"description": "The payload for bulk response event.",
"type": "object",
"properties": {
"eventId": {
"description": "The unique id identifying the event generated by DMI for this bulk response event.",
"type": "string"
},
"eventCorrelationId": {
"description": "The request id passed by NCMP for this bulk response event.",
"type": "string"
},
"eventTime": {
"description": "The time of bulk response event. The expected format is 'yyyy-MM-dd'T'HH:mm:ss.SSSZ'.",
"type": "string"
},
"eventType": {
"description": "The
type
payload of
bulk response
batch event.",
"type": "
string"
}
object",
"
eventSchema
javaType"
: {
"description": "The event schema for bulk response event.",
Single response format for all scenarios bot positive and error, just using optional fields instead
7
Error for 4/5 → Non responding DMI, non existing CM Handles or any other error. NCMP will have to create error message detailing all cm-handle ids affect by that error.
The DMI PLugin should be told (included in request) the client topic so that NCMP does not have to 'remember' to relation between request id and client topic!
...
title
Response 202
collapse
true
...
Code Block
title
Topic: ncmp-async-m2m
collapse
true
Message Details TBD
...
Code Block
title
Topic: <client topic>
collapse
true
Message Details TBD
...
Response message structure ? (Flow no. 5)
Non responding DMI-plugin
cm-1",...,"cm-n"]
}
Single response format for all scenarios bot positive and error, just using optional fields instead See decision # 8