1 Which operation(s) need support for multiple cm handles? Get Create Update (Put) Patch Delete if many what is the priority?
agreed 08 Mar 2023 with kieran mccarthy
Only Get (read)
(in future other operations might be support batch option too)
2 Which datasources should be supported?
Do we need to support passthrough-only no-cached() data only ? (maybe just start with that, support cached data bulk request later)
agreed 08 Mar 2023 with kieran mccarthy :
all passthrough datastores will be supported
Not implemented (yet) response, for non passthrough datastores
3 URL pattern for NCMP bulk endpoints 08 Mar 2023
Existing : /v1/ch/{cm-handle}/data/ds/{datastore-name}
CPS Proposed : /v1/batch/data/ds/{datastore-name}
( include cm handles into payload / body and leave datastore into url)
-----------------------------------------------------------------------------------------------------------------------
17 Apr 2023 Meeting : kafka message schema & batch interface extension
POST http://localhost:8080/ncmp/v1/batch /data &topic=my-topic-name
{ operations: [ { operation: read, datastore: "...", options: "...", resourceIdentifier: "...", cmhandleIds: [4, 6] }, { operation: read, datastore: "...", options: "...", resourceIdentifier: "...", cmhandleIds: [1, 2, 3] } ] }
23 May 2023
POST http://localhost:8080/ncmp/v1/ data&topic=my-topic-name
agreed 08 Mar 2023 with kieran mccarthy : Follow the existing interfaces as much as possible for consistency and efficiency
------------------------------------------------------------------------------------------------------------
23 May 2023 kieran mccarthy
We had an internal review with some of our rApp colleagues around some of the recent proposed NCMP batch interface and they came back with some valid comment.
The proposal is that we should not distinguish batch from bulk or other flavours of read/write.
The aim is to only have a single flavour of interface for read or write for clients. Therefore the proposal is to drop “batch” from the interface URL and just act toward “data” (reads/writes/actions)
4 keep datastore, topic and optional parameters in the URL itself instead into body. CPS prefers keep interface similar as single cm handle interface (consistency and cost)
Existing : ...&topic=topicParamInQuery
agreed 08 Mar 2023 with kieran mccarthy : Follow the existing interfaces as much as possible for consistency and efficiency
5 support in ONAP DMI-plugin
agreed 08 Mar 2023 with Toine Siebelink : ONAP plugin can respond with not implemented yet code,
6 Response always Async ie. topic is compulsory ?
Assume topic is compulsory (defined in OPenApi) → Response therefore wil be 400 if not supplied Agreed 08 Mar 2023 with kieran mccarthy :
Topic is optional but system will respond with 'Not implemented (yet when not specified or blank
7 Should NCMP Amalgamate Async responses from DMI-Plugin before forwarding ? step 6 in flow diagram Agreed 08 Mar 2023 with kieran mccarthy : NCMP wil only forward to client topic no handling tracking or any responses or status of request
8 Handle non responding dmi-plugin Agreed 08 Mar 2023 with kieran mccarthy : No response for 4b then send an error response to the topic given by client
9 Should (can) NCMP check if 'MyTopic' specified by client exist Consider Access Control too. For now NCMP can log error. Client is responsible for topic setup Agreed 08 Mar 2023 with kieran mccarthy : NCMP can log error when forward to 'client topic' Security not in scope (yet)
10 How to handle non-existing CM-Handles (id) Suggestions
Silently ignore (initial) error responseshould we combine all errors in one message? Agreed 08 Mar 2023 with kieran mccarthy :
Additional error (messages) response with all cm-handles that cannot be resolved, also a separate error message wil be sent for each failed DMI
11 Overlap/clash with Deutsche Telekom user story:
Jiraserver ONAP Jira serverId 425b2b0a-557c-3c0c-b515-579789cceedb key CPS-1377
?!
08 Mar 2023 discussed in weekly ONAP meeting the DT user story is affect CPS-Core interface (not NCMP) and the requirement is to execute a query over ALL cm-handles (cached only?!) instead of a given list of cm-handles (~anchors)
12 Schema of bulk response
Jiraserver ONAP Jira columnIds issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 425b2b0a-557c-3c0c-b515-579789cceedb key CPS-1556
1) Agreement required on the structure of the response. Please see response structure below. 2) Does 'eventTime' field which holds the timestamp of the bulk response event, required or can it be dropped?
Code Blocktitle Batch Event Headers Schema collapse true
{
"$schema": "https://json-schema.org/draft/2019-09/schema",
"$id": "urn:cps:org.onap.cps.ncmp.events.async:batch-event-headers:1.0.0",
"$ref": "#/definitions/BatchEventHeaders",
"definitions": {
"BatchEventHeaders": {
"description": "The header information of the Batch event.",
"type": "object",
"javaType" : "org.onap.cps.ncmp.events.async.v1.BatchEventHeaders",
"properties": {
"eventId": {
"description": "The unique id for identifying the event.",
"type": "string"
},
"eventCorrelationId": {
"description": "The request id received by NCMP as an acknowledgement.",
"type": "string"
},
"eventTime": {
"description": "The time of the event. It should be in RFC format ('yyyy-MM- dd'T'HH:mm:ss.SSSZ').",
"type": "string"
},
"eventTarget": {
"description": "The destination topic to forward the consumed event.",
"type": "string"
},
"eventSource": {
"description": "The source of the event.",
"type": "string"
},
"eventType": {
"description": "The type of the Batch event.",
"type": "string"
},
"eventSchema": {
"description": "The schema of the Batch event payload.",
"type": "string"
},
"eventSchemaVersion": {
"description": "The schema version of the Batch event payload.",
"type": "string"
}
},
"required": [
"eventId",
"eventCorrelationId",
"eventTarget",
"eventType",
"eventSchema",
"eventSchemaVersion"
],
"additionalProperties": false
}
}
}
Code Blocktitle Sample Batch Event Headers collapse true
{
"eventId": "4cb32729-85e3-44d1-aa6e-c923b9b059a5",
"eventCorrelationId": "68f15800-8ed4-4bae-9e53-27a9e03e1911",
"eventTime": "2023-03-28T14:29:23.876+0000",
"eventTarget": "client-topic"
"eventSource": "dmi-plugin:enm-1"(dmi service name)
"eventType": "org.onap.cps.ncmp.event.model.BulkResponseEvent",
"eventSchema": "urn:cps:org.onap.cps.ncmp.events.async:batch-event-schema:1.0.0",
"schemaVersion": "1.0.0",
}
Code Blocktitle Batch Event Payload Schema collapse true
{
"$schema": "https://json-schema.org/draft/2019-09/schema",
"$id": "urn:cps:org.onap.cps.ncmp.events.async:batch-event-schema:1.0.0",
"$ref": "#/definitions/BatchDataResponseEvent",
"definitions": {
"BatchDataResponseEvent": {
"description": "The payload of batch event.",
"type": "object",
"javaType" : "org.onap.cps.ncmp.events.async.v1.BatchEvent",
"properties": {
"event": {
"description": "The content of Batch event.",
"type": "object",
"existingJavaType": "java.lang.Object",
"additionalProperties": false
}
},
"required": [
"event"
],
"additionalProperties": false
}
}
}
Code Blocktitle Sample Batch Event Payload collapse true
{
"event": {
"payload": "response of batch cm handles"
}
}
Need to follow schema structure there in #15 under Issues & Decisions section and it is agreed on 11 May 2023
13 Schema for Bulk Response event forwarding to client specified topic
CPS-1557 - NCMP : forward bulk response messages to client topic
Not keeping the 'eventTarget' which comes from the (BulkResponseEvent(In progress of the structure agreement)).
Note: Will consider 'eventTarget', If it finalized the schema from
Jiraserver ONAP Jira columnIds issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 425b2b0a-557c-3c0c-b515-579789cceedb key CPS-1556
Agreed on 11 May 2023 to keep one schema for both events (DMI → NCMP) and (NCMP → ClientApps)
14 How NCMP would forward response? (Response data : ref. message flow #6)
Jiraserver ONAP Jira columnIds issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 425b2b0a-557c-3c0c-b515-579789cceedb key CPS-1556
1. Do we need to send only one response containing all the requested cm handles ? 2. Send single response for each cm handle? 3. Single response message per DMI plugin?kieran mccarthy - Please share your inputs on the above listed questions.
15 Message format for the batch interface 23 May 2023 kieran mccarthy
We have had some internal discussions including with some O-RAN standards representatives and one of the outcomes is that it would be good if we aligned with the some community standards for event header definitions. IT is proposed (initially from AT&T) that we should follow Cloud Native Computing Foundation (CNCF) specification as defined in their cloudevents incubator project.
This would mean a small update to the event header names. Not changing the contents – only the names that we already have.
This is a summary of the cloudevents headers.
kafka Headers Field Type Description M=Mandatory, O=Optional Notes Decision eventId string The unique id identifying the event M
Generated by DMI-Plugin Agreed on11 May 2023 with kieran mccarthy
23 May 2023 kieran mccarthy
Deided field name as "id"
eventCorrelationId string The request id passed by NCMP M
It's requestId that NCMP sent to client as an ACK.
Example : “request-1234" (UUID)
Agreed 11 May 2023 with kieran mccarthy
eventTime string The timestamp when original event occurred O
Example of Internet date/time format.
1985-04-12T23:20:50.52Z
This represents 20 minutes and 50.52 seconds after the 23rd hour of
April 12th, 1985 in UTC.
Agreed 11 May 2023 with kieran mccarthy
The timestamp should follow that on https://www.rfc-editor.org/rfc/rfc3339#section-5 .
This follows ISO 8601 and is what is used/referenced in 3GPP standards
23 May 2023 kieran mccarthy
Deided field name as "time"
eventTarget string The destination topic of the client M
Example : my-topic Agreed 11 May 2023 with kieran mccarthy eventSource string The source of the event O
Example : dmi-plugin:enm-1 (dmi service name) Agreed 11 May 2023 with kieran mccarthy
23 May 2023 kieran mccarthy
Deided field name as "source"
eventType string The type of the event M
NCMP would own eventType that would be used by dmi-plugin kieran mccarthy confirmed :
event type proposed name :
BatchDataXXXEvent ?
For example :
BatchDataResponseEvent
'eventType ' value as below 'org.onap.cps.ncmp.events. BatchDataXXXEvent '
23 May 2023 kieran mccarthy
Deided field name as "type"
eventSchema string The schema of the Batch event payload. M
NCMP would own eventSchema that would be used by dmi-plugin kieran mccarthy confirmed :
'org.onap.cps.ncmp.events:batch-event-schema'
23 May 2023 kieran mccarthy
Deided field name as "datacontenttype"
schemaVersion string The schema version of the Batch event payload. M
For example a version might be 1.0.0 (without “v”)
On versioning O-RAN follows https://semver.org .
kieran mccarthy confirmed "eventSchemaVersion "
specversion (default 1.0)String ?? Event Payload event Event The payload of an event M
Agreed on 11 May 2023 with kieran mccarthy event.batch-responses[0, 1, 2, ...] Array contains an array or batch response that includes both success and failure. M
Agreed on 15 May 2023 with kieran mccarthy
NCMP will not amalgamate any data
event.batch-responses[0].operationId String specified to distinguish multiple operations using same cmhandleId M
Agreed on 15 May 2023 with kieran mccarthy NCMP wil not check uniqueness of 'operationId'
event.batch-responses[0].ids String cmhandle-ids M
Example :
["0df4d39af6514d99b816758148389cfd"]
kieran mccarthy confirmed on 16 May 2023
Question : If read there will only be a single cmhandle in this array ??
Ans :
Ids array should contain only a single element in the array in case of success messages and In case of error it can have any number of elements.
event.batch-responses[0].status-code String M
Common NCMP defined error codes:
status-code 0-99 is reserved for any succeess response status-code from 100 to 199 is reserved for any failed response. Agreed 15 May 2023 with kieran mccarthy
event.batch-responses[0].status-message String M
Example :
status-code status-message 1 "Successfully applied changes" 101 "cmHandle(s) do not exist"
Agreed 15 May 2023 with kieran mccarthy
event.batch-responses[0].data Object O
In case of success :Optional, for write operations then no need to return configurations application/yang-patch+json | application/yang-data+json In case of failure :Optional, any supplementary error data matching the error status-code Agreed 15 May 2023 with kieran mccarthy
16 NCMP wil send only one request to each DMI
Code Blocklanguage bash title NCMP batch endpoint collapse true
curl --location 'http: //localhost:8080/ncmp/v1/batch/ data?topic=my-topic-name' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic Y3BzdXNlcjpjcHNyMGNrcyE=' \
--data '{
"operations": [
{
"operation": "read",
"operationId": "12",
"datastore": "ncmp-datastore:passthrough-operational",
"options": "(fields=schemas/schema)",
"resourceIdentifier": "parent/child",
"targetIds": [
"836bb62201f34a7aa056a47bd95a81ed",
"202acb75b4a54e43bb1ff8c0c17a8e08"
]
},
{
"operation": "read",
"operationId": "14",
"datastore": "ncmp-datastore:passthrough-running",
"targetIds": [
"ec2e9495679a43c58659c07d87025e72",
"0df4d39af6514d99b816758148389cfd"
]
}
]
}'
Code Blocklanguage bash title NCMP batch endpoint response (200->OK) collapse true
{
"requestId": "4753fc1f-7de2-449a-b306-a6204b5370b3"
}
Code Blocklanguage bash title DMI service batch endpoint collapse true
DMI Service 1 (POST): http://172.26.202.25:8783/dmi/v1/batch/ data?topic=my-topic-name&requestId=4753fc1f-7de2-449a-b306-a6204b5370b3 ->
[
{
"operationType": "read",
"operationId": "14",
"datastore": "ncmp-datastore:passthrough-running",
"cmHandles": [
{
"id": "ec2e9495679a43c58659c07d87025e72",
"cmHandleProperties": {
"neType": "RadioNode"
}
},
{
"id": "0df4d39af6514d99b816758148389cfd",
"cmHandleProperties": {
"neType": "RadioNode"
}
}
]
}
]
DMI Service 2 (POST) : http://172.26.202.26:8783/dmi/v1/batch /data?topic=my-topic-name&requestId=4753fc1f-7de2-449a-b306-a6204b5370b3 ->
[
{
"operationType": "read",
"operationId": "12",
"datastore": "ncmp-datastore:passthrough-operational",
"options": "(fields=schemas/schema)",
"resourceIdentifier": "parent/child",
"cmHandles": [
{
"id": "836bb62201f34a7aa056a47bd95a81ed",
"cmHandleProperties": {
"neType": "RadioNode"
}
},
{
"id": "202acb75b4a54e43bb1ff8c0c17a8e08",
"cmHandleProperties": {
"neType": "RadioNode"
}
}
]
}
]
Agreed 11 May 2023 with Csaba Kocsis and kieran mccarthy
requestId would be send as a path param from NCMP to DMI-plugin.
Example
NCMP batch endpoint : http://localhost:8080/ncmp/v1/ batch data ?&topic=my-topic-name
DMI-Plugin batch endpoint : http://172.26.202.25:8783/dmi/v1/ batch data ?topic=my-topic-name&requestId=e6fa4d26-4dc1-4877-aa3c-45e99f840708
Mandatory Fileds : "operation": "read" "operationId": "12" "datastore": "ncmp-datastore:passthrough-operational" "targetIds": [ "0df4d39af6514d99b816758148389cfd", "ec2e9495679a43c58659c07d87025e72" ]
Optional Fields : "options": "(fields=schemas/schema)" "resourceIdentifier": "parent/child"