Page History

Existing : /v1/ch/{cm-handle}/data/ds/{datastore-name}

CPS Proposed : /v1/batch/data/ds/{datastore-name}

      ( include cm handles into payload / body and leave datastore into url)

-----------------------------------------------------------------------------------------------------------------------

17 Apr 2023 Meeting : kafka message schema & batch interface extension

POST http://localhost:8080/ncmp/v1/batch/data&topic=my-topic-name

{
              operations: [
                             {
                                           operation: read,
                                           datastore: "...",
                                           options: "...",
                                           resourceIdentifier: "...",
                                           cmhandleIds: [4, 6]
                             },
                             {
                                           operation: read,
                                           datastore: "...",
                                           options: "...",
                                           resourceIdentifier: "...",
                                           cmhandleIds: [1, 2, 3]
                             }
              ]
}

23 May 2023 

POST http://localhost:8080/ncmp/v1/data&topic=my-topic-name

agreed 08 Mar 2023 with kieran mccarthy : Follow the existing interfaces as much as possible for consistency and efficiency

------------------------------------------------------------------------------------------------------------

23 May 2023 kieran mccarthy 

We had an internal review with some of our rApp colleagues around some of the recent proposed NCMP batch interface and they came back with some valid comment. 

The proposal is that we should not distinguish batch from bulk or other flavours of read/write. 

The aim is to only have a single flavour of interface for read or write for clients.  Therefore the proposal is to drop “batch” from the interface URL and just act toward “data”  (reads/writes/actions)

CPS prefers keep interface similar as single cm handle interface (consistency and cost)

Existing : ...&topic=topicParamInQuery

support in ONAP DMI-plugin

Response always Async ie. topic is compulsory ?

Suggestions

1. Silently ignore
2. (initial) error response
   1. should we combine all errors in one message?

Schema of bulk response

1) Agreement required on the structure of the response.  Please see response structure below.
2) Does 'eventTime' field which holds the timestamp of the bulk response event,  required or can it be dropped?

{
  "$schema": "https://json-schema.org/draft/2019-09/schema",
  "$id": "urn:cps:org.onap.cps.ncmp.events.async:batch-event-headers:1.0.0",
  "$ref": "#/definitions/BatchEventHeaders",
  "definitions": {
    "BatchEventHeaders": {
      "description": "The header information of the Batch event.",
      "type": "object",
      "javaType" : "org.onap.cps.ncmp.events.async.v1.BatchEventHeaders",
      "properties": {
        "eventId": {
          "description": "The unique id for identifying the event.",
          "type": "string"
        },
        "eventCorrelationId": {
          "description": "The request id received by NCMP as an acknowledgement.",
          "type": "string"
        },
        "eventTime": {
          "description": "The time of the event. It should be in RFC format ('yyyy-MM-		     dd'T'HH:mm:ss.SSSZ').",
          "type": "string"
        },
        "eventTarget": {
          "description": "The destination topic to forward the consumed event.",
          "type": "string"
        },
        "eventSource": {
          "description": "The source of the event.",
          "type": "string"
        },
        "eventType": {
          "description": "The type of the Batch event.",
          "type": "string"
        },
        "eventSchema": {
          "description": "The schema of the Batch event payload.",
          "type": "string"
        },
        "eventSchemaVersion": {
          "description": "The schema version of the Batch event payload.",
          "type": "string"
        }
      },
      "required": [
        "eventId",
        "eventCorrelationId",
        "eventTarget",
        "eventType",
        "eventSchema",
        "eventSchemaVersion"
      ],
      "additionalProperties": false
    }
  }
}

{
  "eventId":            "4cb32729-85e3-44d1-aa6e-c923b9b059a5",
  "eventCorrelationId": "68f15800-8ed4-4bae-9e53-27a9e03e1911",
  "eventTime":          "2023-03-28T14:29:23.876+0000",
  "eventTarget":        "client-topic"
  "eventSource":        "dmi-plugin:enm-1"(dmi service name)
  "eventType":          "org.onap.cps.ncmp.event.model.BulkResponseEvent",
  "eventSchema":        "urn:cps:org.onap.cps.ncmp.events.async:batch-event-schema:1.0.0",
  "schemaVersion":      "1.0.0",

}

{
  "$schema": "https://json-schema.org/draft/2019-09/schema",
  "$id": "urn:cps:org.onap.cps.ncmp.events.async:batch-event-schema:1.0.0",
  "$ref": "#/definitions/BatchDataResponseEvent",
  "definitions": {
    "BatchDataResponseEvent": {
      "description": "The payload of batch event.",
      "type": "object",
      "javaType" : "org.onap.cps.ncmp.events.async.v1.BatchEvent",
      "properties": {
        "event": {
          "description": "The content of Batch event.",
          "type": "object",
          "existingJavaType": "java.lang.Object",
          "additionalProperties": false
        }
      },
      "required": [
        "event"
      ],
      "additionalProperties": false
    }
  }
}  

{     
  "event": {
     "payload": "response of batch cm handles"
  } 
}

Not keeping the 'eventTarget' which comes from the (BulkResponseEvent(In progress of the structure agreement)).

23 May 2023 kieran mccarthy 

We have had some internal discussions including with some O-RAN standards representatives and one of the outcomes is that it would be good if we aligned with the some community standards for event header definitions.  IT is proposed (initially from AT&T) that we should follow Cloud Native Computing Foundation (CNCF) specification as defined in their cloudevents incubator project.

This would mean a small update to the event header names.  Not changing the contents – only the names that we already have.

This is a summary of the cloudevents headers.

                                                                                                   Image Removed

Batch interface message events structure

kafka Headers

Field 

Type

Description

M=Mandatory, O=Optional   

Notes

    Decision        

eventId

M

Agreed  on11 May 2023 with kieran mccarthy 

23 May 2023 kieran mccarthy

Deided field name as "id" 

eventCorrelationId

M

It's requestId that NCMP sent to client as an ACK.

Example : “request-1234" (UUID)

eventTime

O

Example of Internet date/time format.

      1985-04-12T23:20:50.52Z

   This represents 20 minutes and 50.52 seconds after the 23rd hour of

   April 12th, 1985 in UTC.

Agreed 11 May 2023 with kieran mccarthy

The timestamp should follow that on https://www.rfc-editor.org/rfc/rfc3339#section-5.

This follows ISO 8601 and is what is used/referenced in 3GPP standards

23 May 2023 kieran mccarthy

Deided field name as "time"

data

eventTarget

M

eventSource

O

Agreed 11 May 2023 with kieran mccarthy

23 May 2023 kieran mccarthy

Deided field name as "source"

eventType

M

• NCMP would own eventType that would be used by dmi-plugin 

eventSchema

M

• NCMP would own eventSchema that would be used by dmi-plugin 

kieran mccarthy confirmed :

'org.onap.cps.ncmp.events:batch-event-schema'

26 May 2023 Csaba Kocsiswould update ???? 

23 May 2023 kieran mccarthy

Deided field name as "datacontenttype"

schemaVersion

M

For example a version might be 1.0.0 (without “v”)

On versioning O-RAN follows https://semver.org.

kieran mccarthy confirmed  "eventSchemaVersion"  

Event Payload

M

data

.responses[0, 1, 2, ...]

M

Agreed on 15 May 2023 with kieran mccarthy

NCMP will not amalgamate any data 

data

.responses[0].operationId

M

NCMP wil not check uniqueness of 'operationId'

data

.responses[0].ids

kieran mccarthy  confirmed on 16 May 2023 

Question : If read there will only be a single cmhandle in this array ??

Ans : 

data.responses[0].

statusCode

Agreed 15 May 2023 with kieran mccarthy

data.responses[0].

statusMessage

M

Successfully applied changes.

100

cm handle id(s) is(are) not found.

cm handle id(s) is(are) in non ready state.

Agreed 15 May 2023 with kieran mccarthy

curl --location 'http: //localhost:8080/ncmp/v1/data?topic=my-topic-name' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic Y3BzdXNlcjpjcHNyMGNrcyE=' \
--data '{
    "operations": [
        {
            "operation": "read",
            "operationId": "12",
            "datastore": "ncmp-datastore:passthrough-operational",
            "options": "

some option",
            "resourceIdentifier": "parent/child",
            "targetIds": [
                "836bb62201f34a7aa056a47bd95a81ed",
                "202acb75b4a54e43bb1ff8c0c17a8e08"
            ]
        },
        {
            "operation": "read",
            "operationId": "14",
            "datastore": "ncmp-datastore:passthrough-running",
            "targetIds": [
                "ec2e9495679a43c58659c07d87025e72",
                "0df4d39af6514d99b816758148389cfd"
            ]
        }
    ]
}'

{
    "requestId": "4753fc1f-7de2-449a-b306-a6204b5370b3"
}

DMI Service 1 (POST): http://172.26.202.25:8783/dmi/v1/data?topic=my-topic-name&requestId=4753fc1f-7de2-449a-b306-a6204b5370b3 -> 

[
    {
        "operationType": "read",
        "operationId": "14",
        "datastore": "ncmp-datastore:passthrough-running",
        "cmHandles": [
            {
                "id": "ec2e9495679a43c58659c07d87025e72",
                "cmHandleProperties": {
                    "neType": "RadioNode"
                }
            },
            {
                "id": "0df4d39af6514d99b816758148389cfd",
                "cmHandleProperties": {
                    "neType": "RadioNode"
                }
            }
        ]
    }
]

DMI Service 2 (POST) : http://172.26.202.26:8783/dmi/v1/data?topic=my-topic-name&requestId=4753fc1f-7de2-449a-b306-a6204b5370b3 -> 

   [
    {
        "operationType": "read",
        "operationId": "12",
        "datastore": "ncmp-datastore:passthrough-operational",
        "options": "

some option",
        "resourceIdentifier": "parent/child",
        "cmHandles": [
            {
                "id": "836bb62201f34a7aa056a47bd95a81ed",
                "cmHandleProperties": {
                    "neType": "RadioNode"
                }
            },
            {
                "id": "202acb75b4a54e43bb1ff8c0c17a8e08",
                "cmHandleProperties": {
                    "neType": "RadioNode"
                }
            }
        ]
    

Agreed 11 May 2023 with Csaba Kocsis  and kieran mccarthy

requestId would be send as a path param from NCMP to DMI-plugin.

Example 

NCMP batch endpoint : http://localhost:8080/ncmp/v1/data?&topic=my-topic-name

DMI-Plugin batch endpoint : http://172.26.202.25:8783/dmi/v1/data?topic=my-topic-name&requestId=e6fa4d26-4dc1-4877-aa3c-45e99f840708

}
]

{
  "$schema": "https://json-schema.org/draft/2019-09/schema",
  "$id": "urn:cps:org.onap.cps.ncmp.events.async:batch-event-schema:1.0.0",
  "$ref": "#/definitions/BatchDataResponseEvent",
  "definitions": {
    "BatchDataResponseEvent": {
      "description": "The payload of batch event.",
      "type": "object",
      "javaType" : "org.onap.cps.ncmp.events.async.v1.BatchEvent",
      "properties": {
        "event": {
          "description": "The content of Batch event.",
          "type": "object",
          "existingJavaType": "java.lang.Object",
  

Flow Step

Short description

Message Details

Notes

Bulk Get Request

body:
["cm-1",...,"cm-n"]

'http://localhost:8080/ncmp/v1/batch/data/ds/ncmp-datastore:passthrough-running?resourceIdentifier=parent/child%26options=(a=1,b=2)&topic=my-topic-name&options=(fields=schemas/schema)' \
--header 'Authorization: Basic Y3BzdXNlcjpjcHNyMGNrcyE=' \
--header 'Content-Type: application/json' \
--data '[ "40137a9771f84459affa795fa1d633ab", "f5a92ec7a7db4d6fbb0e0ce2803a86cc" ]'

{"requestId":"123"}

body: {"Cmhandles":["cm-1",...,"cm-n"],"requestId":123}

The DMI PLugin should be told (included in request) the client topic so that NCMP does not have to 'remember' to relation between request id and client topic!

        "

additionalProperties": false
  

      }
    

},
      "

required": 

[
        "

event

"
      ],
      "

additionalProperties": false
    }
  

}
}  

{     
  "

event": {
     "

payload": "response of batch cm handles"
  } 
}

{     
  "event": {
     "payload": "response of batch cm handles"
  } 
}

{
  "

timestamp":"2023-03-

01T23:

00:

00.345-0400", 
  "

requestId":123,
  "error": "<error-message>",
  

{     
  "event": {
     "payload": "response of batch cm handles"
  } 
}

Response message structure ? (Flow no. 5)

Non responding DMI-plugin 

{
  "timestamp":"2023-03-01T23:00:00.345-0400", 
  "requestId":123,
  "error": "DMI Service Unavailable, {service-name}",
  "Cmhandles":["cm-1",...,"cm-n"]
  }

Response message structure ? (Flow no. 5)

Non existing cm handles

{
	"timestamp":"2023-03-01T23:00:00.345-0400",
    "requestId":123,
	"error":"Cm-Handle not found",
	"Cmhandles":["cm-1",...,"cm-n"]
}

{
	"timestamp":"2023-03-01T23:00:00.345-0400",
    "requestId":123,
	"error":"Cm-Handle not in READY state.",
	"Cmhandles":["cm-1",...,"cm-n"]
}

Existing DMI endpoints are : 

/v1/ch/{cmHandle}/data/ds/{datastore-name}

datastore-name:

1. ncmp-datastore:passthrough-operational
2. ncmp-datastore:passthrough-running

 ...&topic=topicParamInQuery

CPS Proposed :

/v1/ch/batch/data/ds/{datastore-name}

 ...&topic=topicParamInQuery

cm handle ids and requestid into body

delayed because Schema/Headers issue

+4 days