Page History

...

#

Issue

Notes

Decision

1

Which operation(s) need support for multiple cm handles?

Get
Create
Update (Put)
Patch
Delete

if many what is the priority?

agreed 08 Mar 2023 with kieran mccarthy

Only Get (read)

(in future other operations might be support batch option too)

2

Which datasources should be supported?

Do we need to support passthrough-only no-cached() data only ?
(maybe just start with that, support cached data bulk request later)

agreed 08 Mar 2023 with kieran mccarthy :

all passthrough datastores will be supported

Not implemented (yet) response, for non passthrough datastores

3

URL pattern for NCMP bulk endpoints

~~08 Mar 2023~~

~~Existing : /v1/ch/{cm-handle}/data/ds/{datastore-name}~~

~~CPS Proposed : /v1/batch/data/ds/{datastore-name}~~

~~( include cm handles into payload / body and leave datastore into url)~~

~~-----------------------------------------------------------------------------------------------------------------------~~

~~17 Apr 2023 Meeting : kafka message schema & batch interface extension~~

~~POST http://localhost:8080/ncmp/v1/batch&topic=my-topic-name~~

{
operations: [
{
operation: read,
datastore: "...",
options: "...",
resourceIdentifier: "...",
cmhandleIds: [4, 6]
},
{
operation: read,
datastore: "...",
options: "...",
resourceIdentifier: "...",
cmhandleIds: [1, 2, 3]
}
]
}

~~agreed~~ 08 Mar 2023 ~~with kieran mccarthy : Follow the existing interfaces as much as possible for consistency and efficiency~~

------------------------------------------------------------------------------------------------------------

4

keep datastore, topic and optional parameters in the URL itself instead into body.

CPS prefers keep interface similar as single cm handle interface (consistency and cost)

Existing : ...&topic=topicParamInQuery

agreed 08 Mar 2023 with kieran mccarthy : Follow the existing interfaces as much as possible for consistency and efficiency

5

support in ONAP DMI-plugin

agreed 08 Mar 2023 with Toine Siebelink : ONAP plugin can respond with not implemented yet code,

6

Response always Async ie. topic is compulsory ?

Assume topic is compulsory (defined in OPenApi) → Response therefore wil be 400 if not supplied

Agreed 08 Mar 2023 with kieran mccarthy :

Topic is optional but system will respond with 'Not implemented (yet when not specified or blank

7

Should NCMP Amalgamate Async responses from DMI-Plugin before forwarding ?

step 6 in flow diagram

Agreed 08 Mar 2023 with kieran mccarthy : NCMP wil only forward to client topic no handling tracking or any responses or status of request

8

Handle non responding dmi-plugin

Agreed 08 Mar 2023 with kieran mccarthy :
No response for 4b then send an error response to the topic given by client

9

Should (can) NCMP check if 'MyTopic' specified by client exist

Consider Access Control too. For now NCMP can log error. Client is responsible for topic setup

Agreed 08 Mar 2023 with kieran mccarthy :
NCMP can log error when forward to 'client topic' Security not in scope (yet)

10

How to handle non-existing CM-Handles (id)

Suggestions

Silently ignore
(initial) error response
1. should we combine all errors in one message?

Agreed 08 Mar 2023 with kieran mccarthy :

Additional error (messages) response with all cm-handles that cannot be resolved, also a separate error message wil be sent for each failed DMI

11

Overlap/clash with Deutsche Telekom user story:

Jira

server	ONAP Jira
serverId	425b2b0a-557c-3c0c-b515-579789cceedb
key	CPS-1377

?!

08 Mar 2023 discussed in weekly ONAP meeting the DT user story is affect CPS-Core interface (not NCMP) and the requirement is to execute a query over ALL cm-handles (cached only?!) instead of a given list of cm-handles (~anchors)

12

Schema of bulk response

Jira

server	ONAP Jira
columnIds	issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columns	key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId	425b2b0a-557c-3c0c-b515-579789cceedb
key	CPS-1556

1) Agreement required on the structure of the response. Please see response structure below.
2) Does 'eventTime' field which holds the timestamp of the bulk response event, required or can it be dropped?

Code Block

title	Schema of bulk response
collapse	true

{
  "$schema": "https://json-schema.org/draft/2019-09/schema",
  "$id": "urn:cps:org.onap.cps.ncmp.events:bulk-response-event-schema:v1",
  "$ref": "#/definitions/BulkResponseEvent",
  "definitions": {
    "BulkResponseEvent": {
      "description": "The payload for bulk response event.",
      "type": "object",
      "properties": {
        "eventId": {
          "description": "The unique id identifying the event generated by DMI for this bulk response event.",
          "type": "string"
        },
        "eventCorrelationId": {
          "description": "The request id passed by NCMP for this bulk response event.",
          "type": "string"
        },
        "eventTime": {
          "description": "The time of bulk response event. The expected format is 'yyyy-MM-dd'T'HH:mm:ss.SSSZ'.",
          "type": "string"
        },
        "eventType": {
          "description": "The type of bulk response event.",
          "type": "string"
        },
        "eventSchema": {
          "description": "The event schema for bulk response event.",
          "type": "string"
        },
        "eventSchemaVersion": {
          "description": "The event schema version for bulk response event.",
          "type": "string"
        },
        "event": {
        "$ref": "#/definitions/Event"
        }
      },
      "required": [
        "eventId",
        "eventCorrelationId",
        "eventTime",
        "eventType",
        "eventSchema",
        "eventSchemaVersion"
      ]
    },
    "Event": {
      "description": "The bulk response event content.",
      "type": "object",
      "existingJavaType": "java.lang.Object"
    }
  }
}

Code Block

title	Sample bulk response
collapse	true

{
  "eventId": "4cb32729-85e3-44d1-aa6e-c923b9b059a5",
  "eventCorrelationId": "68f15800-8ed4-4bae-9e53-27a9e03e1911",
  "eventTime": "2023-03-28T14:29:23.876+0000",
  "eventType": "org.onap.cps.ncmp.event.model.BulkResponseEvent",
  "eventSchema": "urn:cps:org.onap.cps.ncmp.event.model.BulkResponseEvent",
  "eventSchemaVersion": "v1",
  "event": {
    "payload": "response of batch cm handles"
  }
}

13

Schema for Bulk Response event forwarding to client specified topic

CPS-1557 - NCMP : forward bulk response messages to client topic

Not keeping the 'eventTarget' which comes from the BulkResponseEvent.

Note: Will consider 'eventTarget', If it finalized the schema from

Jira

server	ONAP Jira
columnIds	issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columns	key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId	425b2b0a-557c-3c0c-b515-579789cceedb
key	CPS-1556

Code Block

title	Schema of bulk response
collapse	true

{
  "$schema": "https://json-schema.org/draft/2019-09/schema",
  "$id": "urn:cps:org.onap.cps.ncmp.events:async-bulk-response-event-schema:v1",
  "$ref": "#/definitions/AsyncBulkResponseEvent",
  "definitions": {
    "AsyncBulkResponseEvent": {
      "description": "The payload for bulk response event.",
      "type": "object",
      "properties": {
        "eventId": {
          "description": "The unique id identifying the event generated by DMI for this bulk response event.",
          "type": "string"
        },
        "eventCorrelationId": {
          "description": "The request id passed by NCMP for this bulk response event.",
          "type": "string"
        },
        "eventTime": {
          "description": "The time of bulk response event. The expected format is 'yyyy-MM-dd'T'HH:mm:ss.SSSZ'.",
          "type": "string"
        },
        "eventType": {
          "description": "The type of bulk response event.",
          "type": "string"
        },
        "eventSchema": {
          "description": "The event schema for bulk response event.",
          "type": "string"
        },
        "eventSchemaVersion": {
          "description": "The event schema version for bulk response event.",
          "type": "string"
        },
        "event": {
        "$ref": "#/definitions/Event"
        }
      },
      "required": [
        "eventId",
        "eventCorrelationId",
        "eventTime",
        "eventType",
        "eventSchema",
        "eventSchemaVersion"
      ]
    },
    "Event": {
      "description": "The bulk response event content.",
      "type": "object",
      "existingJavaType": "java.lang.Object"
    }
  }
}

Code Block

title	Sample bulk response
collapse	true

{
  "eventId": "4cb32729-85e3-44d1-aa6e-c923b9b059a5",
  "eventCorrelationId": "68f15800-8ed4-4bae-9e53-27a9e03e1911",
  "eventTime": "2023-03-28T14:29:23.876+0000",
  "eventType": "org.onap.cps.ncmp.event.model.AysncBulkResponseEvent",
  "eventSchema": "urn:cps:org.onap.cps.ncmp.event.model.AysncBulkResponseEvent",
  "eventSchemaVersion": "v1",
  "event": {
    "payload": "response of batch cm handles"
  }
}

<Note. use green for closed issues, yellow for important ones if needed>

Implementation

Bulk Request Message Flow

Gliffy Diagram

macroId	5966a68a-1641-430a-8c78-27c938f06e15
displayName	Cm Handle Batch Message Flow
name	Cm Handle Batch Interface Design
pagePin	21

Message Flow details

14

How NCMP would to forward response? (Response data : ref. message flow #6)

Jira

server	ONAP Jira
columnIds	issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columns	key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId	425b2b0a-557c-3c0c-b515-579789cceedb
key	CPS-1556

1. Do we need to send only one response containg all the requested cm handles ?
2. Send single response for each cm handle?
3. Single response message per dmi plugin?

<Note. use green for closed issues, yellow for important ones if needed>

Implementation

Bulk Request Message Flow

Gliffy Diagram

macroId	5966a68a-1641-430a-8c78-27c938f06e15
displayName	Cm Handle Batch Message Flow
name	Cm Handle Batch Interface Design
pagePin	21

Message Flow details

Flow Step

Short description

Message Details

Notes

1

Bulk Get Request

Code Block

title	/v1/batch/data/ds/{datastore-name}
collapse	true

body:
["cm-1",...,"cm-n"]

Code Block

title	Example CURL cmd : curl --location --request POST
collapse	true

'http://

Flow Step

Short description

Message Details

Notes

1

Bulk Get Request

Code Block

title	/v1/batch/data/ds/{datastore-name}
collapse	true

body:
["cm-1",...,"cm-n"]

Code Block

title	Example CURL cmd : curl --location --request POST
collapse	true

'http://localhost:8080/ncmp/v1/batch/data/ds/ncmp-datastore:passthrough-running?resourceIdentifier=parent/child%26options=(a=1,b=2)&topic=my-topic-name&options=(fields=schemas/schema)' \
--header 'Authorization: Basic Y3BzdXNlcjpjcHNyMGNrcyE=' \
--header 'Content-Type: application/json' \
--data '[ "40137a9771f84459affa795fa1d633ab", "f5a92ec7a7db4d6fbb0e0ce2803a86cc" ]'

Define new get operation "getResourceDataForCmHandles" into ncmp.yml

2

Ack Client Request

Code Block

title	Response 200
collapse	true

{"requestId":"123"}

3

DMI Bulk Request

Code Block

title	/v1/ch/batch/data/ds/{datastore-name}&topic=topicParamInQuery
collapse	true

body: {"Cmhandles":["cm-1",...,"cm-n"],"requestId":123}

The DMI PLugin should be told (included in request) the client topic so that NCMP does not have to 'remember' to relation between request id and client topic!

4

Ack Client Request

Code Block

title	Response 202
collapse	true

5

Kafka Messages from DMI to NCMP

Code Block

title	Topic: ncmp-async-m2m
collapse	true

Message Details TBD

6

Kafka Message(s) from NCMP to Client

Code Block

title	Topic: <client topic>
collapse	true

Message Details TBD

7

Alternative for 4/5 → Non responding DMI. NCMP will have to create error message detailing cm-handles

See decision # 8 and 9

8

Response message structure ? (Flow no. 5)

Non responding DMI-plugin

Code Block

title	Non responding DMI-plugin
collapse	true

{
  "timestamp":"2023-03-01T23:00:00.345-0400", 
  "requestId":123,
  "error": "DMI Service Unavailable, {service-name}",
  "Cmhandles":["cm-1",...,"cm-n"]
  }

9

Response message structure ? (Flow no. 5)

Non existing cm handles

Code Block

title	Non existing cm handles
collapse	true

{
	"timestamp":"2023-03-01T23:00:00.345-0400",
    "requestId":123,
	"error":"Cm-Handle not found",
	"Cmhandles":["cm-1",...,"cm-n"]
}

10

Non Ready cm handles

Code Block

title	Non READY existing cm handles
collapse	true

{
	"timestamp":"2023-03-01T23:00:00.345-0400",
    "requestId":123,
	"error":"Cm-Handle not in READY state.",
	"Cmhandles":["cm-1",...,"cm-n"]
}

11

URL pattern for DMI-Plugin bulk endpoints

Existing DMI endpoints are :

/v1/ch/{cmHandle}/data/ds/{datastore-name}

datastore-name:

ncmp-datastore:passthrough-operational
ncmp-datastore:passthrough-running

...&topic=topicParamInQuery

CPS Proposed :

/v1/ch/batch/data/ds/{datastore-name}

...&topic=topicParamInQuery

cm handle ids and requestid into body

Proposed JIRAs :

endpoints

Existing DMI endpoints are :

/v1/ch/{cmHandle}/data/ds/{datastore-name}

datastore-name:

ncmp-datastore:passthrough-operational
ncmp-datastore:passthrough-running

...&topic=topicParamInQuery

CPS Proposed :

/v1/ch/batch/data/ds/{datastore-name}

...&topic=topicParamInQuery

cm handle ids and requestid into body

Proposed JIRAs :

3ONAP Jira1555155681558

#

Component

Description

JIRAs

Estimate

1

DMI-Plugin

Accept datastore name as param into URL

Jira

server	ONAP Jira
serverId	425b2b0a-557c-3c0c-b515-579789cceedb
key	CPS-1550

5 Days

2

NCMP

Expose REST endpoint to accept collection of cm handles for GET operation (Passthrough only)

Jira

server	ONAP Jira
serverId	425b2b0a-557c-3c0c-b515-579789cceedb
key	CPS-1553

15 Days

3

DMI-Plugin

Expose endpint for ONAP not impl. and Stub impl. for testing/demo

Jira

server	ONAP Jira
serverId	425b2b0a-557c-3c0c-b515-579789cceedb
key	CPS-1555

5 Days

4

Stubbed DMI-Plugin

Include code to send response messages to internal kafka topic with delay

#

Component

Description

JIRAs

Estimate

1

DMI-Plugin

Accept datastore name as param into URL

Jira

server	ONAP Jira
serverId	425b2b0a-557c-3c0c-b515-579789cceedb
key	CPS-15501556

5 10 Days

25

NCMP

Expose REST endpoint to accept collection of cm handles for GET operation (Passthrough only)Forward response messages to client given kafka topic

Jira

server	ONAP Jira
serverId	425b2b0a-557c-3c0c-b515-579789cceedb
key	CPS-15531557

15 5 Days

6

NCMP

DMI-Plugin

Expose endpint for ONAP not impl. and Stub impl. for testing/demo

Handle non-existing cm handles

Jira

server	ONAP Jira
columnIds	issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columns	key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution

Jira

server


serverId	425b2b0a-557c-3c0c-b515-579789cceedb
key	CPS-

1583

5 Days

7

4

Stubbed DMI-Plugin

Include code to send response messages to internal kafka topic with delay

NCMP

Error handling for non-ready cm handle state

Jira

server	ONAP Jira
serverId	425b2b0a-557c-3c0c-b515-579789cceedb
key	CPS-

1614

10 5 Days

58

NCMP

Forward response messages to client given kafka topicHandle non responding DMI-Plugin

Jira

server	ONAP Jira
serverId	425b2b0a-557c-3c0c-b515-579789cceedb
key	CPS-15571558

5 Days

69

NCMP

NCMP: Update existing REST endpoint that accepts bulk request for GET operationHandle non-existing cm handles

Jira

server	ONAP Jira
columnIds	issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columns	key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId	425b2b0a-557c-3c0c-b515-579789cceedb
key	CPS-1583

5 Days

7

NCMP

Error handling for non-ready cm handle state

server	ONAP Jiraresolution
serverId	425b2b0a-557c-3c0c-b515-579789cceedb
key	CPS-16141535

5 DaysTBD

10

NCMP

MI-Plugin

Handle non responding DMI-Plugin : Update endpoint to accept bulk request

Jira

server	ONAP Jira
columnIds	issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columns	key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId	425b2b0a-557c-3c0c-b515-579789cceedb
key	CPS-

1536

5 DaysTBD

911

CSIT test for demo

Jira

server	ONAP Jira
serverId	425b2b0a-557c-3c0c-b515-579789cceedb
key	CPS-1560

5 Days

...

Space shortcuts

Page tree

Versions Compared

Old Version 77

New Version 78

Key

Implementation

Bulk Request Message Flow

Message Flow details

Implementation

Bulk Request Message Flow

Message Flow details