Page History

...

Jira
server ONAP Jira
columnIds issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId 425b2b0a-557c-3c0c-b515-579789cceedb
key CPS-1616

Jira
server ONAP Jira
columnIds issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId 425b2b0a-557c-3c0c-b515-579789cceedb
key CPS-1615

Jira
server ONAP Jira
columnIds issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId 425b2b0a-557c-3c0c-b515-579789cceedb
key CPS-1812

Jira
server ONAP Jira
columnIds issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId 425b2b0a-557c-3c0c-b515-579789cceedb
key CPS-1865

Impl. Proposal CM Event Subscription LCM: Create
Impl. Proposal CM Event Subscription LCM: Delete
for Merging of Subscriptions ( Positive Scenarios )

Requirements

Functional

	Interface	Requirement	Additional Information	Signoff
1	CPS-E-08e	NCMP will forward 'CM Notifications Subscription' LCM Create events to the appropriate DMI Plugins responsible for the CM Handles (ids) defined in the LCM Events. The message sent to DMI should include the private properties for each CM-Handle		24 Jul 2023
2	CPS-E-08e	NCMP will forward 'CM Notifications Subscription' LCM Delete events to the appropriate DMI Plugins responsible for the CM Handles is defined in the LCM Events. The message sent to DMI should include the private properties for each CM-Handle. A delete should only be forwarded if there is no other subscription for cmhandle datastore path.		13 Sep 2023
3	CPS-E-08e	NCMP responds using Kafka message(s) with the result of the LCM operation These messages will use agreed status fields: StatusCode (to be included in documentation) StatusMessage result errorInfo (NEW to be fleshed out)	Action: Define error details later
4	CPS-E-08e	All events should use Cloud Events format	https://cloudevents.io/	24 Jul 2023
5	CPS-E-08e	Messages should be processed in the order they are created (received)	It will depend on DMIs responding within 30s. Subscription-id should be used as Kafka 'key' to ensure this.	05 Sep 2023
6	CPS-E-08.e	NCMP is to merge CM Notification Subscriptions create request and forward those to DMI plugin. (Interface to be added).	Prevent unnecessary subscription updates to nodes already involved in a subscription to the same path and datastore. For possible combinations, see table below	30 Aug 2023
7	CPS-E-08.e	Last lights out: upon subscription Delete request only when there is no more subscription for a cm-handle & xpath & datastore combination a subscription-delete request will be sent to the relevant DMI(s).		30 Aug 2023
8	CPS-E-08.e	A single client subscription request should result into a maximum of one request per DMI. Of course there can be several messages if more than 1 DMI is involved.		30 Aug 2023
9	CPS-E-08.e	Amalgamate response should include rejected/accepted/pending DMI responses received within 30 seconds. A client shall be notified of available DMI subscription information after 30 seconds. Subsequent DMI subscription updates shall be notified to clients as they become available.	Same schema for all notifications. Subsequent notifications contains the state of all cmhandles involved in the subscription.	30 Aug 2023
10	CPS-NCMP-I-01	CM Handle deletion should NOT update subscription details.	Do NOT delete dmi-subscription entry until owning subscription is deleted, see issue #4 below	30 Aug 2023
11	CPS-E-08.e	~~Backward compatible with 'basic' created/delete operations..~~		30 Aug 2023

...

	Assumption	Notes
1	Targets is a collection of xpaths	can be implemented as a json array of strings
2	for the DMI plugin cm-handle-id, datastore and xpath are the key to a unique subscription	the client-id and subscription name are not relevant for the DMI (should not be included in the request) as this would break the concept of merging!

Issues & Decisions

	Issue	Notes	Decision
1	Is it intended that CM Notification subscription request cover (all) descendants of the given xpath too?!	e.g.. if a child is removed and there is a subscription for the parent node, will a notification be send (grandchild, child leaf updates etc.) I hope NOT! Consider: Risk of client effectively subscribing to ALL data in a cm handle by specifying top level datanode(s) Complexity (i.e. cost of) of merge operation. It might even required NCMP to check relevant dat model Future use of wildcards, could be a viable alternative for including descendants	24 Aug 2023
2	Could xpath point to an element that does not exist (yet)	if not how, how can I client be informed about a create event?	24 Aug 2023
3	Should NCMP support re-homing, moving of a CM Handle from one DMI to another?	assume only trough delete & create	24 Aug 2023
4	CM Handle Delete: Should DMI or Clients be sent a subscription update	do NOT delete dmi-subscription entry until owning subscription is deleted (just ignore upon future delete if cm handle is gone altogether) Note. LCM is already broadcast (today)	24 Aug 2023 Need to upgrade to CloudEevent format Add subscription id (name+client id) to header (so client can filter)
5	Validation of xpath	options order of implement and also performance cost! none xpath-parser model check instance check	24 Aug 2023
6	can DMI plugin 'reject' a subscription create (for a given cm-handle-xpath combination)	As NCMP might not validate as per issue#5 the DMI=plugin or component further down might have to reject an invalid xpath...	Priyank Maheshwari yes currently DMI can use response to say which cm handles are not accepted i.e. rejected' (but not 'pending') 24 Aug 2023
7	implementation question: should 'rejected' DMI-subscriptions be stored	not needed as whole subscription should be rejected	24 Aug 2023
8	Dimensioning of DB depends on #cm handles, #subscriptions and #xpaths per subscription, this could be too big for fast processing of updates!	Need to agree maximum and possibly realistic average/total number of entries based on the characteristics above The team is blocked until this becomes clear as it will affect the way the data needs to be modelled exactly.	13 Sep 2023 - kieran mccarthy- Dimensioning has been agreed.
9	Maximum (error) message size	theoretically all cm handles and all xpaths combinations could be rejected or pending leading to a very large error message!	24 Aug 2023
10	can each CM-Handle have different set of xpath(s) per subscription	the current 'basic' solution only supports a common set of datastore/xpaths (filter)	24 Aug 2023
11	can the same cm handle/xpath have different subscriptions with different datastores, does that affect the cm data notifications send (which datastore applies)	can register on any (passthrough) datastore running operational compulsory, validation needed part of KEY (unique entry) so cmHandle+datastore+xpath is the key	28 Aug 2023 Although this wiki-page will not be updated where cmHandle+xpath is mentioned as unique entry etc. this should now include a third field: datastore as well.
12	Will migration from 'basic' be supported	Preferred to ask customers to create new subscriptions	24 Aug 2023
13	list v list instances filtering	/p/c1 /p/c1[@key=23] Are these different paths or not (from ncmp point of view) *List instances might not always be supported, depending DMI Plugin impl.	24 Aug 2023
14	confirm subscription id (currently subscription name + client id)		05 Sep 2023
15	what subscription details should be send when there is a change (in the union)	just delta just union union and delta (delta flagged)	24 Aug 2023
16	one DMI rejects whole (see decision #6) subscription (affected cmhandles) but other DMI accepts the same subscription, is this possible how to handle		28 Aug 2023
17	What all datastores are supported ?	ncmp-datastore:passthrough-running or ncmp-datastore:passthrough-operational ?? or both?	13 Sep 2023 - kieran mccarthy Both will be supported
18	When NCMP sends subscription request to DMI and then DMI responds back to NCMP , do we need to have a correlationId to track/map the request and respone.
19	Which status codes to be used when the DMI Plugins respond back to NCMP ? We already have codes defined in NCMP. Do we reuse the same status codes when DMI Plugin responds back to NCMP with the subscription status ?

Solution Proposal

State handling for initial (basic)subscription create/delete use cases (not using concept of 'merging')

...

an operation on subscription 'A' for client id '10
an operation on subscription 'B' for client id '52'

#

Existing Subscription A-10

Client Create Subscription B-52 Request

DMI Sub Create Request

Data Model Before (2nd) Create

Data Model After Create

1

CH-1, [ /p/c1, /p/c2 ]

CH-1, [ /p/c1 ]

None

Expand

title	Data Model...

Client Subscription		DMI Subscription
name	client-id	cm-handle	xpath	subscribers
A	10	CH-1	/p/c1	[ A-10 ]
		CH-1	/p/c2	[ A-10 ]

Expand

title	Data Model...

Client Subscription		DMI Subscription
name	client-id	cm-handle	xpath	subscribers
A	10	CH-1	/p/c1	[ A-10, B-52 ]
B	52	CH-1	/p/c2	[ A-10 ]

2

CH-1, [ /p/c1, p/c2 ]

CH-1, [ /p/c2, /p/c3 ]

CH-1, [ /p/c2, /p/c3 ]

Expand

title	Data Model...

Client Subscription		DMI Subscription
name	client-id	cm-handle	xpath	subscribers
A	10	CH-1	/p/c1	[ A-10 ]
		CH-1	/p/c2	[ A-10 ]

Expand

title	Data Model...

Client Subscription		DMI Subscription
name	client-id	cm-handle	xpath	subscribers
A	10	CH-1	/p/c1	[ A-10 ]
B	52	CH-1	/p/c2	[ A-10, B-52 ]
		CH-1	/p/c3	[ B-52 ]

3

CH-1, [ /p/c1]
CH-2, [ /p/c1]

CH-2, [ /p/c1]
CH-3, [ /p/c1]

CH-3, [ /p/c1]

Expand

title	Data Model...

Client Subscription		DMI Subscription
name	client-id	cm-handle	xpath	subscribers
A	10	CH-1	/p/c1	[ A-10 ]
		CH-2	/p/c1	[ A-10 ]

Expand

title	Data Model...

Client Subscription		DMI Subscription
name	client-id	cm-handle	xpath	subscribers
A	10	CH-1	/p/c1	[ A-10 ]
B	52	CH-2	/p/c1	[ A-10, B-52 ]
		CH-3	/p/c1	[ B-52 ]

4

CH-1, [ /p/c1]

CH-1, [ /p/c1/gc1 ]

*as per decision 1 the xpath to a parent does NOT include any children...

Expand

title	Data Model...

Client Subscription		DMI Subscription
name	client-id	cm-handle	xpath	subscribers
A	10	CH-1	/p/c1	[ A-10 ]

Expand

title	Data Model...

Client Subscription		DMI Subscription
name	client-id	cm-handle	xpath	subscribers
A	10	CH-1	/p/c1	[ A-10 ]
B	52	CH-1	/p/c1/gc1	[ B-52 ]

Create 'Merge' Diagram

Below diagram shows and example for two subscriptions with party overlapping CM Handles and XPaths.

...

* Note 1: given the possible combinations the message to DMI needs to be able to specify different xpaths per cm-handle. So a more complex structure is needed for this even ie. an array of CM Handles objects each having their own list of (target) xpaths!
Note 2: as per decision #11'datastore' should be included in all messages as well

Delete Combinations

#

Existing Subscription A-10

Existing Subscription B-52

Client Delete A-10 Request

DMI Sub update/delete request

Data Model^* Before Before

Data Model^* After Delete

1

CH-1, [ /p/c1, p/c2 ]

CH-1, [ /p/c1 ]

CH-1, [ p/c2 ]

Expand

title	Data Model...

Client Subscription		DMI Subscription
name	client-id	cm-handle	xpath	subscribers
A	10	CH-1	/p/c1	[ A-10, B-52 ]
B	52	CH-1	/p/c2	[ A-10 ]

Expand

title	Data Model...

Client Subscription		DMI Subscription
name	client-id	cm-handle	xpath	subscribers
A	10	CH-1	/p/c1	[ A-10 ]
		CH-1	/p/c2	[ A-10 ]

2

CH-1, [ /p/c1]
CH-2, [ /p/c1]

CH-2, [ /p/c1]
CH-3, [ /p/c1]

CH-1, [ /p/c1]

Expand

title	Data Model...

Client Subscription		DMI Subscription
name	client-id	cm-handle	xpath	subscribers
A	10	CH-1	/p/c1	[ A-10 ]
B	52	CH-2	/p/c1	[ A-10, B-52]
		CH-3	/p/c1	[ B-52 ]

Expand

title	Data Model...

Client Subscription		DMI Subscription
name	client-id	cm-handle	xpath	subscribers
B	52	CH-2	/p/c1	[ B-52 ]
		CH-3	/p/c1	[ B-52 ]

3

CH-1, [ /p/c1]

CH-1, [ /p/c1]
CH-2, [ /p/c1]

None

Expand

title	Data Model...

Client Subscription		DMI Subscription
name	client-id	cm-handle	xpath	subscribers
A	10	CH-1	/p/c1	[ A-10 ]
B	52	CH-1	/p/c2	[ A-10, B-52 ]

Expand

title	Data Model...

Client Subscription		DMI Subscription
name	client-id	cm-handle	xpath	subscribers
B	52	CH-2	/p/c1	[ B-52 ]

* Note: as per decision #11'datastore' should be included in the 'DMI susbcription' data model too now.

Error-Upon-Error Combinations

	Previous Interaction	Current Interaction	Expectation	Notes
1	any operation on rejected for non-existing cm-handle	operation for same non-existing cm-handle	listed in 'rejected' immediately	behavior as normal
2	create operation rejected by DMI	create for same cm-handle/xpath	try again ?!
3	create pending	create for same cm-handle/xpath	TBD send again or just remain pending?!
4	create pending	delete for same cm-handle/xpath
5	delete pending	delete for same cm-handle/xpath
6	delete pending	create for same cm-handle/xpath

Client-Schema Update

Based on Issue#10 , the incoming schema from DME to NCMP should

...

We have the below configuration managed by CPS-NCMP

# cmhandle dmi-plugin
1 ch-1 dmi-1
2 ch-2 dmi-1
3 ch-3 dmi-2
4 ch-4 dmi-2
5 ch-5 non-responding-dmi

We get a Subscription Create Request with unique subscription-id and a list of predicates(list of targets , datastore and list of filters)

Code Block

title	Subscription Create Message
collapse	true

{
"subscriptionId" : "A-10",
"predicates" : [
  {
    "targets": [ch-1,ch-2],
    "datastore" : "ncmp-datastore:passthrough-operational",
    "datastore-xpath-filter" : ["p1/c1","p2/c2"]
    },
  { "targets": [ch-3],
    "datastore" : "ncmp-datastore:passthrough-operational",
    "datastore-xpath-filter" : ["p3/c3"]
    }
]
 
}

We persist the incoming request as is in our persistent store using a relevant subscription model ( this model also needs to be revised )
We also maintain a distributed datastructure in Hazelcast to keep track of the request and response to the client.

SubscriptionId dmi-plugin affectedCmHandles status
A-10 dmi-1 ["ch-1","ch-2"] PENDING
A-10 dmi-2 ["ch-3"] PENDING
Read the requests from the distributed data structure and form the request for the DMI-Plugin and publish it to the internal kafka channel.
If the DMI-Plugin responds back within the configured time , so the request will either be ACCEPTED or REJECTED.
1. if whole subscription request is ACCEPTED , update the distributed data structure and then store the subscription to the ActiveSubscriptions in cps-cache.
2. if whole subscription request is REJECTED , update the distributed data structure and send back the response to the client.
3. if the DMI Plugin fails to respond then the status would remain PENDING only as there is no change.
  
  SubscriptionId dmi-plugin affectedCmHandles status
  A-10 dmi-1 ["ch-1","ch-2"] ACCEPTED
  A-10 dmi-2 ["ch-3"] REJECTED
  
  If there are no more PENDING requests , then we can right away send the response without updating the cache.

We store the "ActiveSubscriptions" using a new model which will look like below.

SubscriptionId	cmHandle	filter	datastore
A-10	ch-1	p1/c1	ncmp-datastore:passthrough-operational
A-10	ch-1	p2/c2	ncmp-datastore:passthrough-operational
A-10	ch-2	p1/c1	ncmp-datastore:passthrough-operational
A-10	ch-2	p2/c2	ncmp-datastore:passthrough-operational

** More on the storage structure of this.

Now based on whatever we have in the distributed map , we will send the response to the clients and may be clear the in-memory structure if required.

...

We get a new subscription create request.

Code Block

language	xml
title	2nd Subscription Create Message
collapse	true

{
"subscriptionId" : "B-52",
"predicates" : [
  {
    "targets": [ch-2],
    "datastore" : "ncmp-datastore:passthrough-operational",
    "datastore-xpath-filter" : ["p2/c2" , "p3/c3"]
    }
]
}

We now look at the "ActiveSubscriptions" if we already have an active subscription either for a combination of "cmhandle and xpath filter and the datastore".

Current view of ActiveSubscriptions

SubscriptionId	cmHandle	filter	datastore
A-10	ch-1	p1/c1	ncmp-datastore:passthrough-operational
A-10	ch-1	p2/c2	ncmp-datastore:passthrough-operational
A-10	ch-2	p1/c1	ncmp-datastore:passthrough-operational
A-10	ch-2	p2/c2	ncmp-datastore:passthrough-operational

Check if we have ch-2 , p2/c2 and ncmp-datastore:passthrough-operational then directly add to the ActiveSubscription.Similarly check for ch-3 , p3/c3 and ncmp-datastore:passthrough-operational ( which in this case will not be directly to the active subscription)

Now the state of ActiveSubscription is :

SubscriptionId	cmHandle	filter	datastore
A-10	ch-1	p1/c1	ncmp-datastore:passthrough-operational
A-10	ch-1	p2/c2	ncmp-datastore:passthrough-operational
A-10	ch-2	p1/c1	ncmp-datastore:passthrough-operational
A-10	ch-2	p2/c2	ncmp-datastore:passthrough-operational
B-52	ch-2	p2/c2	ncmp-datastore:passthrough-operational

and state of the in-memory data structure is as follows.

SubscriptionId	dmi-plugin	affectedCmHandles	status
B-52	dmi-1	["ch-2"]	PENDING

Now we send the request to DMI plugin (dmi-1) with just the private properties of the ch-2 with the delta xpath.
1. DMI Plugin could either apply the whole subscription request or reject the whole subscription. i.e respond with accepted/rejected
Based on the response from DMI plugin the in-memory structure is updated and response is sent back to the clients.

if DMI says it is accepted then :
SubscriptionId dmi-plugin affectedCmHandles status
B-52 dmi-1 ["ch-2"] ACCEPTED

if DMI says it is rejected then :
SubscriptionId dmi-plugin affectedCmHandles status
B-52 dmi-1 ["ch-2"] REJECTED

if we get no response within the configured time then :

SubscriptionId dmi-plugin affectedCmHandles status
B-52 dmi-1 ["ch-2"] PENDING

But we still send some response within the pre-configured time.

if the above request was accepted then we should add it to the list of "ActiveSubscriptions" as well.

SubscriptionId	cmHandle	filter	datastore
A-10	ch-1	p1/c1	ncmp-datastore:passthrough-operational
A-10	ch-1	p2/c2	ncmp-datastore:passthrough-operational
A-10	ch-2	p1/c1	ncmp-datastore:passthrough-operational
A-10	ch-2	p2/c2	ncmp-datastore:passthrough-operational
B-52	ch-2	p2/c2	ncmp-datastore:passthrough-operational
B-52	ch-2	p3/c3	ncmp-datastore:passthrough-operational

If the status was accepted/rejected then after sending the response to the client we can clear the in-memory structure for subscriptionId B-52.

...

Operation Specific Impl. Proposals

Create

Impl. Proposal CM Event Subscription LCM: Createfor Merging of Subscriptions ( Positive Scenarios )

Delete

Impl. Proposal CM Event Subscription LCM: Delete // to be deprecated

Space shortcuts

Page tree

Versions Compared

Old Version 39

New Version 40

Key

Requirements

Functional

Issues & Decisions

Solution Proposal

Create 'Merge' Diagram

Delete Combinations

Error-Upon-Error Combinations

Client-Schema Update

Operation Specific Impl. Proposals

Create

Delete