z
| Jira | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
| Table of Contents |
|---|
Requirements
- support 'basic cm notification subscription requirements define before: CM Event Subscription LCM
Functional
...
Requirement
...
- Prevent unnecessary subscription updates to nodes already involved in a subscription to the same path and datastore.
- For possible combinations, see table below
...
kieran mccarthy Signed off .
...
Last lights out: upon subscription Delete request only when there is no more subscription for a cm-handle & xpath & datastore combination a subscription-delete request will be sent to the relevant DMI(s).
...
...
...
- Same schema for all notifications.
- Subsequent notifications contains the state of all cmhandles involved in the subscription.
...
...
...
...
Error Handling
...
An error scenario on a second subscription for the same cm-handle/xpath as a previous subscription which did not complete successfully (yet)
...
Characteristics
...
kieran mccarthy confirmed
...
kieran mccarthy to confirm
Note: consider limit and future 'simple' ALL (wildcard) option when above that limit
...
...
kieran mccarthy order of 30 seconds (NCMP side)
Out-of-scope
- CM Notification Forwarding Check: When forwarding CM Notification NCMP will not check the content to see if the is a valid active subscription. It is assumed that the DMI Plugin as acted on the 'delete subscription' request (that request is NCMPs responsibility). And of course there wil be timing issues it also possible a CM Notification was send just after the subscription-delete was send (from client) but before the whole change had acted upon that.
- Retry: NCMP will only report when actions are pending or rejected. NCMP will not implement a retry mechanism
- Wildcards: Wildcards or similar functionality where one string represent 0 or more xpaths is not covered as part of this requirement but it should be kept in mind as a future possibility
- Dynamic Topic: Topic for CM Data Notifications back to client will be hardcode for now.
Study shoudl consider compatibility with an 'ALL' cm handles options
Assumptions
...
Issues & Decisions
...
- Risk of client effectively subscribing to ALL data in a cm handle by specifying top level datanode(s)
- Complexity (i.e. cost of) of merge operation. It might even required NCMP to check relevant dat model
- Future use of wildcards, could be a viable alternative for including descendants
...
kieran mccarthy descendant not covered by 'basic' paths Wildcard will cover this kind of function in future
...
kieran mccarthy yes, xpath can point to things that don't exist yet (not even in the curernt model when an upgraded is pending)
...
kieran mccarthy out of scope
...
do NOT delete dmi-subscription entry until owning subscription is deleted
(just ignore upon future delete if cm handle is gone altogether)
Note. LCM is already broadcast (today)
...
- kieran mccarthy re-use existing LCM event → also send it on topic for he subscription (currently fixed topic)
- Need to upgrade to CloudEevent format
- Add subscription id (name+client id) to header (so client can filter)
...
- none
- xpath-parser
- model check
- instance check
...
kieran mccarthy not required right now
...
yes currently DMI can use response to say which cm handles are not accepted i.e. rejected' (but not 'pending')
kieran mccarthyconfirms DMI can reject whole subscription
...
kieran mccarthyconfirms
...
Toine Siebelink no longer relevant given and max decided
...
kieran mccarthy Yes, clients will create only one subscription and they might have different needs for different nodes (cm handles)
...
kieran mccarthy datastore included.
Although this wiki-page will not be updated where cmHandle+xpath is mentioned as unique entry etc. this should now include a third field: datastore as well.
...
kieran mccarthy no need to support (migration) of 'basic' solution. Development of 'basic' solution can be stopped!
...
/p/c1
/p/c1[@key=23]
Are these different paths or not (from ncmp point of view)
*List instances might not always be supported, depending DMI Plugin impl.
...
kieran mccarthy yes, from NCMP point of view a list and a list entry are different xpaths
...
kieran mccarthy to clarify ASAP including exact format
all prefer a single id-field
...
- just delta
- just union
- union and delta (delta flagged)
...
(after initial meeting) kieran mccarthy & Toine Siebelink agreed that because of the possible size of the union (200x200x10=400,000) it is only feasible to send the delta ie option 1.
...
kieran mccarthy only sends message back to client about rejected DMIs i.e. the subscription can be partially 'active'
Solution Proposals
Current state handling for 'basic' (not merged) subscription create/delete (under development)
...
Scenario 1
1) Create Sub from Client: CH-1, CH-2
2) All handled by DM1
3) NCMP to DMI request Create sub: CH-1, CH-2
DMI responds within 30 seconds: Status OK
NCMP Send message to client: Statsu OK (no list with Rejected/Pending)
...
NCMP Send messge to client: Pending [CH-1, CH-2]
Scenario 3
1) Create Sub from Client: CH-1, CH-2
2) All handled by DM1
3) NCMP to DMI request Create sub: CH-1, CH-2
DMI responds within 30 seconds: Rejected [CH-2]
NCMP Send message to client: Rejected [CH-2]
Scenario 4:
4 different DMIs each handling 2 Cm Handles, create sub for all CHs
Scenario 4-part a
DMI1 CH-1, CH-2 All accepted within 30 secs
DMI2 CH-3, CH-4 Mixed result within 30 seconds e.g. Rejected [CH-4]
DMI3 CH-5, CH-6 No result after 30 seconds
DMI4 CH-7, CH-8 No result after 30 seconds
NCMP -> Client : Rejected [CH-4], Pending [ CH-5, CH-6, CH-7, CH-8 ]
4-part b
DMI3 PLugin responds after 40 seconds; status OK
NCMP -> Client : Rejected [CH-4], Pending [ CH-7, CH-8 ]
4-part c
DMI4 PLugin responds after 50 seconds; Rejected [CH-7]
NCMP -> Client : Rejected [CH-4, CH-7]
Note. The above algorithm depends ons storing (in DB using yang modelled data) a cm-handle status for each cm-handle for each subscription!
Merge → Split
- NCMP treats a Client Subscription as one or more (small) DMI Subscriptions, each of which wil have there own state.
Each DMI Subscription is related to 1 or more client subscriptions.
If there is no more related client subscription the DMI Subscription can be deleted (once accepted by DMI Plugin)!
Data Model (not a great diagram...)
...
Create Combinations
Below examples demonstrate what should happen when two separate subscriptions operations are performed:
- an operation on subscription 'A' for client id '10
- an operation on subscription 'B' for client id '52'
...
| title | Data Model... |
|---|
...
| title | Data Model... |
|---|
...
| title | Data Model... |
|---|
...
| title | Data Model... |
|---|
...
| title | Data Model... |
|---|
...
| title | Data Model... |
|---|
...
CH-1, [ /p/c1/gc1 ]
*as per decision 1 the xpath to a parent does NOT include any children...
...
| title | Data Model... |
|---|
...
| title | Data Model... |
|---|
...
Create 'Merge' Diagram
Below diagram shows and example for two subscriptions with party overlapping CM Handles and XPaths.
| Gliffy Diagram | ||||||||
|---|---|---|---|---|---|---|---|---|
|
* Note 1: given the possible combinations the message to DMI needs to be able to specify different xpaths per cm-handle. So a more complex structure is needed for this even ie. an array of CM Handles objects each having their own list of (target) xpaths!
Note 2: as per decision #11'datastore' should be included in all messages as well
Delete Combinations
...
| title | Data Model... |
|---|
...
| title | Data Model... |
|---|
...
CH-1, [ /p/c1]
...
| title | Data Model... |
|---|
...
| title | Data Model... |
|---|
...
| title | Data Model... |
|---|
...
| title | Data Model... |
|---|
...
* Note: as per decision #11'datastore' should be included in the 'DMI susbcription' data model too now.
Error-Upon-Error Combinations
...
Client-Schema Update
Based on Issue#10 , we need to change the incoming schema from DME to NCMP.
- A single subscription contains multiple predicates
- Predicates should be an array of Predicates ( currently a single instance )
- Each predicate contains an array of targets ( cmhandles) - as is.
- Each predicate contains a single datastore (mandatory)
- datastore-xpath-filter should become an array of xpath-filters (instead of pipe separated single string)
Performance considerations
Below options (partly) explored dring CPS teem meeting could explore further once we know the characteristics, see issue #8 !
- DMI record could reach in millions of instances
- In memory solution (ie hazelcast) can handle 10s - 100s of Megabytes (add some sample calc)
- still need persistence if we don't have enough instances
- use many small messages reduce need for amalgamation
- amalgamated string-based solution....
Solution Proposal - Storage & Merging
Storage
...
| Code Block | ||
|---|---|---|
| ||
{
"subscriptionId" : "A-10",
"predicates" : [
{
"targets": [ch-1,ch-2],
"datastore" : "ncmp-datastore:passthrough-operational",
"datastore-xpath-filter" : ["p1/c1","p2/c2"]
},
{ "targets": [ch-3],
"datastore" : "ncmp-datastore:passthrough-operational",
"datastore-xpath-filter" : ["p3/c3"]
}
]
} |
...
Merging ( in-progress )
...
| Code Block |
|---|
{
"subscriptionId" : "B-52",
"predicates" : [
{
"targets": [ch-2],
"datastore" : "ncmp-datastore:passthrough-operational",
"datastore-xpath-filter" : ["p2/c2" , "p3/c3"]
}
]
} |
...
- DMI Plugin could either apply the whole subscription request or reject the whole subscription. i.e respond with accepted/rejected
...
Proposed JIRAs
...
This study is now included in CM Data Notification Subscription LCM (incl. merge)