...
- DMI record could reach in millions of instances
- In memory solution (ie hazelcast) can handle 10s - 100s of Megabytes (add some sample calc)
- still need persistence if we don't have enough instances
- use many small messages reduce need for amalgamation
- amalgamated string-based solution....
Storage Solutions
Proposed Data Storage
Use a combination of
- Database (yang modelled) for permanent subscription details, subscribers, predicates
- In-Memory (hazelcast) for transient subscription details: state (acceptep, rejected, pending)
Example Flow 1: First Subscription
...
We get a Subscription Create Request with unique subscription-id and a list of predicates(list of targets , datastore and list of filters)
| Code Block | ||||
|---|---|---|---|---|
| ||||
{
"subscriptionId" : "A-10",
"predicates" : [
{
"targets": [ch-1,ch-2],
"datastore" : "ncmp-datastore:passthrough-operational",
"datastore-xpath-filter" : ["p1/c1","p2/c2"]
},
{ "targets": [ch-3],
"datastore" : "ncmp-datastore:passthrough-operational",
"datastore-xpath-filter" : ["p3/c3"]
}
]
} |
...
Example Flow 2: Second Subscription (merge)
We get a new subscription create request.
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
{
"subscriptionId" : "B-52",
"predicates" : [
{
"targets": [ch-2],
"datastore" : "ncmp-datastore:passthrough-operational",
"datastore-xpath-filter" : ["p2/c2" , "p3/c3"]
}
]
} |
...
- DMI Plugin could either apply the whole subscription request or reject the whole subscription. i.e respond with accepted/rejected
...
Kafka Topics
| # | Topic Name | Environment Variable | Source | Destination | Notes |
|---|---|---|---|---|---|
| 1 | subscription | NCMP_CM_AVC_SUBSCRIPTION | DME | DME/Client-Apps | |
| 2 | ncmp-dmi-cm-avc-subscription-<dmi-name> | NCMP_FORWARD_CM_AVC_SUBSCRIPTION | NCMP | DMI Plugin | |
| 3 | dmi-ncmp-cm-avc-subscription | NCMP_RESPONSE_CM_AVC_SUBSCRIPTION | DMI Plugin | NCMP | |
| 4 | subscription-response | NCMP_OUTCOME_CM_AVC_SUBSCRIPTION | NCMP | DME/Client-Apps | |
| 5 | dmi-cm-events | DMI_CM_EVENTS_TOPIC | DMI Plugin | NCMP | actual data notifications as per the subscriptions |
| 6 | cm-events | NCMP_CM_EVENTS_TOPIC | NCMP | DME/Client-Apps | NCMP to forward the notifications to the client-apps. |
...