...
# | Issue | Notes | Decision |
---|---|---|---|
1 | How fast should CPS (and DB) be able to process max heart beat failures? | is 60K really realistic if ENM goes down we should get a notification for each node do we ?! | PoC has shown 60 seconds is reasonable |
2 | Restart of NCMP | Should/Can this be handled? | As of now, there is no such case is being considered. |
3 | Does DMI Plugin provide NCMP with a health check URL during registration? Either, just rely on the default one provided with Spring boot actuator? | Document the contract. Its just the interface that matters and not the implementation. | Spring boot actuator interface |
4 | Error during cmHandle registration | If an error occurs during registration what trustlevel should the cmHandle be set to? IN eth following scenarios
| Agreed to Leave as is, if notification for a node already registered, we can process the other notification separately Team Notes: [Team] When state was provided to 'COMPLETE' or 'NONE' and the registration fails , state if trust level is still set to the provided state regardless of the current state of the cm handle (deleted/deleting, advise, ready, locked) |
5 | Module sync watchdog issues/error scenarios | If cmHandle is set to none/incomplete module sync will automatically retry (Is this acceptable?) If the module sync fails we will still send a Complete message (Is this acceptable?) Registering all cmHandles could take up to 20 mins, what should happen if the last sync fails as the notification would have been sent 20 mins ago? | When CMLevel is in: DELETING/DELETED - No Truslevel notification update ADVISE - No trustLevel notification update READY - Truslevel notification update LOCKED -Truslevel notification update
Team Notes: 12/10/2023 [Team] Notification SHALL only be sent when the Cm handle is set to Ready and locked regardless of the report from DMI Do we still update the cache? Yes. |
6 | When cm handle trustLevel state stays the same | Do we include that cm handle ID or not for notifications? | No you don't if no changes if it stays the the same
Team Notes: 12/10/2023 Scenario: DMI plugin up/down the previous state of the cm handle (trsutLevel) should be considered for notifications |
Description
- Define scenarios which cause a CM Handle to go stale.
- Implement changes to support tracking of CM Handle Freshness/Staleness.
...
- dmi plugin identifies that the device is no longer contactable.
- dmi plugin identifies that an underlying device manager managing the device (node) is out of sync with the device itself.
Requirements
Functional
# | Interface | Requirement | Additional Information | Sign-off |
---|---|---|---|---|
1 | CPS-NCMP-E-05 | The 'trustlevel' is visible on all REST methods that currently include the 'cm handle state' | existing endpoints |
|
2 | CPS-NCMP-E-05 | CM Handles can be queried (filter condition) on 'trustlevel' | using a new 'trustLevel' condition (cannot use cpsPath condition) |
|
3 | CPS-NCMP-I-01 | During registration, DMI plugin can report initial trustlevel. If the state is not 'complete', it should be considered as 'Trustlevel change' (See req 5) | Initial trust level will be backward compatible if not set, we assume trustlevel is 'complete' For a new cm-handle where the trustlevel is 'complete' this is NOT considered a chance and no notifications should be sent |
|
4 | CPS-NCMP-E-05 | Once DMI (plugin) is detected to be down the trust-level for all affected CM Handles should be set to be 'NONE'. This wil also lead to many notifcations as per req. #5 | this might lead to a high level (20K) of notifications (need to discuss capabilities) |
|
5 | CPS-NCMP-E-05.e | NCMP notification shall be sent when the trustlevel changes | Notification be sent externally based on Kafka many small or bulk: Agreed Many notifications, one for each cm-handle |
|
6 | CPS-NCMP-I-01.e | It shall be possible to report any trustlevel of one CM Handle DMI plugin can report the current trustLevel of a single cm handle id | i.e. the DMI can tell NCMP the trustLevel is 'NONE' when a node heartbeat failure is detected and 'COMPLETE' once it is restored. Again this should lead to notifications on the external interface as per req #5 |
|
Error Handling
# | Error Scenario | Expected behavior | Sign-off |
---|---|---|---|
1 | NCMP restart (all instances) | To be discussed, not sure if it can/should be handled TrustLevels should be 'NONE' and need to be restored using an audit-request (not in scope) | If we restart, it should go into COMPLETE STATE. No way of getting out of NONE State Audit was agreed to be handled in a separate epic - Prioritise audit epic
Team Notes: 12/10/2023 **If all instances of NCMP restarts [fresh start], there would be nothing in the cache |
Characteristics
# | Parameter | Expectation | Notes | Sign-off |
---|---|---|---|---|
1 | dmi-down detection speed | 60 seconds | It's a configurable value. Agreed - Should be in parallel with device heartbeat. |
|
2 | device heartbeat frequency (message emitted by DMI plugin for each device) | 60 seconds | Can be removed - out of scope for this epic | |
3 | maximum supported devices (by NCMP) | 60,0000 | Given #2 and #3 this means NCMP needs to process 60,000 message / minute! - Can be removed, separate epic - out of scope for this epic | |
4 | maximum number of cm-handles down report by DMI in one request and/or per minute | 30,000 / minute | a peak can be processed within 60 seconds |
|
5 | processing of all trustLevel time for DMI-Down and/or peak load by DMI | 1 second | Agreed to go with 30,000 / minute as no 4 |
|
6 | If we incorporate into searches endpoints the speed should not be impacted | 30 seconds | Speed shouldn't be affected - Agreed - It's across 60,0000 cmHandle Open for improvement in respect to performance |
|
...
Proposal for Notification's Schema
12/10/2023 - awaiting for approval
(Reference CPS-1104 Agree LCM Event schemas)
kafka-key : cmHandleId/deviceId
Code Block | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
id : "random UUID" source : "NCMP" specversion : "1.0" type : "deviceTrustLevelNotification" time : "notificationTime" dataschema : "org.onap.cps.ncmp.events.trustlevel.DeviceTrustLevelNotification:1.0.0" correlationid : "deviceId+trustLevel" "data": { "deviceId": "CmHandle1", // or we can call it cmHandleId as well [mandatory] "oldValues": { // optional "trustLevel" : "NONE/PARTIAL/COMPLETE", "deviceId": "CMHandle1"" }, "newValues" : { // optional "trustLevel" : "COMPLETE" } } |