You are viewing an old version of this page. View the current version .
Compare with Current
View Page History
« Previous
Version 112
Next »
Overview
CPS-1628Getting issue details...
STATUS
This page gives a description of CPS events including their fields Events will be split into events and Kafka headers # Description Notes Decision 1 No Event properties defined for DMI AVC Event Priyank Maheshwari will need to specify and agreed event structure for DMI AVC Event with stakeholders ie. provide Jira ticket
Event Body should be compatible with RFC8641
kieran mccarthy
Priyank Maheshwari created JIRA to create the event body schema.
2 Bulk Operation events details have not yet be defined (just headers) Sourabh Sourabh to provide Jira tickets
CPS-1658Getting issue details...
STATUS
3 Should all the events have same kafka headers kieran mccarthy 26 Apr 2023 Possibly Common (base) set of headers but mandatory aspect might differ. In practice we might need a separate headers (schema?) for each event
4 Clarify the format of the version eventSchemaVersion v1 or 1.0
EX: 1.0.0 (without 'v')
kieran mccarthy to check ORAN preference
kieran mccarthy confirms through email on 12 May 2023 to use semantic versioning which ORAN follows https://semver.org .
5 What to do with additional event headers (from DMI Plugins) kieran mccarthy 26 Apr 2023 if DMI produce additional headers NCMP will discard those i.e. not included in forwarded events
6 Event(Content) field in DMI Async Request Response Event has inconsistent name (compared with other schemas) Add V2 file Deprecate V1 Support both versions for a while Delete the V1 version (after some time ) CPS Team 27 Apr 2023 Create a V2 of the schema and rename eventContent as event data. Do it as part of the schema addition.
7 NCMP Async Request Response Event (#5) contains both an Event and ForwardedEvent ForwardedEvent is not wrapped inside Event but question now is if we need 2 events at all?!
Sourabh Sourabh and Raviteja Karumuri can check how it is actually working and then we decide ( create a JIRA ticket )
Wiki for the Study on NCMPAsyncRequestResponse event schema
Conclusion: events not designed as proposed, very inconsistent. Never go a bug because these async events aren't used at all (confirmed by Csaba Koscis) Instead bulk request wil be used for topology use cases.
10 May 2023 kieran mccarthy and team agreed to:
Temporary disable the legacy async request feature (task created:
CPS-1694Getting issue details...
STATUS
) As part of a lower priority work items (but during Montreal) fix related events with learnings from the new batch-usecase. (task created
CPS-1693Getting issue details...
STATUS
) 8 Dmi Data AVC Event, use of eventSource field Priyank Maheshwari wanted to store 'datastore' in this field but kieran mccarthy explained it to use for different purposes
kieran mccarthy Clients can use this field as per their requirements.
9 Can Kafka Headers be described with 'schema's owned and managed by NCMP POC to follow.
CPS-1657Getting issue details...
STATUS
Defining header schema.
Integration of header with kafka.
Naming and versioning convention for the header schemas. 'id'
Does the headers schema have a version too?!
03 May 2023 Priyank Maheshwari confirmed headers can be described with a separate schema.
Both header schema's and event schemas will be published on https://docs.onap.org/projects/onap-cps/en/latest/cps-events.html
Header schema name and version will be maintained in the 'id' metadata field of header's schema .
10 Depending #10 can schema inherit/extend a common schema for common headers Commonly define them and then define what are mandatory(required) or optional as per the schema extending it.
If a field is not used in the extended schema then it should be able to handle it.
Extend the POC ( on #9 ) to cover this.
09 May 2023 Priyank Maheshwari did the POC and the conclusion of that was that :-
One schema can extend the other schema. We cannot override the mandatory/optional parameter from the Parent schema. 09 May 2023 Toine Siebelink agrees to go ahead with separate schema/headers per event. There will be some duplication but it will have its advantage when versioning.
11 Is anyone using Async Request feature? See
CPS-1660Getting issue details...
STATUS
we need to re-design event #4 (Covers Point 6 as well) and #5 in a backward incompatible way. If no-one is using this feature right now (suspected) we can do this more easily/cheaper!
03 May 2023 Csaba Kocsis confirmed this is not used by Ericsson currently. No plans to use soon for single-cmhandle requests either (TBC). Need to decide priority (Csaba Kocsis to find out of fixing the legacy schema(s)
12 Do we need additionalProperties for DMI ASync Data Request respondes (events #4, #5) The original code populates a framework defined 'additionalProperties' field with a singel key-value pair: "response-data",{<json data>}. No other (private) properties are added either in DMI PLugin or NCMP code. The name is just coincidence and misleading. In fact this 'additionalProperties' field should NOT have been used at all! No, the new schema should NOT add 'additionalProperties' field at all use 'additionalProperties:no' in the schema 13 AVC Subscription Event (DMI → NCMP) (events #3) Want to understand what 'data' is datatype referring to under Subscription Event?What value comes under 'schemaName' & 'SchemaVersion' of Datatype definition under AVC Subscription Event? Reconfirmation needed on 'schemaName' & 'SchemaVersion' should be in the payload? In meeting26 Apr 2023 kieran mccarthy updated #3 is ON HOLD to analyse further.
Agreed with Toine Siebelink on18 May 2023 that Priyank Maheshwari will look into this from now as they are working on something related to this.
14 Align headers with CNCF Cloud Events Using standard headers as defined by Cloud Events and possibly common header extension See Table below, CNFC Cloud Event alignment Cloud Events 3PP for all current and legacy events to ensure common format 23 May 2023 kieran mccarthy and Toine Siebelink agreed on general idea but exact list of common headers need to be agreed
CPS-1717Getting issue details...
STATUS
15 During the meeting we saw that the Kafka header fields were prefixed with "ce_ " so need to check if we are ok with that.
16 Do will still need/ can we stil use schemas for header details with CNCF library ?! How to publish info about non-standard headers like correlationid ? 17 inconsistent casing convention for header v. json data Just observing. All header ion CNCF are lowercase whereas json field are camelCase. Don't want to change but want to make sure agree... 18 Confirm 'source'. is to be added to ALL events (declared mandatory for CNFC events)
#1 LCM Event (NCMP → Client Apps) Description
LCM Event Details
Current LCM Event Object Field Type Description Kafka Header Event Payload Field Notes CloudEvents Mapping eventID string The unique id identifying the event x Mandatory standard ( id ) eventCorrelationId string The id identifying the event. In LCMEvent this is always a cmhandleId x Optional extension ( correlationid ) eventTime string The timestamp when original event occurred x Mandatory standard ( time ) eventSource string The source of the event x Mandatory standard ( source ) eventType string The type of the event x Mandatory standard ( type ) eventSchema string The schema that this event adheres to. x Mandatorycps:org.onap.ncmp.cmhandle.lcm-event:1.0.0
standard (dataschema) eventSchemaVersion string The version of the schema that this event adheres to
x Mandatory
e.g v1.0
N/A included in 'dataschema' event Event The payload of an event Mandatory standard ( data ) N/A N/A The version of CNCF Specification Specified by CNCF (value hardcoded) standard ( specversion )
Type:Event (cps:org.onap.ncmp.cmhandle.lcm-event:v1) Field Type Description Kafka Header Event Payload Field Notes cmHandleId string cmHandle id x oldValues Values Values that represents the state of a cmHandle
x Defined by values object below newValues Values Values that represents the state of a cmHandle x Defined by values object below
Type:Values Field Type Description Kafka Header Event Payload Field Notes dataSyncEnabled string cmHandle id x cmHandleState string State of cmHandle
x Enum: ["ADVISED", "READY", "LOCKED", "DELETING", "DELETED"]
cmHandleProperties object cmHandle properties x List of Java <String, String> Maps
#2 DMI Data AVC Event (ONAP DMI → NCMP) Description A normalized AVC Event that ONAP DMI Plugin will send to NCMP. NCMP can process the event and update cached data if needed. See 5 in CPS Data Notifications Overview#ComponentDiagram eventSchema ' Depending on the AVC subscription details those events might or might not be forwarded to the Client Apps.
DMI Data AVC Event
AVC Event Object Field Type Description Kafka Header Event Payload Field Notes CloudEvents Mapping eventID string The unique id identifying the event x Mandatory standard (id) eventCorrelationId string The id identifying the event x Optional extension(correlationid) eventTime string The timestamp when original event occurred x Optional standard (time) eventSource string The source of the event x Optional standard (source) eventType string The type of the event x Mandatory standard (type) eventSchema string The schema that this event adheres to x Mandatory (e.g. cps:org.onap.cps.ncmp.events:avc-event-schema) standard (dataschema) eventSchemaVersion string The version of the schema that this event adheres to
x Mandatory N/A included in 'dataschema' event Event The payload of an event Mandatory standard (data)
Type: Event (cps:org.onap.cps.ncmp.events:avc-event-schema:v1) Field Type Description Kafka Header Event Payload Field Notes
#3 AVC Subscription Event (DME → NCMP) Description AVC Subscription Event (External Clients Apps → NCMP) : ON HOLD - kieran mccarthy to analyze further
Priyank Maheshwari We need to revisit this schema and include the cloud events fields. ( At the moment this doesn't have proper fields )
AVC Subscription Event
Subscription Event Object
Field Type Description Kafka Header Event Payload Field Notes CloudEvents Mapping version string The event type version X ekiemcc : included in dataschema eventType string The event type
X ekiemcc :
Enum:
["CREATE"]
standard ( type ) source NEW TBC, see issue #18 event Event The payload for an event X Defined by event object below standard ( data )
Event Object (cps:org.onap.cps.ncmp.events:avc-subscription-event:v1) Field Type Description Kafka Header Event Payload Field Notes subscription Subscription The subscription details x Defined by subscription object below dataType DataType The datatype content
x Defined by dataType object below predicates Predicates Additional values to be added into the subscription
x Defined by predicates object below
Type: Subscription Field Type Description Kafka Header Event Payload Field Notes clientID string The client ID
x name string The name of the subscription
x isTagged boolean optional parameter, default is false
x default: false
Type: DataType Field Type Description Kafka Header Event Payload Field Notes dataspace string The dataspace name
x dataCategory string The category type of the data
x dataProvider string The provider name of the data
x schemaName string The name of the schema
x schemaVersion string The version of the schema
x
Type: Predicates Field Type Description Kafka Header Event Payload Field Notes targets array CM Handles to be targeted by the subscription
x datastore string datastore which is to be used by the subscription
x xpath-filter string filter to be applied to the CM Handles through this event
x
#4 DMI Async Request Response Event (DMI → NCMP) Description This event is for the asynchronous responses from DMI to NCMP following (synchronous) requests (from NCMP) specifying a (response) topic. See 3b → 4a in CPS-821 Spike: Support Async read-write operations on CPS-NCMP interface#ProposedDesign
DMI Async Request Response Event
DmiAsyncRequestResponse Event Object
Field Type Description Kafka Header Event Payload Field Notes CloudEvents Mapping eventID string The unique id identifying the event generated by DMI x standard ( id ) eventCorrelationId string The request id passed by NCMP
x extensions ( correlationid ) eventTime string The timestamp when original event occurred x standard ( time) eventTarget string The target of the event x extensions ( target) eventType string The type of the event x standard ( type ) eventSchema string The event schema for async request response events
x standard ( dataschema ) eventSchemaVersion string The event schema version for async request response events
x N/A included in 'dataschema' eventSource string The source of the event x standard ( source ) eventContent EventContent The payload of an event The name of this fields is inconsistent with all other event schemas, see issue #6
standard ( data )
Type: EventContent (cps:org.onap.cps.ncmp.events:dmi-async-request-response-event-schema:v1) Field Type Description Kafka Header Event Payload Field Notes repsonse-data-schema string The schema of response data
x response-status string The status of the response
x response-code string The code of the response
x response-data object The data payload x contains payload of type object
#5 NCMP Async Request Response Event (NCMP → Client App) Description This event is for the asynchronous responses from NCMP to Client Apps following (synchronous) requests (from client) specifying a (response) topic. See 4b → 5 in CPS-821 Spike: Support Async read-write operations on CPS-NCMP interface#ProposedDesign
NCMP Async Request Response Event
NcmpAsyncRequestResponse Event Object
Field Type Description Kafka Header Event Payload Field Notes CloudEvents Mapping eventID string The unique id identifying the event generated by DMI x standard ( id ) eventCorrelationId string The request id passed by NCMP
x extensions ( correlationid ) eventTime string The timestamp when original event occurred x standard ( time) eventTarget string The target of the event x extensions ( target) eventType string The type of the event x standard ( type ) eventSchema string The event schema for async request response events
x standard ( dataschema ) eventSchemaVersion string The event schema version for async request response events
x N/A included in 'dataschema' event Event The payload of an event Defined by event object below standard ( source ) forwardedEvent ForwardedEvent The payload of a forwarded event Relation to Event field unclear, do we need 2 events at all see issue #7
standard ( data )
Type: Event (cps:org.onap.cps.ncmp.events:ncmp-async-request-response-event-schema:v1) Field Type Description Kafka Header Event Payload Field Notes eventID string The unique id identifying the event generated by DMI x Only in payload in v1 of the payload schema. To be removed in v2 version (will be in the header only) eventCorrelationId string The request id passed by NCMP
x as above eventTime string The timestamp when original event occurred x as above eventTarget string The target of the event x as above eventType string The type of the event x as above eventSchema string The event schema for async request response events
x as above eventSource string The source of the event
x response-data-schema string The received schema of response data
x response-status string The received status of the response
x
response-code string The received code of the response
x response-data object The data payload
x contains payload of type object
Type: forwarded Event (cps:org.onap.cps.ncmp.events:ncmp-async-request-response-event-schema:v1) Field Type Description Kafka Header Event Payload Field Notes eventID string The unique id identifying the event generated by DMI ? Can NCMP put these in the header and remove them from the 'forwardedEvent' eventCorrelationId string The request id passed by NCMP
? as above eventTime string The timestamp when original event occurred ? as above eventTarget string The target of the event ? as above eventType string The type of the event ? as above eventSchema string The event schema for async request response events
? as above eventSchemaVersion string The event schema version for async request response events
? as above eventSource string The source of the event
? as above response-data-schema string The received schema of response data
x response-status string The received status of the response
x
response-code string The received code of the response
x response-data object The data payload
x contains payload of type object
Proposed New Schema:
CPS-1660Getting issue details...
STATUS
Field Type Description Kafka Header Event Payload Field Notes CloudEvents Mapping eventID string The unique id identifying the event generated by DMI standard (id) eventCorrelationId string The request id passed by NCMP
extensions(correlationid) eventTime string The timestamp when original event occurred standard (time) eventSource string The source of the event What value should contain in this? Needs clarification, see issue #8 standard (source) eventType string The type of the event standard (type) eventSchema string The event schema for async request response events
standard (dataschema) eventSchemaVersion string The event schema version for async request response events
N/A included in 'dataschema' event Event The payload of an event Defined by event object below standard (data)
Type: Event (cps:org.onap.cps.ncmp.events:ncmp-async-request-response-event-schema:v1) Field Type Description Kafka Header Event Payload Field Notes eventCorrelationId string The request id passed by NCMP
as above @Toine asked to add this property. responseStatus string The received status of the response
responseCode string The received code of the response
responseData object The data payload
contains payload of type object
#6 Batch Response Event (DMI → NCMP) Description Batch (data) request will always result in asynchronous events (responses) sent to the client. This event is the response from DMI to NCMP. See 5 in CPS-1515: Spike: Support Multiple CM-Handles for NCMP Get Operation (bulk / batch operations)#BulkRequestMessageFlow
DMI Async Batch Response Event
BatchResponseEvent Object
Field Type Description Kafka Header Event Payload Field Notes CloudEvents Mapping eventID string The unique id identifying the event generated by DMI x Mandatory
Generated by DMI-Plugin eventCorrelationId string The request id passed by NCMP
x Mandatory. This is extensions (correlationid) eventTarget string The destination topic of the client x Mandatory. TBC example: my-topic extensions ( target TBC) source string The source of the event Optional. Example : dmi-plugin:enm-1 (dmi service name) eventTime string The timestamp when original event occurred x Optional. The timestamp should follow that on https://www.rfc-editor.org/rfc/rfc3339#section-5 . This follows ISO 8601 and is what is used/referenced in 3GPP standards
standard ( time) eventType string The type of the event x Mandatory. CPS-1515 Spike standard ( type ) eventSchema string The event schema for async request response events
x Mandatory.CPS-1515 Spike standard ( dataschema ) eventSchemaVersion string The event schema version for async request response events
x N/A included in 'dataschema' event Event The payload of an event x java object not yet defined by schema, see issue #2 standard ( data ) specversion String Version of CNCF spec x hardcoded value of 1.0 standard (hardcode value)
Type: Event (cps:org.onap.cps.ncmp.event.async:dmi-async-batch -response-event-schema:v1)
Field Type Description Kafka Header Event Payload Field Notes No Properties defined (Entire event treated as single object)
#7 Batch Response Event (NCMP → Client App) Description Batch (data) request will always result in asynchronous events (responses) sent to the client. This event is the response from DMI to NCMP. See 6 in CPS-1515: Spike: Support Multiple CM-Handles for NCMP Get Operation (bulk / batch operations)#BulkRequestMessageFlow
NCMP Async Batch Response Event - Client Topic
BatchResponseEvent Object
Field Type Description Kafka Header Event Payload Field Notes CloudEvents Mapping eventID string The unique id identifying the event generated by DMI x standard ( id ) eventCorrelationId string The request id passed by NCMP
x extensions ( correlationid ) eventTime string The timestamp when original event occurred x standard ( time ) eventType string The type of the event x standard ( type) eventSchema string The event schema for async request response events
x standard ( dataschema) eventSchemaVersion string The event schema version for async request response events
x N/A included in 'dataschema' event Event The payload of an event x Java object not yet defined by schema, see issue #2 standard ( data )
Type: Event (cps:org.onap.cps.ncmp.event.async:ncmp-async-batch -response-event-schema:v1)
Field Type Description Kafka Header Event Payload Field Notes No Properties defined (Entire event treated as single object)
CNCF Cloud Event alignment Introduction
CNCF CloudEvents is a specification for describing event data in common formats to provide interoperability across services, platforms and systems.
These cloud events will be applied to all the internal and external events we have in CPS , NCMP and DMI Plugin. Cloud events will be taking care of the fields which are part of kafka headers or part of the actual payload ( fields other than "data" are sent as kafka headers ) These CNCF cloud events will be applied to all the events listed in above sections ( LCM , DMI Data AVC etc. ) Library References https://github.com/cloudevents/spec
https://github.com/cloudevents/sdk-java https://mvnrepository.com/artifact/io.cloudevents Examples : https://github.com/cloudevents/sdk-java/tree/main/examples/kafka/src/main/java/io/cloudevents/examples/kafka The table below shows how previously proposed headers should be mapped to pre-defined CNFC headers.
Before
After
CloudeEvent builder method Example Value Notes eventId
id
.withId() Mandatory eventSource
source
.withSource() Mandatory N/A
specversion (default 1.0)
.v1() 1.0 Mandatory - This is the version of the cloud eventseventType
type
.withType() Mandatory eventTime
time
.withTime() Optional eventSchema
dataschema
.withDataSchema() Optional includes the version of the schema
datacontenttype withDataContentType() application/json Optional eventCorrelationId correlationid .withExtension() Optional This will be part of the extensions field in the cloud events and all the restrictions of the attribute field naming applies to it.
i.e these fields will be in the small case.
This is marked as optional as it only applies to some events.
event data .withData(json TBC) Mandatory actual event/payload now would be under "data" field.
