Jira item: CPS-474 - Getting issue details... STATUS

Overview

For now, sending data updated event notifications from CPS Core is controlled at application level using "notification.data-updated.enabled" application property indicator.

For optimization and more fine grain control on the events that are published, there is a need to consider additional criteria within the notification process. One of these criteria is the dataspace. The requirement is to be able to publish notifications for data belonging to some specific dataspace and do not send any notifications for some other dataspaces.

The data updated configuration at application level has priority over the configuration at dataspace level. It means, no notification at all can be sent if the configuration is not enabled at application level, no matter which configuration is set at dataspace level.

Proposition 1: Extending existing entities with notification configuration

This proposition consists in extending current existing CPS concepts with additional properties to enable or disabling the notification for a specific instance.

For dataspace, it means adding a "data updated notification indicator" information to the entity.

Database Model

At database level, the following column is added to dataspace table (in cps core db schema):

  • Name: data_updated_notification_indicator
  • Type: boolean
  • Nullable: false
  • Default value: false

DataspaceEntity class is modified accordingly to have this new column added as an entity property.

Service Interface

At service layer, missing Dataspace model class is created to reflect the dataspace structure and to be mapped to DataspaceEntity.

→ Creation

To create a new dataspace the following interface method is added for CpsAdminPersistenceService class:

  • Dataspace createDataspace(@NonNull Dataspace);

If ever the current interface is exposed externally, then existing "void createDataspace(@NonNull String dataspaceName)" method needs to be kept and its implementation changed to call the new method above with null of false notification indicator. This method can be made deprecated for future removal.

→ Modification

To update an existing dataspace a new method is created for CpsAdminPersistenceService class:

  • Dataspace updateDataspace(@NonNull Dataspace);

Updating the name of the dataspace is not allowed.

REST API

→ Creation

At REST API level, dataspace creation requires a new parameter for the notification indicator.

To keep the change as non breaking, the existing dataspace endpoint is kept unchanged:

  • POST - /cps/api/v1/dataspaces?dataspace-name=<name>

and its implementation is modified to create a dataspace with the default false value for "data-updated-notification-indicator".

Unfortunately, this endpoint does not align with REST conventions and does not prepare for future changes. Then, redesigning the endpoint for improvement is considered:

  • POST - /cps/api/v1/dataspaces (or /cps/api/v2/dataspaces ?)

with all payload information for the dataspace to be created provided in the HTTP request body ("dataspace-name" being mandatory and "data-updated-notification-indicator" optional):

{
  "dataspace-name": "<name>",
  "data-updated-notification-indicator": "<indicator>"
}

and the dataspace created is contained in the HTTP response body.

Both v1 and v2 endpoints could be supported for a transition period, with v1 marked as deprecated for future removal.

→ Modification

Dataspace modification requires a new endpoint:

  • PUT /cps/api/v1/dataspaces?dataspace-name=<name> (or /cps/api/v2/dataspaces?dataspace-name=<name> ?)

with all payload information for the dataspace to be created provided in the HTTP request body ("dataspace-name" being mandatory, consistent with the query parameter and "data-updated-notification-indicator" optional):

{
  "dataspace-name": "<name>",
  "data-updated-notification-indicator": "<indicator>"
}

and the dataspace modified is contained in the HTTP response body.

Updating the name of the dataspace is not allowed.

Service Processing

NotificationService class is changed to complete following checks in order to publish an event when a data node is updated:

  1. Data updated event notification is enabled at application level (notification.data-updated.enabled application property, already implemented)
  2. Data updated event notification is enabled for the dataspace the updated data belongs to (to be implemented)

All checks above needs to be successful to send the notification.

For performance optimization, caching dataspace information is considered to avoid requesting the database each time this information is needed.

Pro and cons

  • Pros
    • Effort is limited to implement filtering for dataspaces
  • Cons
    • Requires evolution in current existing APIs (service and REST) to manage the entity
    • Potential risk to spread notification configuration in many different elements depending on future filtering requirements to be implemented.

Proposition 2: Create new entities dedicated to notification configuration

This proposition consists in implementing new dedicated entities to support event notification filtering, without impacting existing ones.

A specific class is created for each type of notification that can be configured (application, dataspace, ...). Any instance of these classes is defining if the corresponding entity should be sent for data updated notification. Configuration for application notification can not obviously have more than one instance. Also, a class is created to define the default behaviour, for each type, when no specific configuration is provided for a specific instance.

The implementation includes:

  • Entity classes
  • Repository classes for persistence
  • Model classes
  • Service classes to support notification configuration management (create, read, update, delete)
  • Controller classes to support web REST API to manage notification configuration (create, read, update, delete)

Below is the entity/model classes diagram and a data sample example:

Pro and cons

  • Pros
    • Dedicated entity and model does not impact existing ones
    • Information required to determine if a data update should be published or not is not spread in multiple entities, then could be most efficient to retrieve.
    • Generic and flexible solution that should be able to support consistent evolution
  • Cons
    • Design is a bit more complex to handle all cases. Need to be refined.
    • Not necessary a con but design for Yang model filtering still need to be detailed:

Proposition 3: Use application properties for notification configuration

This proposition consists in a simple and straight forward solution to support dataspace notification filtering by using application configuration properties.

Notification configuration is specified in the application.yaml with the following:

  • Application level enabled flag
  • A list of datasapces that are enabled for notification. These dataspaces are identified with their name. Regular expressions car be provided to specify multiple names. If the dataspace list is empty, then no publication is made even if it is enabled at the application level.

Ideally, this configuration should be also easily provided using environment variables when the application is starting.

Snippet below represents notification configuration section from application.yaml file.

application.properties
notification:
  data-updated:
    enabled: false
    filters:
      enabled-datspaces:
        - *-published
    topic: ${CPS_CHANGE_EVENT_TOPIC:cps.cfg-state-events}

Pro and cons

  • Pros
    • Simpler and quicker to implement with limited code change to be made. There is no need for an API to persist and manage notification configuration.
    • Good fit for a transition implementation, giving more time to refine future additional notification filtering requirement and design.
  • Cons
    • Any change in the configuration requires a service restart
    • Limited basic configuration options

 Conclusion

As per the analysis made for the 3 propositions above, the decision is made to implement Proposition 3 (application properties) for Istanbul release.

All 3 propositions will be re-evaluated during next release depending on the notification filtering requirement that will be considered at that time.

  • No labels