- CPS-821Getting issue details... STATUS
Description/Scope
The scope of this spike is to ascertain:
- How to use messaging (producer, agree topic etc))
- Using existing rest endpoint with additional flag indicating async response
- Also consider asynchronous request option using messaging in the proposal
Associated Jira Created for Implementation
Issues/Decisions
Color | Meaning |
---|---|
New/Open | |
Reopened | |
Agreed/Closed |
# | Issue | Notes/Jira | Decision |
---|---|---|---|
1 | What topic to use for client? | Topic provided by client as a parameter which will be injected into our environment and used for asynchronous requests sent back to client. | Update: Toine Siebelink had a conversation with Tony Finnerty it has been agreed CPS is to configure a (temporary) dedicated client-topic as final destination for the async replies: we now want to propose to use a common ‘cps-broadcast’ topic which also will used for broadcasting messages like cm handles being added o removed etc. Still it is only a temporary solution until the client have a better way of declaring their own topics. It does not affect our proposed interfaces, just the name of the topic that will be accepted by NCMP (for now) --------------------------------------------------------------------------------------- Update: After discussion with Fiachra Corcoran regarding Kafka-strimzi it has been agreed that we will define a topic for client responses in advance (defined upfront). After release K it is envisioned that auto creation of topics will be disbaled by default thus the client will have to use our predefined topic. Agreement on topic name outstanding ---------------------------------------------------------------------------------------
|
2 | What topic to use for private DMI-NCMP? | Contact Fiachra Corcoran regarding ONAP conventions. Response was that there aren't any conventions to speak of but we would use dashes (i.e. my-new-topic) instead of dot notation (i.e. my.new.topic) for topic name | ncmp-async-m2m |
3 | Are adding a new REST endpoint for async or modifying an existing endpoint? | To facilitate asynchronous requests to DMI we will need to either create a new endpoint or modify existing endpoint to include /async flag. The second solution may not be backwards compatible. However creating a new endpoint solely for a flag is also not ideal. We could add async to list of options (but this might interfere with the purpose of /options. Additionally, considered adding a new endpoint for async which simply re-routes the response to the original endpoint while adding the logic for OK response to the client. However, would this lead to a change in the schema? If so, would this be backwards compatible? | /ncmp/v1/data/ch/123ee5/ds/ncmp-datastore:*?topic=<topic-name> |
4 | Agree URL for async once #2 is clarified | CPS R10 Release Planning#NCMPRequirements #11. | /ncmp/v1/data/ch/123ee5/ds/ncmp-datastore:*?topic=<topic-name> |
5 | Passthrough request need to be able to handle different response types (using accept header) but the async option would have a fixed and possibly different response type. | CPS R10 Release Planning#NCMPRequirements #11. | The async response can 'wrap' the native response inside a json object identical to the synchronous response |
6 | Should we create a standalone app to demo or are tests sufficient? | CSIT tests may require more involved effort - perhaps we could add standalone app to nexus and use it as part of CSIT test? | See #13 |
7 | Do we need to persist the generated requestID? | We should be be stateless | No - Further disc |
8 | Error Reporting - Topic Correctness/Availability | At a minimum we should report to the client if a topic was not found or if the topic name was incorrect | In Scope |
9 | Error Reporting - Kafka Issues | Issues such full buffer/queue, drop messages, failure not in scope | Out of scope |
10 | Async Request Option using Messaging | See: https://wiki.onap.org/display/DW/CPS-821+Spike%3A+Support+Async+read-write+operations+on+CPS-NCMP+interface#CPS821Spike:SupportAsyncreadwriteoperationsonCPSNCMPinterface-AsyncRequestOptionusingMessaging(OutofScope) | Out of scope |
11 | Do we actually require futures in this implementation proposal? | It could be argued that the need for futures is made redundant by the fact we call dmi from ncmp through rest and the response will be consumed via Kafka. What benefit would future give us in this case? | Not needed |
12 | ID Generation | Which mechanism to use? Look at CPS-Temporal and follow to keep consistency | See: https://wiki.onap.org/display/DW/CPS-821+Spike%3A+Support+Async+read-write+operations+on+CPS-NCMP+interface#CPS821Spike:SupportAsyncreadwriteoperationsonCPSNCMPinterface-CanRobotFrameworkverifyKafkaEvents |
13 | Can robot framework verify if Kafka events have been sent/received | This would be less work and overhead (rather than creating/.maintaining client app) Will need to verify if 3PP libraries are safe to introduce into codebase. If so, what is the process? Do they need to be FOSSed? | Integration testing should be carried out by a client of NCMP. Demo can be performed up the point NCMP produces message for the client. |
14 | Can Webflux do this work with less code/impl? | Sourabh Sourabh suggested using this to compliment our existing approach. By adding webflux we add an event loop to synchronize and access I/O connections to the database. | No, It will compliment the design by adding an event loop for I/O synchronization and access. See: CPS-850 |
15 | ONAP may be deprecating PLAINTEXT for Kafka. Strimzi Kafka might need to be used | No relevant information could be found relating to this. See: https://wiki.onap.org/display/DW/CPS-821+Spike%3A+Support+Async+read-write+operations+on+CPS-NCMP+interface#CPS821Spike:SupportAsyncreadwriteoperationsonCPSNCMPinterface-KafkaStrimziInvestigation | The underlying implementation won't be affected. The config will contain relevant configuration for protocol (e.g. PLAINTEXT, SASL) and connections. This information needs to be made configurable when implementing. |
16 | Should we have something like the message count in the response or an indication of the last message at least.? | On the topic=Xyz async analysis, One item I forgot to mention was when a client might know when they have received the last response message for a request. If async there may be many responses (otherwise there's no need for async) Therefore should it not have something like the message count in it or an indication of the last message at least. | This is another step in an evolution of the product. Multiple response messages might become a necessity for asycnc bulk operations and maybe that is when we can discuss it in more detail. I don’t think it has to affect our current implementation additional field to handle that can always be added in a backward compatible way. |
17 | Do we need Kafka messaging between DMI and NCMP? | This was discussed as it was argued that the client and CPS ecosystem is decoupled and async using only bus between ncmp and client. However, this would not rectify blocking calls between ncmp and dmi. | It was decided to have two separate kafka topics for client → ncmp & ncmp → dmi |
18 | Can NCMP list all existing topics | - CPS-828Getting issue details... STATUS AC Point 3 : Topic does not exists (not allowed to create new topics, assuming NCMP can 'list' existing topics) Gareth Roper Need your suggestion. | |
19 | AC Points : 2. cm handle does not exists 3. No DMI registration for CM Handle | AC point 2 : We agreed to check only if cm handle exists. If cm does not exists will get Http status 404 as below. { AC point 3 . Is not valid. | |
20 | Client Kafka User & Privileges | After discussion with Fiachra Corcoran regarding Kafka-strimzi we will need to ensure client can access Kafka user and be allocated sufficient privileges (principle of least privilege) | |
21 | agree client response format | Message gets forwarded to original Client Topic |
Proposed Design
High-level Steps/Possible Tickets:
- Modify REST endpoint to include param topic (1)
- Add logic to send response and request (2a)
- Send request if async is implemented or NotSupportedException if not (2b)
- Add producer to DMI (implementation and config) (31 & 3b)
- Add consumer to NCMP (implementation and config) (4a)
- Add Producer to NCMP (implementation and config) (4b)
- Demo & Test (5)
Alternative Approaches
# | Person | Approach |
---|---|---|
1 | [Relates to point 3 from high-level steps] It would be good to have async implementation only dependent on NCMP. So that it does not break or work with only few DMI plugins. 2. No async request get lost because of pod restart or JVM crash. |
Proposed Message Format
Success Scenario
- Is this not NCMP to DMI rather than DMI to NCMP?
Kieran McCarthy : No, above is what is sent from DMI to NCMP. This is the response that goes back to NCMP including the target client topic name- Joseph Keenan Perfect thanks - I was thinking of the reponse once the async is recieved
- Should response code above be OK?
Kieran McCarthy : as all calls are passthrough in the scope of this step/story thats up to the schema chosen for the dmi plugin.
Normally end client applications prefer to work with codes rather than strings.- Joseph Keenan Understood thanks
- Is there too much information being returned to NCMP?
- Should we just return OK??
Kieran McCarthy : again this is up to the chosen dmi plugin to decide on what the schema is - in this case for it would be sufficient just return minimal data.
I guess if it is a GET request then I suppose the response event will have some objects returned also meaning the schema would also need to support.- Joseph Keenan Understood Thanks
- Should we just return OK??
Failure Scenario
Kafka config & Implementation
Example Kafka Consumer Implementation from CPS-Temporal
The below code snippet taken from cps-temporal can be used in the same way in NCMP to listen to message from DMI substituting the topics and errorHandler
Example Kafka Consumer Config from CPS-Temporal
Example Kafka Producer Implementation from CPS-NCMP
Example Kafka Producer Config from CPS-NCMP
Example Kafka Docker-Compose
Future or alternative (Out of Scope)
What are Futures?
A Java Future, java.util.concurrent.Future
, represents the result of an asynchronous computation. When the asynchronous task is created, a Java Future
object is returned. This Future
object functions as a handle to the result of the asynchronous task. Once the asynchronous task completes, the result can be accessed via the Future
object returned when the task was started
source: http://tutorials.jenkov.com/java-util-concurrent/java-future.html
CompletableFuture (Java8+)
Java 8 introduced the CompletableFuture class. Along with the Future interface, it also implemented the CompletionStage interface. This interface defines the contract for an asynchronous computation step that we can combine with other steps.
CompletableFuture is at the same time a building block and a framework, with about 50 different methods for composing, combining, and executing asynchronous computation steps and handling errors.
source: https://www.callicoder.com/java-8-completablefuture-tutorial/
Alternatives - Thread
# | Type | Pros | Cons | Recommend |
---|---|---|---|---|
1 | Future | Futures return value | Y | |
2 | Thread | threads does not return anything as the run() method returns void . We could possibly implement mechanism to trigger a response but this is unnecessary as futures do this | N |
Request ID Generation
Type | Method | Ease of implementation | Decision |
---|---|---|---|
UUID |
| Easy | ~ |
Custom | We generate our own (example exists in NCMP (notificationPublisher - confrm)) | Medium | - |
HTTP Request ID | Further investigation required | ~ | |
Kafka Event ID | Further investigation required | ~ |
How do NCMP → CPS-Temporal perform ID Generation for Events?
Async Request Option using Messaging (Out of Scope)
This was for a future completely message driven solution (for now we start with a REST request that will generate an async message eventually. In future we could also send a message that will trigger the same.
Webflux Investigation (Out of Scope)
What is Webflux?
Spring WebFlux is a web framework that’s built on top of Project Reactor, to give you asynchronous I/O, and allow your application to perform better. The original web framework included in the Spring Framework, Spring Web MVC, was purpose-built for the Servlet API and Servlet containers. The reactive-stack web framework, Spring WebFlux, was added later in version 5.0. It is fully non-blocking, supports Reactive Streams back pressure, and runs on such servers as Netty, Undertow, and Servlet 3.1+ containers.
We suggest that you consider the following specific points:
If you have a Spring MVC application that works fine, there is no need to change. Imperative programming is the easiest way to write, understand, and debug code. You have maximum choice of libraries, since, historically, most are blocking.
In a microservice architecture, you can have a mix of applications with either Spring MVC or Spring WebFlux controllers or with Spring WebFlux functional endpoints. Having support for the same annotation-based programming model in both frameworks makes it easier to re-use knowledge while also selecting the right tool for the right job.
A simple way to evaluate an application is to check its dependencies. If you have blocking persistence APIs (JPA, JDBC) or networking APIs to use, Spring MVC is the best choice for common architectures at least. It is technically feasible with both Reactor and RxJava to perform blocking calls on a separate thread but you would not be making the most of a non-blocking web stack.
If you have a Spring MVC application with calls to remote services, try the reactive
WebClient
. You can return reactive types (Reactor, RxJava, or other) directly from Spring MVC controller methods. The greater the latency per call or the interdependency among calls, the more dramatic the benefits. Spring MVC controllers can call other reactive components too.If you have a large team, keep in mind the steep learning curve in the shift to non-blocking, functional, and declarative programming. A practical way to start without a full switch is to use the reactive
WebClient
. Beyond that, start small and measure the benefits. We expect that, for a wide range of applications, the shift is unnecessary. If you are unsure what benefits to look for, start by learning about how non-blocking I/O works (for example, concurrency on single-threaded Node.js) and its effects.Webflux supports:
- Annotation-based reactive components
- Functional routing and handling
Source: https://docs.spring.io/spring-framework/docs/current/reference/html/web-reactive.html
Pros & cons
Pros | Cons |
---|---|
|
|
Links to materials:
https://www.baeldung.com/spring-webflux
https://www.youtube.com/watch?v=1F10gr2pbvQ
Kafka Strimzi Investigation
Could not find relevant information for ONAP strimzi but there is a PoC to move CPS to Strimzi:
- DMAAP-1681Getting issue details... STATUS
Can Robot Framework verify Kafka Events?
It does not appear to be possible to verify Kafka in Robotframework natively, but there are third party libraries that would aid in this:
- https://robooo.github.io/robotframework-ConfluentKafkaLibrary/
- https://github.com/opencord/kafka-robot
Demo/Test
Existing Groovy tests exist for Kafka in cps-service/src/test/groovy/org/onap/cps/notification
- CPS-834Getting issue details... STATUS
For a demo it will be sufficient to show that the response is being produced on the client topic.
The client should conduct integration testing.