...
| Jira | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Issues/Decisions
...
| # | Issue | Decision | Notes/Jira | Decision | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | What topic to use for client? | To be supplied by cient | Topic provided by client as a parameter which will be injected into our environment and used for asynchronous requests sent back to client. | To be supplied by cient | |||||||||||
| 2 | What topic to use for private DMI-NCMP? | e.g. ncmp-async-private but decision needs to be made with current best practices. | |||||||||||||
| 3 | Are adding a new REST endpoint for async or modifying an existing endpoint?/ncmp/v1/data/ch/123ee5/ds/ncmp-datastore:*?topic=<topic-name> | To facilitate asynchronous requests to DMI we will need to either create a new endpoint or modify existing endpoint to include /async flag. The second solution may not be backwards compatible. However creating a new endpoint solely for a flag is also not ideal. We could add async to list of options (but this might interfere with the purpose of /options. Additionally, considered adding a new endpoint for async which simply re-routes the response to the original endpoint while adding the logic for OK response to the client. However, would this lead to a change in the schema? If so, would this be backwards compatible?
| 4Agree URL for async once #2 is clarified | /ncmp/v1/data/ch/123ee5/ds/ncmp-datastore:*?topic=<topic-name> | |||||||||||
| 4 | Agree URL for async once #2 is clarified | CPS R10 Release Planning#NCMPRequirements #11CPS R10 Release Planning#NCMPRequirements #11. Based on this additional path parameter we no longer require additional /async flag in url. | /ncmp/v1/data/ch/123ee5/ds/ncmp-datastore:*?topic=<topic-name> | ||||||||||||
| 5 | Passthrough request need to be able to handle different response types (using accept header) but the async option would have a fixed and possibly different response type. | CPS R10 Release Planning#NCMPRequirements #11. | We should, by default, be able to accept multiple contnet-types | .CPS R10 Release Planning#NCMPRequirements #11. | |||||||||||
| 6 | Should we create a standalone app to demo or are tests sufficient? | Yes, standalone app would be helpful, along with traditional tests. | CSIT tests may require more involved effort - perhaps we could add standalone app to nexus and use it as part of CSIT test? TODO: Check if robot framework can verify if Kafka events have been sent/recieved | ||||||||||||
| 77 | Do we need to persist the generated requestID? | No | We should be be stateless | No | |||||||||||
| 8 | Error Reporting | Out of scope | |||||||||||||
| 9 | CSIT Testing | Will this be required to add to CSIT tests? If so, client app will need to be created and stored in gerrit for cloning during setup phase. | 10 | Async Request Option using Messaging Out of scope | See: AsyncRequestOptionusingMessaging | Out of scope | 11|||||||||
| 10 | Do we actually require futures in this implementation proposal? | It could be argued that the need for futures is made redundant by the fact we call dmi from ncmp through rest and the response will be consumed via Kafka. What benefit would future give us in this case? | Not needed | ||||||||||||
| 11 | ID Generation | Which mechanism to use? Look at CPS-Temporal and follow to keep consistency | |||||||||||||
| 12 | Are there any Kafka specific environment variables or config variables to be set for this work? | We will need to add new topic for DMI → NCMP M2M Kafka - which file(s)? through rest and the response will be consumed via Kafka.What benefit would future give us in this case? |
Proposed Diagram
...
| draw.io Diagram | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
...
- Modify REST endpoint to include param topic (1)
- Add logic to send response and request (2a & 2b)
- Add producer to DMI (implementation and config) (31 & 3b)
- Add consumer to NCMP (implementation and config) (4)Ther eis
- already a kafka producer in NCMP (KafkaProducerListenerAdd consumer to NCMP (implementation and config) (5)
- Demo & Test (5, 6 , & 7)
Kafka config & Implementation
...
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
kafka:
image: confluentinc/cp-kafka:6.1.1
container_name: kafka
ports:
- "19092:19092"
depends_on:
- zookeeper
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,CONNECTIONS_FROM_HOST://localhost:19092
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,CONNECTIONS_FROM_HOST:PLAINTEXT
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 |
Future
...
or alternative (Out of Scope)
...
What are Futures?
A Java Future, java.util.concurrent.Future, represents the result of an asynchronous computation. When the asynchronous task is created, a Java Future object is returned. This Future object functions as a handle to the result of the asynchronous task. Once the asynchronous task completes, the result can be accessed via the Future object returned when the task was started
...
| # | Type | Pros | Cons | Recommend |
|---|---|---|---|---|
| 1 | Future | Futures return value | Y | |
| 2 | Thread | threads does not return anything as the run() method returns void. We could possibly implement mechanism to trigger a response but this is unnecessary as futures do this | N |
RequestID Generation
...
| Type | Method | Ease of implementation | Recommend |
|---|---|---|---|
| UUID | | Easy | Y |
| Custom | We generate our own | Medium - Hard | N |
| HTTP Request ID | |||
| Kafka ID |
Async Request Option using Messaging (Out of Scope)
...