...
| PlantUML Macro |
|---|
@startuml alt "Deploying the instance" activate "ACM Runtime" "ACM Runtime" -> "Participant-intermediary" : [ASYNC] Deploying the instance deactivate "ACM Runtime" activate "Participant-intermediary" activate Participant "Participant-intermediary" -> Participant : Create Deploy thread deactivate "Participant-intermediary" note right Deploy thread is stuck end note end alt "Instance in Timeout" activate "ACM Runtime" "ACM Runtime" -> "ACM Runtime" : set instance in Timeout deactivate "ACM Runtime" end alt "Undeploying the instance" activate "ACM Runtime" activate "Participant-intermediary" "ACM Runtime" -> "Participant-intermediary" : [ASYNC] Undeploying the instance deactivate "ACM Runtime" "Participant-intermediary" -> Participant : Terminate Deploy thread deactivate Participant "Participant-intermediary" -> Participant : Create Undeploy thread activate Participant deactivate "Participant-intermediary" Participant -> "Participant-intermediary" : instance Undeployed activate "Participant-intermediary" deactivate Participant "Participant-intermediary" -> "ACM Runtime" : [ASYNC] instance Undeployed deactivate "Participant-intermediary" end @enduml |
Solutions
Solution
...
Add dynamic participantId support and add database support,
Can a participant share the database with other replicas? Yes, if participants do not work with same composition/instance at same time.
Shared database with same supported element Type: a composition will be connected to a specific participantId, so only one participant will do actions with this composition and his instances. But it can fetch all compositions that are sharing.
In a restarting scenario the participant will change the participantId, and it can still fetch compositions and instances. ACM-runtime decides who has to work with, and Participant will do actions based on the message.
1: Replicas and Dynamic participantId
- UUID participantId will be generated in memory instead to fetch it in properties file.
- consumerGroup
Changes in Participant:
- UUID participantId will be generated in memory instead to fetch it in properties file.cosumerGroup will be empty (kafka configuration): any intermediary-participant will have unique Kafka queue, so they will receive same message that will be filtered by participantId.
Participants create randomly participantId and Kafka consumerGroup. This solution has been tested and has the issue to create a new Kafka queue in restarting scenario.
During restart scenario, a new consumerGroup is created, that cause some missing initial messages due the creation of new Kafka queue . The result is that to fail to receive messages from ACM to restore compositions and instances.
Solution 2: StatefulSets
Solution 3: Replicas and Database support
Changes in Participant:
- Add client support for no-sql databasedatabase (MariaDB or PostgreSQL).
- Add no-sql mock database or mock for Unit Tests.
- Refactor CacheProvider to ParticipantProvider to support insert/update, intermediary-participant can still use the cache in memory.with transactions.
- Refactor Intermediary to use insert/update of ParticipantProviderAny new/change composition and instance will be saved in database.
- Refactor Participants that are using own cache HashMap in memory (Policy Participant saves policy and policy type in memory)
...
Changes in docker/Kubernetes environment
- Refactor CSIT to support no-sql databaseRefactor performance and stability test to support no-sql databasedatabase configuration for participants
- Refactor OOM to support no-sql database
Database in Ericsson ADP marketplace
...
- database
...
- Distributed Coordinator ED (Etcd): Distributed systems use etcd as a consistent key-value store for configuration management, service discovery, and coordinating distributed work. Many organizations use etcd to implement production systems such as container schedulers, service discovery services, and distributed data storage.
- Document Database PG (PostgreSql)
- Key Value Database AG (Apache Geode): Apache Geode provides a database-like consistency model, reliable transaction processing and a shared-nothing architecture to maintain very low latency performance with high concurrency processing.(https://geode.apache.org/docs/guide/114/getting_started/intro_to_clients.html) or (https://docs.spring.io/spring-boot-data-geode-build/1.7.5/reference/html5/).
- Key Value Database RD (Redis): RDB is NOT good if you need to minimize the chance of data loss in case Redis stops working.
- Wide Column Database CD (Apache Cassandra)
- Search Engine (Open Search): It supports Rest Api, https://opensearch.org/docs/latest/clients/java/
What open source client we can use already implemented? (custom, open-search client, ...)
Open Search
Create a composition index
PUT https://{{address}}:9200/compositions
| Code Block | ||||
|---|---|---|---|---|
| ||||
{
"settings": {
"index": {
"number_of_shards": 2,
"number_of_replicas": 1
}
},
"mappings": {
"properties": {
"compositionId": {
"type": "text"
}
}
}
} |
Create a composition document
POST https://{{address}}:9200/compositions/_doc/0da7f17d-68d4-47f3-9dbf-b380413c5fad
| Code Block | ||||
|---|---|---|---|---|
| ||||
{
"compositionId": "0da7f17d-68d4-47f3-9dbf-b380413c5fad",
"automationCompositionElementDefinitionList": [
{
"acElementDefinitionId": {
"name": "onap.policy.clamp.ac.element.K8S_StarterAutomationCompositionElement",
"version": "1.2.3"
},
"automationCompositionElementToscaNodeTemplate": {
"type": "org.onap.policy.clamp.acm.K8SMicroserviceAutomationCompositionElement",
"type_version": "1.0.0",
"properties": {
"provider": "ONAP",
"startPhase": 0,
"uninitializedToPassiveTimeout": 180,
"podStatusCheckInterval": 30
},
"name": "onap.policy.clamp.ac.element.K8S_StarterAutomationCompositionElement",
"version": "1.2.3",
"metadata": {},
"description": "Automation composition element for the K8S microservice for AC Element Starter"
},
"outProperties": {}
},
{
"acElementDefinitionId": {
"name": "onap.policy.clamp.ac.element.K8S_BridgeAutomationCompositionElement",
"version": "1.2.3"
},
"automationCompositionElementToscaNodeTemplate": {
"type": "org.onap.policy.clamp.acm.K8SMicroserviceAutomationCompositionElement",
"type_version": "1.0.0",
"properties": {
"provider": "ONAP",
"startPhase": 0,
"uninitializedToPassiveTimeout": 180,
"podStatusCheckInterval": 30
},
"name": "onap.policy.clamp.ac.element.K8S_BridgeAutomationCompositionElement",
"version": "1.2.3",
"metadata": {},
"description": "Automation composition element for the K8S microservice for AC Element Bridge"
},
"outProperties": {}
},
{
"acElementDefinitionId": {
"name": "onap.policy.clamp.ac.element.K8S_SinkAutomationCompositionElement",
"version": "1.2.3"
},
"automationCompositionElementToscaNodeTemplate": {
"type": "org.onap.policy.clamp.acm.K8SMicroserviceAutomationCompositionElement",
"type_version": "1.0.0",
"properties": {
"provider": "ONAP",
"startPhase": 0,
"uninitializedToPassiveTimeout": 180,
"podStatusCheckInterval": 30
},
"name": "onap.policy.clamp.ac.element.K8S_SinkAutomationCompositionElement",
"version": "1.2.3",
"metadata": {},
"description": "Automation composition element for the K8S microservice for AC Element Sink"
},
"outProperties": {}
}
]
} |
Fetch a composition document
GET https://{{address}}:9200/compositions/_doc/0da7f17d-68d4-47f3-9dbf-b380413c5fad
Create an instance index
PUT https://{{address}}:9200/instances
| Code Block | ||||
|---|---|---|---|---|
| ||||
{
"settings": {
"index": {
"number_of_shards": 2,
"number_of_replicas": 1
}
},
"mappings": {
"properties": {
"instanceId": {
"type": "text"
}
}
}
} |
Create an instance document
POST https://{{address}}:9200/instances/_doc/caf50cde-11a2-4915-a49c-609762714a6f
| Code Block | ||||
|---|---|---|---|---|
| ||||
{
"instanceId": "caf50cde-11a2-4915-a49c-609762714a6f",
"compositionId": "0da7f17d-68d4-47f3-9dbf-b380413c5fad",
"deployState": "DEPLOYED",
"lockState": "LOCKED",
"elements": {
"709c62b3-8918-41b9-a747-d21eb79c6c23": {
"id": "709c62b3-8918-41b9-a747-d21eb79c6c23",
"definition": {
"name": "onap.policy.clamp.ac.element.Http_SinkAutomationCompositionElement",
"version": "1.2.3"
},
"deployState": "DEPLOYED",
"lockState": "LOCKED",
"properties": {
"baseUrl": "http://10.101.0.249:30802",
"httpHeaders": {
"Content-Type": "application/json",
"Authorization": "Basic YWNtVXNlcjp6YiFYenRHMzQ="
},
"configurationEntities": [
{
"configurationEntityId": {
"name": "onap.policy.clamp.ac.sink",
"version": "1.0.0"
},
"restSequence": [
{
"restRequestId": {
"name": "request3",
"version": "1.0.1"
},
"httpMethod": "POST",
"path": "/activate",
"body": "{ \"receiverId\": { \"name\": \"onap.policy.clamp.ac.sink\", \"version\": \"1.0.0\" }, \"timerMs\": 20000, \"elementType\": \"SINK\", \"topicParameterGroup\": { \"server\": \"message-router\", \"listenerTopic\": \"POLICY_UPDATE_MSG\", \"publisherTopic\": \"AC_ELEMENT_MSG\", \"fetchTimeout\": 15000, \"topicCommInfrastructure\": \"dmaap\" } }",
"expectedResponse": 201
}
]
}
]
},
"outProperties": {}
},
"709c62b3-8918-41b9-a747-d21eb79c6c24": {
"id": "709c62b3-8918-41b9-a747-d21eb79c6c24",
"definition": {
"name": "onap.policy.clamp.ac.element.Http_StarterAutomationCompositionElement",
"version": "1.2.3"
},
"deployState": "DEPLOYED",
"lockState": "LOCKED",
"properties": {
"baseUrl": "http://10.101.0.249:30800",
"httpHeaders": {
"Content-Type": "application/json",
"Authorization": "Basic YWNtVXNlcjp6YiFYenRHMzQ="
},
"configurationEntities": [
{
"configurationEntityId": {
"name": "onap.policy.clamp.ac.starter",
"version": "1.0.0"
},
"restSequence": [
{
"restRequestId": {
"name": "request1",
"version": "1.0.1"
},
"httpMethod": "POST",
"path": "/activate",
"body": "{ \"receiverId\": { \"name\": \"onap.policy.clamp.ac.startertobridge\", \"version\": \"1.0.0\" }, \"timerMs\": 20000, \"elementType\": \"STARTER\", \"topicParameterGroup\": { \"server\": \"message-router:3904\", \"listenerTopic\": \"POLICY_UPDATE_MSG\", \"publisherTopic\": \"AC_ELEMENT_MSG\", \"fetchTimeout\": 15000, \"topicCommInfrastructure\": \"dmaap\" } }",
"expectedResponse": 201
}
]
}
]
},
"outProperties": {}
},
"709c62b3-8918-41b9-a747-d21eb79c6c25": {
"id": "709c62b3-8918-41b9-a747-d21eb79c6c25",
"definition": {
"name": "onap.policy.clamp.ac.element.Http_BridgeAutomationCompositionElement",
"version": "1.2.3"
},
"deployState": "DEPLOYED",
"lockState": "LOCKED",
"properties": {
"baseUrl": "http://10.101.0.249:30801",
"httpHeaders": {
"Content-Type": "application/json",
"Authorization": "Basic YWNtVXNlcjp6YiFYenRHMzQ="
},
"configurationEntities": [
{
"configurationEntityId": {
"name": "onap.policy.clamp.ac.bridge",
"version": "1.0.0"
},
"restSequence": [
{
"restRequestId": {
"name": "request2",
"version": "1.0.1"
},
"httpMethod": "POST",
"path": "/activate",
"body": "{ \"receiverId\": { \"name\": \"onap.policy.clamp.ac.bridgetosink\", \"version\": \"1.0.0\" }, \"timerMs\": 20000, \"elementType\": \"BRIDGE\", \"topicParameterGroup\": { \"server\": \"message-router:3904\", \"listenerTopic\": \"POLICY_UPDATE_MSG\", \"publisherTopic\": \"AC_ELEMENT_MSG\", \"fetchTimeout\": 15000, \"topicCommInfrastructure\": \"dmaap\" } }",
"expectedResponse": 201
}
]
}
]
},
"outProperties": {}
}
},
"stateChangeResult": "NO_ERROR"
} |
Fetch an instance document
...
- for participants