...
4. Concepts and Relationships on CLAMP Control Loops
5.
...
APIs and
...
Protocols
The design and implementation of TOSCA Control Loops in CLAMP is described in the following pagesAPIs and Protocols used by CLAMP for Control Loops are described on the pages below:
- Defining Control Loops in TOSCA for CLAMP
- REST APIs for CLAMP Control Loops
- The CLAMP Control Loop Participant Protocol
Design and Implementation
The design and implementation of TOSCA Control Loops in CLAMP is described for each executable entity on the pages below:
- The CLAMP Runtime Server
- CLAMP Participants
- The CLAMP GUI
- Building and running CLAMP
- Testing CLAMP
| Warning |
|---|
This page is updated for Istanbul to this point, the information below this point may or may |
| Warning |
This page is updated for Istanbul to this point, the information below this point may or may not be correct for Istanbul. |
...
4.4
...
ControlLoop Runtime Swagger REST APIs:
ControlLoop_Runtime_Swagger_API.yml
Participant Swagger REST APIs:
3: APIs and Sequence Diagrams
3.1: Commissioning
Ajay Deep Singh to pad out this section
This section defines Commissioning/CRUD Operations that can be performed on ControlLoops.
A Client, in this case CLAMP, can perform CRUD operations or can commission ControlLoops from DesignTime to RunTime Inventory Database.
DesignTime/RunTime Catalogue/Inventory Database stores ControlLoop definitions, CRUD operations on database supported by REST Endpoints like Get, Delete, Create allowing selection of a particular ControlLoop to be addressed, below sequence diagram will help you understand flow how a client(Clamp) application can initiate Rest call for performing different operations on Database.
API_Gateway Service is for interacting to different database DesignTime/RunTime and should be responsible for responding success or failure status on different operations.
The commissioning of ControlLoops definition from DesignTime Catalogue to RunTime Inventory Database can we achived using the commissioning Rest Endpoint, in this process when a rest request is initiated from a client(Clamp) the API_Gateway Service take cares of fetching ControlLoops metadata from DesignTime and creates in RunTime Inventory Database, Commissioning API ControlLoop Sequence diagram will help you understand the flow.
| Warning |
|---|
In future commissioning Rest Endpoint might be updated to push ControlLoops not only in RunTime Database but to the participants involved in ControlLoop. |
3.1.1: Commissioning REST API
3.1.2: Commissioning Sequence Diagrams
GET, DELETE, CREATE API ControlLoop Sequence Diagram
Commission API ControlLoop Sequence Diagram
3.2: Instantiation
Robertas Rimkus to pad out this section
This section refers to Instantiation of a Commissioned control loop. A client, in this case CLAMP (potentially DCAEMOD, etc in the future) will render the commissioned control loops allowing selection of a particular control loop to be instantiated. User will then provide the configurations needed to instantiate the selected control loop which will be sent onto the CL_Instance_Control Service. The service will then distribute the configurations to DMaaP topic. Participants (agents) will pull the event containing the config and pick out their control loop components to be instantiated and start/set up those particular components. CL_Instance_Control Service will be waiting for a response back from all participants involved in the instantiation of the control loop, in regards to the state of instantiation. In successful response case the service will store the CL Instance LCM (Life Cycle management) data into the runtime DB as well as providing a message back to the client of the successful instantiation. In failure to receive the response case, a timeout will be called, which will result in a teardown event being sent to DMaaP. The participants will then receive the event and proceed to teardown the components that were instantiated or check that they have failed to instantiate in the first place and send a Teardown ACK back to the CL_Instance_Control Service. No CL Instance LCM data will be stored and a message indicating failure to instantiate the CL along side with the error will be sent back to the client (CLAMP).
3.2.1: Instantiation REST API
3.2.2: Instantiation Sequence Diagrams
3.2.3: Instantiation DMaaP API
Initial Thought for an event to be sent from CL_Instance_Control onto DMaaP for Participants to consume. The event would go onto an output topic which the Participants would be polling/subscribed to
e.g url : https://{{ONAPIP}}:{{DMaaPPort}}/events/CL_INSTANCE_CONTROL_OUTPUT
*Preferred solution is to send TOSCA in the body. Meaning we could reuse the parsing code which is already present and provide it to the participant.
3.2.4: Instantiation Participant API
*Suggestion was to put JAVA API code in this section for the participant talking to DMaaP. TBD
3.3: Monitoring
In this case it refers to monitoring the data that the participants will provide to DMaaP. Participants will send events to DMaaP which will be pulled by the CL_Supervision_Service in to the runtime database. Monitoring service provides APIs to display the statistics data from runtime database to the Monitoring GUI. The data provided should include a reference id to the control loops that are instantiated on the participant, as well as the applications that have been instantiated as a part of that control loop for that participant. Data should also include the time that the application has started, state of it (running/terminated) and any other critical information which would help to determine the health of an instantiated control loop and its components. Idea is for the participant to provide events every certain period of time, similar to a health-check, in order to provide consistent monitoring.
3.3.1: Monitoring REST API
3.3.2: Monitoring Sequence Diagrams
3.3.3: Monitoring DMaaP API
Participants will send an event containing monitoring data to a DMaaP topic at a set interval after participant has received an event to instantiate a control loop
e.g url: https://{{ONAPIP}}:{{DMaaPPort}}/events/CL_MONITORING_SERVICE_INPUT
3.3.4: Monitoring Participant API
Presume similar thinking to Instantiation Participant API
*Suggestion was to put JAVA API code in this section for the participant talking to DMaaP. TBD
3.4: Supervision
Supervision is responsible for ensuring that
- control loops are established once their initiation has been ordered
- control loops are running correctly once their initiation is completed
- control loops are correctly removed once their removal has been ordered
3.4.1: Supervision Sequence Diagrams
3.4.2: Supervision APIs to other components
4: Design
4.1: Server Side
4.1.1 Database Schema and JPA
4.1.2: TOSCA Processing
4.1.3: Instance Control
4.1.4: Execution Monitoring
4.2:
4.3: Client Side
4.3.1: Client SDK: Composition of Control Loop Tosca
4.3.2: Client User Interface
4.4 Other Considerations
4.4.1 Upgrade
Performing a hot upgrade of the Control Loop at run time as well as handling an upgrade of the software in one or more of the participants in an Control Loop is a particularly challenging issue because upgrading must handle the following cases without tearing down the Control Loop:
- Upgrade and changes of the configuration data of participants
- Addition of or removal of participants in an Control Loop
- Upgrade of software in one or more participants in an Control Loop
- Maintenance of compatibility between participants when an update of more than one participant must be done together to ensure compatibility, for example, when a protocol being used by two participants to communicate is upgraded
4.4.2 Scalability
The system is designed to be inherently scalable. The control loop runtime server is stateless, all state is preserved in the run time inventory in the database. When the user requests a control loop operation (such as an instantiation, activation, passivation, or an ininitialization) the server broadcasts the request to participants over DMaaP and saves details of the request to the database. The server does not directly wait for responses to requests.
When a request is broadcast on DMaaP, the request is asynchronously picked up by participants of the types required for the control loop instance and those participants manage the life cycle of its control loop elements. Periodically, each participant reports back on the status of operations it has picked up for the control loop elements it controls, together with statistics on the control loop elements over DMaaP. On reception of these participant messages, the server stores this information to its database.
The server periodically runs a supervion function, which checks the status of all existing control loop instances and the status of outstanding requests. It builds a picture of the current status of each control loop instance from the reports on the elements of the control loop instances. Once the server has a full picture, it checks that the control loop instance is in the correct state as requested by the user of the system. If the control loop is not in the correct state, the supervision function can initiate actions such aas performing retries on operations or issuing alarms or notificaitons on control loop instances.
This approach makes it easy to scale control loop LCM. As control loop instance counts increase, more than one runtime server can be deployed and REST/supervision operations on control loop instances can run in parallel. The number of participants can scale because an asynchronous broadcast mechanism is used for server-participant communication and there is no direct connection or communication channel between participants and runtime servers. Participant state, control loop instance state, and control loop element state is held in the database, so any runtime server can handle operations for any participant. Because many participants of a particular type can be deployed and participant instances can load balance control loop element instances for different control loops of many types across themselves using a mechanism such as a Kubernetes cluster.
5: Goals
5.1: MVP
| Jira | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
5.2: ControlLoop in Tosca LCM Istanbul Jiras
| Jira | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
...
- Support design of multiple control loops*
- Support design of individual control loop component**
- Support composition of control loops**
...
- Participant registration and participant deregistration
- Support commissioning of control loops
- Ingestion with artifact references*
- Ingestion with artifact embedded**
- Support instantiation of control loop
- Support instantiation of control loop TOSCA to DMaaP MR*
- Support instantiation of config for the control loop*
- Support monitoring of control loops
- Receive control loop heartbeat events (heartbeat starts when component of control loop is running)*
- Support supervision of control loops
- Periodically check monitored data, and update state of control loop*
...
- Agent library*
- Reference(test) participant*
- CDS participant*
- DCAE participant*
- Policy participant*
Other Considerations
4.4.1 Upgrade
Performing a hot upgrade of the Control Loop at run time as well as handling an upgrade of the software in one or more of the participants in an Control Loop is a particularly challenging issue because upgrading must handle the following cases without tearing down the Control Loop:
- Upgrade and changes of the configuration data of participants
- Addition of or removal of participants in an Control Loop
- Upgrade of software in one or more participants in an Control Loop
- Maintenance of compatibility between participants when an update of more than one participant must be done together to ensure compatibility, for example, when a protocol being used by two participants to communicate is upgraded
4.4.2 Scalability
The system is designed to be inherently scalable. The control loop runtime server is stateless, all state is preserved in the run time inventory in the database. When the user requests a control loop operation (such as an instantiation, activation, passivation, or an ininitialization) the server broadcasts the request to participants over DMaaP and saves details of the request to the database. The server does not directly wait for responses to requests.
When a request is broadcast on DMaaP, the request is asynchronously picked up by participants of the types required for the control loop instance and those participants manage the life cycle of its control loop elements. Periodically, each participant reports back on the status of operations it has picked up for the control loop elements it controls, together with statistics on the control loop elements over DMaaP. On reception of these participant messages, the server stores this information to its database.
The server periodically runs a supervion function, which checks the status of all existing control loop instances and the status of outstanding requests. It builds a picture of the current status of each control loop instance from the reports on the elements of the control loop instances. Once the server has a full picture, it checks that the control loop instance is in the correct state as requested by the user of the system. If the control loop is not in the correct state, the supervision function can initiate actions such aas performing retries on operations or issuing alarms or notificaitons on control loop instances.
This approach makes it easy to scale control loop LCM. As control loop instance counts increase, more than one runtime server can be deployed and REST/supervision operations on control loop instances can run in parallel. The number of participants can scale because an asynchronous broadcast mechanism is used for server-participant communication and there is no direct connection or communication channel between participants and runtime servers. Participant state, control loop instance state, and control loop element state is held in the database, so any runtime server can handle operations for any participant. Because many participants of a particular type can be deployed and participant instances can load balance control loop element instances for different control loops of many types across themselves using a mechanism such as a Kubernetes cluster.
...