The idea of using control loops to automatically (or autonomously) perform network management has been the subject of much research in the Network Management research community, see this paper for some background. However, it is only with the advent of ONAP that we have a platform that supports control loops for network management. Before ONAP, Control Loops have been deployed by hard-coding components together and hard coding logic into components. ONAP has taken a step forward towards automatic deployment of Control Loops by allowing parameterization of Control Loops that work on the premise that the Control Loops use a set of analytic, policy, and control components connected together in set ways.
The goal of the work is to extend and enhance the current ONAP Control Loop support to provide a complete open-source framework for Control Loops. This will enhance the current support to provide TOSCA based Control Loop definition, development, deployment and run-time management. The participants that comprise a Control Loop and the metadata needed to link the participants together to create a Control Loop are specified in a standardized way using the OASIS TOSCA modelling language. The TOSCA description is then used to deploy and manage the Control Loops in the run time system.
The Design Time Catalogue contains the the metadata primitives for composition of control loops. As shown in the figure above, the Design time component provides a system where Control Loops can be designed and defined in metadata. This means that a Control Loop can have any arbitrary structure and the Control Loop developers can use whatever analytic, policy, or control primitives they like to implement their Control Loop. At deployment time, the user parameterises the Control Loop and deploys it. A design time catalogue is created. This catalogue contains the primitive metadata for any components that can be used to compose a Control Loop. A Control Loop SDK is used to compose a Control Loop by aggregating the metadata for the components chosen to be used in a Control Loop and by constructing the references between the components.
Once the Control Loop is composed, it is packaged (TOSCA uses a CSAR archive, a type of zip file). The package consists of the TOSCA file for the Control Loop together with any other artifacts that should be passed to components participating in a Control Loop when it is instantiated. The package is deployed to the run time part of the system, where it is stored in the run time catalogue and is available for instantiation.
When a user wishes to instantiate a Control Loop, they set values for the parameters of the Control Loop. Once the parameterization has been carried out, the Control Loop instantiated, with the metadata and whatever other artifacts are required being passed to the participants in the Control Loop. At runtime, the Control Loop can be monitored and analysed. It can also be updated as required and can be deleted when it is on longer needed.
The Control Loop Runtime Management will use ONAP services for non-functional aspects such as inventory, topology and data delivery.
A Control Loop service template is made up of several components, those which represent applications, those which represent dynamic config schemas, and the actual node_templates which makes up the loop itself.
Applications can be a DCAE microservice, an operational policy, or any other application as long as it can be modelled, and the targeted ecosystem to has a participant client waiting for the event distributions from CLAMP via DMaaP MR.
Dynamic config on the other hand can be a monitoring policy, or any other resource that provides config to parts of the loop, can be updated after the run time phase has started and is supported by the components hosting the applications in the control loop.
2.1: Control Loop TOSCA file definition
2.1.1 Application Definition
The application definition is truly dynamic, and as long as the participant that the application relates to, it can be anything, however, we have designed a base attribute, that's generic and a can act as a good starting point.
Application Node DefinitionExpand source
node_types:
org.onap.APP:
properties:
application_name:
type: string
description: Human readable name for the application.
required: true
provider:
type: string
description: Provider of the application and of the descriptor.
required: true
application_version:
type: string
description: Software version of the application.
required: true
resource_id:
type: string
description: >The ID of the resource,
should be provided if the resource was uploaded to the entity's inventory already.
required: false
resource_content:
type: string
description: the contents of the application resource, to be uploaded during deploy phase of loop.
required: false
monitoring_policy:
type: string
description: A reference to the monitoring policy if applicable.
required: false
version: 0.0.1
derived_from: tosca.nodes.Root
2.1.2 Loop Definition
The loop definition is implicit in the node_templates within the topology_template, any node contained within is expected to be a part of the control loop managed by CLAMP.
The below example doesn't explicitly include any order, ordering of deployment is to be considered in the future which likely would lead to changes to this
Loop DefinitionExpand source
topology_template:
inputs:
pm_docker_image:
type: string
description: docker image used for deploying the PMSH service.
required: true
operational_policy_name:
type: string
description: name of the operational policy to be triggered on MR when subscription is created.
required: true
outbound_topic_name:
type: string
description: Name of topic used for messages to policy.
required: true
feedback_topic_name:
type: string
description: Name of topic used for feedback from policy.
required: true
inventory_notification_topic_name:
type: string
description: Name of topic used for inventory inventory notifications.
required: true
pmsh_resource_id:
type: string
description: ID of the PMSH blueprint in the inventory.
required: true
node_templates:
org.onap.PM_Subscription_Handler:
type: org.onap.APP
properties:
application_name: PMSH
provider: DCAE
application_Version: '1.0.1'
resource_id: { get_input: pmsh_resource_id }
monitoring_policy: 'monitoring_policy_id'
application_config:
tag_version: { get_input: pm_docker_image }
operational_policy_name: { get_input: operational_policy_name }
pmsh_publish_topic_name: { get_input: outbound_topic_name }
policy_feedback_topic_name: { get_input: feedback_topic_name }
aai_notification_topic_name: { get_input: inventory_notification_topic_name }
2.1.3 Example
Example Control LoopExpand source
tosca_definitions_version: tosca_simple_yaml_1_3
node_types:
org.onap.APP:
properties:
application_name:
type: string
description: Human readable name for the application.
required: true
provider:
type: string
description: Provider of the application and of the descriptor.
required: true
application_version:
type: string
description: Software version of the application.
required: true
resource_id:
type: string
description: >The ID of the resource,
should be provided if the resource was uploaded to the entity's inventory already.
required: false
resource_content:
type: string
description: the contents of the application resource, to be uploaded during deploy phase of loop.
required: false
monitoring_policy:
type: string
description: A reference to the monitoring policy if applicable.
required: false
version: 0.0.1
derived_from: tosca.nodes.Root
topology_template:
inputs:
pm_docker_image:
type: string
description: docker image used for deploying the PMSH service.
required: true
operational_policy_name:
type: string
description: name of the operational policy to be triggered on MR when subscription is created.
required: true
outbound_topic_name:
type: string
description: Name of topic used for messages to policy.
required: true
feedback_topic_name:
type: string
description: Name of topic used for feedback from policy.
required: true
inventory_notification_topic_name:
type: string
description: Name of topic used for inventory inventory notifications.
required: true
pmsh_resource_id:
type: string
description: ID of the PMSH blueprint in the inventory.
required: true
node_templates:
org.onap.PM_Subscription_Handler:
type: org.onap.APP
properties:
application_name: PMSH
provider: DCAE
application_Version: '1.0.1'
resource_id: { get_input: pmsh_resource_id }
monitoring_policy: 'monitoring_policy_id'
application_config:
tag_version: { get_input: pm_docker_image }
operational_policy_name: { get_input: operational_policy_name }
pmsh_publish_topic_name: { get_input: outbound_topic_name }
policy_feedback_topic_name: { get_input: feedback_topic_name }
aai_notification_topic_name: { get_input: inventory_notification_topic_name }
2.2: Modelling from TOSCA to Deployment Data in Run Time Catalogue
2.3: Modelling from TOSCA to Instance Data Run Time Catalogue
Initial Swagger Draft for Deployment APIExpand source
openapi: 3.0.3
info:
description: >-
The ControlLoop Deployment API allows users to search and perform
CRUD action on Controlloop Tosca exported data sets.
version: 1.0.0
title: Control Loop Deployment API
license:
name: Apache 2.0
url: 'http://www.apache.org/licenses/LICENSE-2.0'
contact:
email: lego@est.tech
servers:
- url: '{scheme}://onap/controlloop/v2/'
variables:
scheme:
description: 'The Data Set API is accessible via https and http'
enum:
- 'https'
- 'http'
default: 'http'
tags:
- name: Control Loop
description: Deployment API
- name: design-catalogue
description: CRUD actions on an designtime Control Loop data sets.
- name: runtime-catalogue
description: CRUD actions on an runtime Control Loop data sets.
paths:
/design-catalogue/loops:
get:
tags:
- design-catalogue
operationId: getAllLoopsUsingGET
summary: Returns a list of all available Control Loops.
responses:
'200':
description: OK
content:
application/yaml:
schema:
type: array
items:
$ref: '#/components/schemas/ControlLoop'
/design-catalogue/loop/{controlLoopId}:
get:
tags:
- design-catalogue
summary: >-
Returns a specific control loop.
description: >-
This GET API returns a specific control loop.
operationId: getLoopByIdUsingGET
parameters:
- name: controlLoopId
in: path
description: 'Id of the control loop.'
required: true
example: "org.onap.PM_CDS_Blueprint"
schema:
type: string
responses:
'200':
description: OK
content:
application/yaml:
schema:
$ref: '#/components/schemas/ControlLoop'
delete:
tags:
- design-catalogue
summary: Delete a specific Control Loop by ID.
operationId: deleteLoopByIdUsingDELETE
parameters:
- name: controlLoopId
in: path
description: Identity for the control loop
required: true
style: simple
schema:
type: string
responses:
'200':
description: OK
'404':
description: Control Loop not present
post:
tags:
- design-catalogue
summary: >-
Accepts a Tosca Service Template representing a Control Loop.
description: >-
This API Create a Control Loop.
operationId: createLoopWithIDUsingPOST
parameters:
- name: controlLoopId
in: path
description: Identity for the control loop
required: true
style: simple
schema:
type: string
responses:
'200':
description: OK
'409':
description: Control Loop already exist.
put:
tags:
- design-catalogue
summary: Updates a Control Loop.
operationId: updateLoopWithIDUsingPUT
parameters:
- name: controlLoopId
in: path
description: Identity for the control loop
required: true
style: simple
schema:
type: string
responses:
'200':
description: OK
'404':
description: Control Loop not found
/runtime-catalogue/deploy:
post:
tags:
- runtime-catalogue
summary: >-
Deploys Tosca Service Template in RunTime Catalogue.
description: >-
This API Deploys Tosca Service Template in RunTime Catalogue DB.
operationId: deployLoopWithPOST
responses:
'200':
description: OK
'404':
description: No matching record found for the given criteria.
/runtime-catalogue/loop/{controlLoopId}:
get:
tags:
- runtime-catalogue
summary: >-
Returns a specific control loop.
description: >-
This GET API returns a specific control loop.
operationId: getLoopByIdUsing
parameters:
- name: controlLoopId
in: path
description: 'Id of the control loop.'
required: true
example: "org.onap.PM_CDS_Blueprint"
schema:
type: string
responses:
'200':
description: OK
content:
application/yaml:
schema:
type: array
items:
$ref: '#/components/schemas/ControlLoop'
delete:
tags:
- runtime-catalogue
summary: Delete a specific Control Loop by ID.
operationId: deleteLoopByIdUsing
parameters:
- name: controlLoopId
in: path
description: Identity for the control loop
required: true
style: simple
schema:
type: string
responses:
'200':
description: OK
'404':
description: Control Loop not present
post:
tags:
- runtime-catalogue
summary: >-
Accepts a Tosca Service Template representing a Control Loop.
description: >-
This API Create a Control Loop.
operationId: createLoopWithIDUsing
parameters:
- name: controlLoopId
in: path
description: Identity for the control loop
required: true
style: simple
schema:
type: string
responses:
'200':
description: OK
'409':
description: Control Loop already exist.
put:
tags:
- runtime-catalogue
summary: Updates a Control Loop.
operationId: updateLoopWithIDUsing
parameters:
- name: controlLoopId
in: path
description: Identity for the control loop
required: true
style: simple
schema:
type: string
responses:
'200':
description: OK
'404':
description: Control Loop not found
components:
schemas:
ControlLoop:
title: ControlLoop
type: object
properties:
id:
type: string
ControlLoop:
type: string
3.1.2: Deployment Sequence Diagrams
GET, DELETE, CREATE API ControlLoop Sequence Diagram
This section refers to Instantiation of a Deployed control loop. A client, in this case CLAMP (potentially DCAEMOD, etc in the future) will render the deployed control loops allowing selection of a particular control loop to be instantiated. User will then provide the configurations needed to instantiate the selected control loop which will be sent onto the CL_Instance_Control Service. The service will then distribute the configurations to DmaaP topic. Participants (agents) will pull the event containing the config and pick out their control loop components to be instantiated and start/set up those particular components. CL_Instance_Control Service will be waiting for a response back from all participants involved in the instantiation of the control loop, in regards to the state of instantiation. In successful response case the service will store the CL Instance LCM data into the runtime DB as well as providing a message back to the client of the successful instantiation. In failure to receive the response case, a timeout will be called, which will result in a teardown event being sent to DmaaP. The participants will then receive the event and proceed to teardown the components that were instantiated or check that they have failed to instantiate in the first place and send a Teardown ACK back to the CL_Instance_Control Service. No CL Instance LCM data will be stored and a message indicating failure to instantiate the CL along side with the error will be sent back to the client (CLAMP).
3.2.1: Instantiation REST API
Initial Swagger for Initiation APIExpand source
openapi: 3.0.3
info:
title: Swagger Control Loop Instantiation
description: ""
version: 1.0.0
servers:
- url: '{scheme}://onap/controlloop/v2/'
variables:
scheme:
description: 'The Data Set API is accessible via https and http'
enum:
- 'https'
- 'http'
default: 'http'
tags:
- name: instantiate
description: Control Loop Instantiation
paths:
/runtime-catalogue/instantiate:
post:
tags:
- instantiate
summary: Instantiate a deployed control loop
operationId: instantiateLoop
requestBody:
description: Control loop that is to be instantiated
content:
application/json:
schema:
$ref: '#/components/schemas/ControlLoop'
required: true
responses:
200:
description: OK
content:
application/json:
schema:
type: array
items:
$ref: '#/components/schemas/ApiResponse'
404:
description: No Control Loop found
405:
description: Invalid input
/runtime-catalogue/loops/instantiated:
get:
tags:
- instantiate
summary: Get a list of instantiated control loops
responses:
200:
description: Success
content:
application/json:
schema:
type: array
items:
$ref: '#/components/schemas/ControlLoop'
404:
description: No instantiated control loops found
/runtime-catalogue/loops/instantiated/{loop-id}:
get:
tags:
- instantiate
summary: Get an instantiated control loop by id
parameters:
- name: loop-id
in: path
description: ID of instantiated loop to return
required: true
schema:
type: string
responses:
200:
description: OK
content:
application/json:
schema:
$ref: '#/components/schemas/ControlLoop'
404:
description: Control Loop not found
put:
tags:
- instantiate
summary: Update an instantiated control loop
parameters:
- name: loop-id
in: path
description: ID of instantiated loop
required: true
schema:
type: string
requestBody:
description: Control loop to be updated
content:
application/json:
schema:
$ref: '#/components/schemas/ControlLoop'
required: true
responses:
200:
description: OK
content:
application/json:
schema:
$ref: '#/components/schemas/ControlLoop'
404:
description: Control Loop not found
delete:
tags:
- instantiate
summary: Delete an instantiated control loop
parameters:
- name: loop-id
in: path
description: ID of instantiated loop
required: true
schema:
type: string
responses:
200:
description: OK
404:
description: Control Loop not found
components:
schemas:
ControlLoop:
title: ControlLoop
type: object
properties:
id:
type: string
controlLoopTosca:
type: string
ApiResponse:
type: object
properties:
code:
type: integer
format: int32
status:
type: string
3.2.2: Instantiation Sequence Diagrams
3.2.3: Instantiation DMaaP API
Initial Thought for an event to be sent from CL_Instance_Control onto DmaaP for Participants to consume. The event would go onto an output topic which the Participants would be polling/subscribed to
Preferred solution is to send TOSCA in the body. Meaning we could reuse the parsing code which is already present and provide it to the participant.
3.2.4: Instantiation Participant API
*Suggestion was to put JAVA API code in this section for the participant talking to DmaaP. TBD
3.3: Monitoring
In this case it refers to monitoring the data that participants will provide to DmaaP. Participants will send events to DmaaP which will be monitored by the CL_Monitoring_Service. The data provided should include a reference id to the control loops that are instantiated on the participant, as well as the applications that have been instantiated as a part of that control loop for that participant. Data should also include the time that the application has started, state of it (running/terminated) and any other critical information which would help to determine the health of an instantiated control loop and its components. Idea is for the participant to provide events every certain period of time, similar to a health-check, in order to provide consistent monitoring.
3.3.1: Monitoring REST API
3.3.2: Monitoring Sequence Diagrams
3.3.3: Monitoring DMaaP API
3.3.4: Monitoring Participant API
4: Design
4.1: Server Side
4.1.1 Database Schema and JPA
4.1.2: TOSCA Processing
4.1.3: Instance Control
4.1.4: Execution Monitoring
4.2: Participant Side
4.2.1: Participant Agent
4.2.2: Policy Participant Agent
4.2.3: CDS Participant Agent
4.2.4: DCAE Participant Agent
4.2.5: SO Participant Agent
4.3: Client Side
4.3.1: Client SDK: Composition of Control Loop Tosca
4.3.2: Client User Interface
4.4 Other Considerations
4.4.1 Upgrade
Performing a hot upgrade of the Control Loop at run time as well as handling an upgrade of the software in one or more of the participants in an Control Loop is a particularly challenging issue because upgrading must handle the following cases without tearing down the Control Loop:
Upgrade and changes of the configuration data of participants
Addition of or removal of participants in an Control Loop
Upgrade of software in one or more participants in an Control Loop
Maintenance of compatibility between participants when an update of more than one participant must be done together to ensure compatibility, for example, when a protocol being used by two participants to communicate is upgraded