You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 67 Next »

The idea of using control loops to automatically (or autonomously) perform network management has been the subject of much research in the Network Management research community, see this paper for some background. However, it is only with the advent of ONAP that we have a platform that supports control loops for network management. Before ONAP, Control Loops have been implemented by hard-coding components together and hard coding logic into components. ONAP has taken a step forward towards automatic implementation of Control Loops by allowing parameterization of Control Loops that work on the premise that the Control Loops use a set of analytic, policy, and control components connected together in set ways.

The goal of the work is to extend and enhance the current ONAP Control Loop support to provide a complete open-source framework for Control Loops. This will enhance the current support to provide TOSCA based Control Loop definition and development, commissioning and run-time management. The participants that comprise a Control Loop and the metadata needed to link the participants together to create a Control Loop are specified in a standardized way using the OASIS TOSCA modelling language. The TOSCA description is then used to commission, instantiate, and manage the Control Loops in the run time system.

1 Terminology

Control Loop Type: A definition of a Control Loop in the TOSCA language. This definition describes a certain type of a control loop. The life cycle of instances of a Control Loop Type are managed by CLAMP.

Control Loop Instance: An instance of a Control Loop Type. The life cycle of a Control Loop Instance is managed by CLAMP. A Control Loop Instance is a set of executing elements on which Life Cycle Management (LCM) is executed collectively. For example, a set of microservices may be spawned and executed together to deliver a service. This collection of services is a control loop.

Control Loop Element Type: A definition of a Control Loop Element in the TOSCA language. This definition describes a certain type of Control Loop Element for a control loop in a Control Loop Type.

Control Loop Element Instance: A single entity executing on a participant, with its Life Cycle being managed as part of the overall control loop. For example, a single microservice that is executing as one microservice in a service.

CLAMP Control Loop Runtime: The CLAMP server that holds Control Loop Type definitions and manages the life cycle of Control Loop Instances and their Control Loop Elements in cooperation with participants.

Participant Type: Definition of a type of system or framework that can take part in control loops and a definition of the capabilities of that participant type. A participant advertises its type to the CLAMP Control Loop Runtime.

Participant: A system or framework that takes part in control loops by executing Control Loop Elements in cooperation with the CLAMP Control Loop Runtime. A participant chooses to partake in control loops, to manage Control Loop Elements for CLAMP, and to receive, send and act on LCM messages for the CLAMP runtime.

Property: Metadata defined in TOSCA that is associated with a Control Loop, a Control Loop Element, or a Participant.

TOSCA Property Type: The TOSCA defintion of the type of a property. A property can have a generic type such as string or integer or can have a user defined TOSCA data type.

TOSCA Property Value: The value of a Property Type. Property values are assigned at run time in CLAMP.

Common Property Type: Property Types that apply to all instances of a Control Loop Type.

Common Property Value: The value of a Property Type. It is assigned at run time once for all instances of a Control Loop Type.

Instance Specific Property Type: Property Types that apply to an individual instance of a Control Loop Type.

Instance Specific Property Value: The value of a Property Type that applies to an individual instance of a Control Loop Type. The value is assigned at run time for each control loop instance.

2 Capabilities

We consider the capabilities of Control Loops at Design Time and Run Time.

At Design Time, two capabilities are supported:

  1. Control Loop Element Definition. This capability allows external users and systems (such as SDC or DCAE-MOD) to define the metadata that can be used on and configured on a Control Loop Element Type and to define the Participant Type that will run the Control Loop Element when it is taking part in in a control loop. The post condition of an execution of this capability is that metadata for a Control Loop Element Type is defined in the Control Loop Design Time Catalogue.
  2. Control Loop Type Definition. This capability allows users and other systems to create Control Loop Type definitions by connecting a chain of participants together from the participants that are available in the Control Loop Design Time Catalogue. In an execution of this capability, a user defines the control loop chain for the Control Loop Type and defines the connections between participants. The user also selects the correct metadata sets for each participant in the Control Loop Type and defines the overall Control Loop Type metadata. The user also specifies the Common Property Types that apply to all instances of a control loop type and the Instance Specific Property Types that apply to individual instances of a Control Loop Type. The post condition for an execution of this capability is a Control Loop definition in TOSCA stored in the Control Loop Design Time Catalogue.

At Run Time, the following capabilities are supported:

  1. Participant Registration. This capability allows participants to register and deregister with CLAMP. The post condition for an execution of this capability is that a participant is available for participation in a control loop. Participants can explicitly register with CLAMP at any point after they start, or they can implicitly register when they respond to a control loop initiation request.
  2. Control Loop Commissioning. This capability allows version controlled Control Loop Type definitions to be taken from the Control Loop Design Time Catalogue and be placed in the Control Loop Run Time Inventory. It also allows the values of Common Property Types that apply to all instances of a Control Loop Type to be set. The post condition for an execution of this capability is that the Control Loop definition is in the Control Loop Run Time Inventory.
  3. Control Loop Priming on Participants. A participant is primed to support a Control Loop Type. The definition of a control loop and the values of Common Property Types that apply to all instances of a control loop type on a participant is sent to a participant. The participant can then take whatever actions it need to do to support the control loop type in question. Control Loop Priming can take place explicitly as a separate operation on participants or can be done implicitly in the instantiation message for the first instance of a Control Loop Type. The post condition for an execution of this capability is that all participants in this control loop type are commissioned, that is they are prepared to run instances of this control loop type.
  4. Control Loop Life Cycle Management. This capability allows a Control Loop Instance to be created. The Control Loop Type definition is read from the Control Loop Run Time Inventory and values are assigned to the Instance Specific Property Types defined for instances of the Control Loop Type in the same manner as the existing CLAMP client does. A Control Loop Instance that has been created but has not yet been instantiated on participants is in state UNINITIALIZED. In this state, the Instance Specific Property Type values can be revised and updated as often as the user requires. Once the user is happy with the property values, the Control Loop Instance is instantiated on participants and the Control Loop Elements for this Control Loop Instance are started by participants using the control loop metadata. Once the Control Loop Instance is instantiated on each participant, the Control Loop Instance is set as being in state PASSIVE in the Control Loop Run Time Inventory. The user can now order the participants to change the state of the Control Loop Instance to state RUNNING. Each participant begins accepting and processing control loop events and the Control Loop Instance is set to state RUNNING in the control loop inventory. The post condition for an execution of this capability is that the Control Loop instance is running on participants and is processing events.
  5. Control Loop Monitoring. This capability allows Control Loop Instances to be monitored. Users can check the status of Participants, Control Loop Instances, and Control Loop Elements. Participants report their overall status and the status of Control Loop Elements they are running periodically to CLAMP. Clamp aggregates these status reports into an aggregated Control Loop Instance status record, which is available for monitoring. The post condition for an execution of this capability is that Control Loop Instances are being monitored.

2.1 Control Loop Instance States

When a control loop definition has been commissioned, instances of the control loop can be created, updated, and deleted. The system manages the lifecycle of control loops and control loop elements following the state transition diagram below.

3 Overall Target Architecture

The diagram below shows an overview of the architecture of TOSCA based Control Loop Management in CLAMP.

Following the ONAP Reference Architecture, the architecture has a Design Time part and a Runtime part.

The Design Time part of the archtiecture allows a user to specify metadata for participants. It also allows users to compose control loops. The Design Time Catalogue contains the metadata primitives and control loop definition primitives for composition of control loops. As shown in the figure above, the Design Time component provides a system where Control Loops can be designed and defined in metadata. This means that a Control Loop can have any arbitrary structure and the Control Loop developers can use whatever analytic, policy, or control participants they like to implement their Control Loop. At composition time, the user parameterises the Control Loop and stores it in the design time catalogue. This catalogue contains the primitive metadata for any participants that can be used to compose a Control Loop. A Control Loop SDK is used to compose a Control Loop by aggregating the metadata for the participants chosen to be used in a Control Loop and by constructing the references between the participants. The architecture of the Control Loop Design Time part will be elaborated in future releases.

Composed Control Loops are commissioned on the run time part of the system, where they are stored in the run time inventory and are available for instantiation. The Commissioning component provides a CRUD REST interface for Control Loop Types, and implements CRUD of Control Loop Types. Commissioning also implements validation and persistence of incoming Control Loop Types. It also guarantees the integrity of updates and deletions of Control Loop Types, such as performing updates accordance with semantic versioning rules and ensuring that deletions are not allowed on Control Loop Types that have instances defined.

The Instantiation component manages the Life Cycle Management of Control Loop Instances and their Control Loop Elements. It publishes a REST interface that is used to create Control Loop Instances and set values for Common and Instance Specific properties. This REST interface is public and is used by the CLAMP GUI. It may also be used by any other client via the public REST interface. the REST interface also allows the state of Control Loop Instances to be changed. A user can change the state of Control Loop Instances as described in the state transition diagram shown in section 2 above. The Instantiation component issues update and state change messages via DMaaP to participants so that they can update and mange the state of the Control Loop Elements they are responsible for. The Instantiation component also implements persistence of Control Loop Instances, control loop elements, and their state changes.

The Monitoring component reads updates sent by participants. Participants report on the state of their Control Loop Elements periodically and in response to a message they have received from the Instantiation component. The Monitoring component reads the contents of the participant messages and persists their state updates and statistics records. It also publishes a REST interface that publishes the current state of all Participants, Control Loop Instances and their Control Loop Elements, as well as publishing Participant and Control Loop statistics.

The Supervision component is responsible for checking that Control Loop Instances are correctly instantiated and are in the correct state (UNINITIALIZED/READY/RUNNING). It also handles timeouts and on state changes to Control Loop Instances, and retries and rolls back state changes where state changes failed.

A Participant is an executing component that partakes in control loops. More explicitly, a Participant is something that implements the Participant Instantiation and Participant Monitoring messaging protocol over DMaaP for Life Cycle management of Control Loop Elements. A Participant runs Control Loop Elements and manages and reports on their life cycle following the instructions it gets from the CLAMP runtime in messages delivered over DMaaP.

In the figure above, five participants are shown. A Configuration Perisistence Participant manages Control Loop Elements that interact with the ONAP Configuration Persistence Service to store common data. The DCAE Participant runs Control Loop Elements that manage DCAE microservices. The Kubernetes Participant hosts the Control Loop Elements that are managing the life cycle of microservices in control loops that are in a Kubernetes ecosystem. The Policy Participant handles the Control Loop Elements that interact with the Policy Framework to manage policies for control loops. A Controller Participant such as the CDS Participant runs Control Loop Elements that load metadata and configure controllers so that they can partake in control loops. Any third party Existing System Participant can be developed to run Control Loop Elements that interact with any existing system (such as an operator's analytic, machine learning, or artificial intelligence system) so that those systems can partake in control loops.

4 System Level Dialogues

The CLAMP Control Loop Lifecycle Management uses the following system level dialogues

4.1 Commissioning Dialogues

Commissioning dialogues are used to commission and decommission Control Loop Type definitions and to set the values of Common Parameters.

4.1.1 Commissioning a Control Loop Type Definition using the CLAMP GUI

CLAMP_GUI CLAMP_GUI CLAMP_Runtime CLAMP_RuntimeDatabase Database [REST] Commission Control Loop Type Definition Store Control Loop Type Definition [REST] Type Definition Stored

4.1.2 Commissioning a Control Loop Type Definition using SDC

SDC SDC Policy_Distribution Policy_Distribution CLAMP_Runtime CLAMP_RuntimeDatabase Database [DMaaP] CSAR containing Control Loop Type Definition [REST] Commission Control Loop Type Definition Store Control Loop Type Definition [REST] Type Definition Stored [DMaaP] Deployment Completed

4.1.3 Setting Common Parameters for a Control Loop Type Definition

CLAMP_GUI CLAMP_GUI CLAMP_Runtime CLAMP_Runtime Participant ParticipantDatabase Database [REST] Read Control Loop Type Definitions Read Control Loop Type Definition [REST] Type Definitions returned Select Control Loop Type Definition for Parameterization alt[Control Loop Type Definition not Commissioned on Participants] [REST] Fetch GUI fields Common Parameter Definitions Read Control Loop Type Definition Generate fields for Common Parameters for GUI [REST] Return GUI fields Common Parameter Definitions Render UI for Common Parameters Get values for Common Parameters from user [REST] Set values for Common Parameters Store values for Common Parameters [REST] Common Parameter values stored [Control Loop Type Definition already Commissioned on Participants] [REST] Cannot set Common Parameters onControl Loop Type Defintion that hasbeen commissioned on participants

4.1.3 Commissioning a Control Loop Type Definition on Participants

CLAMP_GUI CLAMP_GUI CLAMP_Runtime CLAMP_Runtime Participant ParticipantDatabase Database Participants Participants [REST] Read Control Loop Type Definitions Read Control Loop Type Definition [REST] Type Definitions returned Select Control Loop Type Definition for Commissioning on Participants alt[Control Loop Type Definition not Commissioned on Participants] [REST] Commission Control Loop Type Definition on Participants Read Control Loop Type Definition Read Common Parameter Values for Control Loop Type Definition alt[Common Parameter values set for Control Loop Type Definition] [DMaaP] Update Participants with Control Loop Type Definition and values for Common Parameters [REST] Control Loop Type Definition commissioning ordered on participants [Control Loop Type Definition already Commissioned on Participants] [REST] Common Parameters not seton Control Loop Type Defintion [Control Loop Type Definition already Commissioned on Participants] [REST] Cannot set Common Parameters onControl Loop Type Defintion that hasbeen commissioned on participants Participant Responses [DMaaP] Result of Update with Control Loop Type Definition and values for Common Parameters Store result of Control Loop Type Definition Update on Participant Supervision Read Control Loop Type Definitions alt[Control Loop Type Definition Participant Commission underway] read result of Control Loop Type Definition Updates from Participant alt[Updates completed on all participants] set Control Loop Type Definition as Commissioned on Participants [Updates not completed] alt[Commissioning of Control Loop Type Definition on participants timed out] set Control Loop Type Definition as timed out on Participants Log error [Wait for updates to complete] [Control Loop Type Definition Participant Commission not underway]

4.1.4 Supervision of Commissioning a Control Loop Type Definition on Participants


4.1.5 Decommissioning a Control Loop Type Definition on Participants

This dialogue allows common parameter values to be altered.

4.1.6 Decommissioning a Control Loop Type Definition in CLAMP

4.1.7 Reading Commissioned Control Loop Type Definitions


5. Life Cycle Management Operations

5.1 Control Loop Version Management

Performing a hot change of a Control Loop Instance at run time is a particularly challenging issue because a change must handle the following cases without tearing down the Control Loop:

  • Changes of the configuration data of Control Loop Instances
  • Coping with changes in Control Loop Definitions

In order to keep version management straightforward and easy to implement, the following version management scheme using semantic versioning is implemented. Each Control Loop Instance and Control Loop Element has a semantic version with 3 digits indicating the major.minor.patch number of the version.

Change constraints:

  1. A Control Loop or Control Loop Element in state RUNNING can be changed to a higher patch level or rolled back to a lower patch level. This means that hot changes that do not impact the structure of a Control Loop or its elements can be executed.
  2. A Control Loop or Control Loop Element in state PASSIVE can be changed to a higher minor/patch level or rolled back to a lower minor/patch level. This means that structural changes to Control Loop Elements that do not impact the Control Loop as a whole can be executed by taking the control loop to state PASSIVE.
  3. A Control Loop or Control Loop Element in state UNINITIALIZED can be changed to a higher major/minor/patch level or rolled back to a lower major/minor/patch level. This means that where the structure of the entire control loop is changed, the control loop must be uninitialized and reinitialized.
  4. If a Control Loop Element has a minor version change, then its Control Loop Instance must have at least a minor version change.
  5. If a Control Loop Element has a major version change, then its Control Loop Instance must have a major version change.

5.2 Scalability

The system is designed to be inherently scalable. The CLAMP runtime is stateless, all state is preserved in the run time inventory in the database. When the user requests an operation such as an instantiation, activation, passivation, or an uninitialization on a Control Loop Instance, the CLAMP runtime broadcasts the request to participants over DMaaP and saves details of the request to the database. The CLAMP runtime does not directly wait for responses to requests.

When a request is broadcast on DMaaP, the request is asynchronously picked up by participants of the types required for the Control Loop Instance and those participants manage the life cycle of its control loop elements. Periodically, each participant reports back on the status of operations it has picked up for the Control Loop Elements it controls, together with statistics on the Control Loop Elements over DMaaP. On reception of these participant messages, the CLAMP runtime stores this information to its database.

The CLAMP runtime periodically runs a supervision function, which checks the status of all existing Control Loop Instances and the status of outstanding requests. It builds a picture of the current status of each Control Loop Instance from the reports on the elements of the Control Loop Instances. Once the CLAMP runtime has a full picture, it checks that each Control Loop Instance is in the correct state as requested by the user of the system. If a Control Loop Instance is not in the correct state, the supervision function can initiate actions such as performing retries on operations or issuing alarms or notifications on that Control Loop Instance.

This approach makes it easy to scale Control Loop life cycle management. As Control Loop Instance counts increase, more than one CLAMP runtime can be deployed and REST/supervision operations on Control Loop Instances can run in parallel. The number of participants can scale because an asynchronous broadcast mechanism is used for runtime-participant communication and there is no direct connection or communication channel between participants and CLAMP runtime servers. Participant state, Control Loop Instance state, and Control Loop Element state is held in the database, so any CLAMP runtime server can handle operations for any participant. Because many participants of a particular type can be deployed and participant instances can load balance control loop element instances for different Control Loop Instances of many types across themselves using a mechanism such as a Kubernetes cluster.

6 APIs and Protocols

The APIs and Protocols used by CLAMP for Control Loops are described on the pages below:

  1. Defining Control Loops in TOSCA for CLAMP
  2. REST APIs for CLAMP Control Loops
  3. The CLAMP Control Loop Participant Protocol

7 Design and Implementation

The design and implementation of TOSCA Control Loops in CLAMP is described for each executable entity on the pages below:

  1. The CLAMP Runtime Server
  2. CLAMP Participants
  3. The CLAMP GUI
  4. Building and running CLAMP
  5. Testing CLAMP







  • No labels