- Created by Alexis de Talhouët, last modified on Aug 19, 2019
You are viewing an old version of this page. View the current version.
Compare with Current View Page History
« Previous Version 99 Next »
In Dublin release, the CDS community has contributed a framework to automate the resolution of resources for instantiation and any config provisioning operation, such as day0, day1 or day2 configuration.
The content of the CBA Package is driven from a catalog of reusable data dictionary, component and workflow, delivering a reusable and simplified self service experience.
TOSCA based JSON formatted model following standard: http://docs.oasis-open.org/tosca/TOSCA-Simple-Profile-YAML/v1.2/csd01/TOSCA-Simple-Profile-YAML-v1.2-csd01.html
Most of the TOSCA modeled entity presented in the bellow documentation can be found here:Â https://github.com/onap/ccsdk-cds/tree/master/components/model-catalog/definition-type/starter-type
Tosca Model Reference:Â
Controller Blueprint Archive (.cba)
The Controller Blueprint Archive is the overall service design, fully model-driven, intent based package needed for SELF SERVICE provisioning and configuration management automation.
The CBA is .zip file, comprised of the following folder structure, the files may vary:
├── Definitions │ ├── blueprint.json Overall TOSCA service template (worfklow + node_template │ ├── artifact_types.json (generated) │ ├── data_types.json (generated) │ ├── node_types.json (generated) │ ├── relationship_types.json (generated) │ └── resources_definition_types.json (generated based on mapping files) ├── Environments ├── Plans Contains Directed Graph ├── Scripts Contains scripts ├── TOSCA-Metadata │ └── TOSCA.meta Meta-data of overall package └── Templates Contains combination of mapping and template
Dynamic API
The nature of the API request and response is meant to be model driven and dynamic. They both share the same definition.
The actionName
, under the actionIdentifiers
refers to the name of a Workflow (see workflow)
The content of the payload
is what is fully dynamic / model driven.
The first top level element will always be either $actionName-request
for a request or $actionName-response
for a response.
Then the content within this element is fully based on the workflow input and output.
Here is how the a generic request and response look like.
{ "commonHeader": { "originatorId": "", "requestId": "", "subRequestId": "" }, "actionIdentifiers": { "blueprintName": "", "blueprintVersion": "", "actionName": "", "mode": "" }, "payload": { "$actionName-request": { } } }
{ "commonHeader": { "originatorId": "", "requestId": "", "subRequestId": "" }, "actionIdentifiers": { "blueprintName": "", "blueprintVersion": "", "actionName": "", "mode": "" }, "payload": { "$actionName-response": { } } }
Enrichment
Once the base CBA zip file is done, the enrichment process will complete the design by providing all the definitions required for the self-service provisioning.
The following shows 2 ways to run CBA enrichment
- REST API request
- CDS UI
Flexible Plug-in
Interaction with external systems is made plug-able, removing development cycle to support new endpoint.
Currently, REST or SQL external systems are supported.
An external system might be used by multiple resources, or by multiple scripts.
In order to share the external system information, TOSCA provides a way to create macros using dsl_definitions:
Expression
TOSCA provides for a set of functions to reference elements within the template or to retrieve runtime values.
Below is a list of supported expressions
get_input
The get_input function is used to retrieve the values of properties declared within the inputs section of a TOSCA Service Template.
get_property
The get_property function is used to retrieve property values between modelable entities defined in the same service template.
get_attribute
The get_attribute function is used to retrieve the values of named attributes declared by the referenced node or relationship template name.
get_operation_output
The get_operation_output function is used to retrieve the values of variables exposed / exported from an interface operation.
get_artifact
The get_artifact function is used to retrieve artifact location between modelable entities defined in the same service template.
Data type
Represents the schema of a specific type of data.
Supports both primitive and complex data types:
Primitive | Complex |
---|---|
|
|
For complex data type, and entry_schema can be required, defining the type of value contained within the complex type (list / map)
Users can create as many data type as needed.
List of existing data type: https://github.com/onap/ccsdk-cds/tree/master/components/model-catalog/definition-type/starter-type/data_type
TOSCA definition: http://docs.oasis-open.org/tosca/TOSCA-Simple-Profile-YAML/v1.2/csd01/TOSCA-Simple-Profile-YAML-v1.2-csd01.html#_Toc494454187
Bellow is a list of existing data types
datatype-resource-assignment
Used to define entries within artifact-mapping-resource
That datatype represent a resource to be resolved. We also refer this as an instance of a data dictionary as it's directly linked to its definition.
Property | Description |
---|---|
property | Defines how the resource looks like (see datatype-property) |
input-param | Whether the resource can be provided as input. |
dictionary-name | Reference to the name of the data dictionary (see Data Dictionary). |
dictionary-source | Reference the source to use to resolve the resource (see Resource source). |
dependencies | List of dependencies required to resolve this resource. |
updated-date | Date when mapping was upload. |
updated-by | Name of the person that updated the mapping. |
{ "version": "1.0.0", "description": "This is Resource Assignment Data Type", "properties": { "property": { "required": true, "type": "datatype-property" }, "input-param": { "required": true, "type": "boolean" }, "dictionary-name": { "required": false, "type": "string" }, "dictionary-source": { "required": false, "type": "string" }, "dependencies": { "required": true, "type": "list", "entry_schema": { "type": "string" } }, "updated-date": { "required": false, "type": "string" }, "updated-by": { "required": false, "type": "string" } }, "derived_from": "tosca.datatypes.Root" }
datatype-property
Used to defined the property entry of a resource assignment.
Property | Description |
---|---|
type | Whether it's a primitive type, or a defined data-type |
description | Description of for the property |
required | Whether it's required or not |
default | If there is a default value to provide |
entry_schema | If the type is a complex one, such as list, define what is the type of element within the list. |
{ "version": "1.0.0", "description": "This is Entry point Input Data Type, which is dynamic datatype, The parameter names will be populated during the Design time for each inputs", "properties": { "type": { "required": true, "type": "string" }, "description": { "required": false, "type": "string" }, "required": { "required": false, "type": "boolean" }, "default": { "required": false, "type": "string" }, "entry_schema": { "required": false, "type": "string" } }, "derived_from": "tosca.datatypes.Root" }
Artifact Type
Represents the type of a file, used to identify the implementation of the functionality supporting this type of artifact.
TOSCA definition: http://docs.oasis-open.org/tosca/TOSCA-Simple-Profile-YAML/v1.2/csd01/TOSCA-Simple-Profile-YAML-v1.2-csd01.html#_Toc494454213
The tosca.artifacts.Implementation
was created, derived from tosca.artifacts.Root
{ "description": "TOSCA base type for implementation artifacts", "version": "1.0.0", "derived_from": "tosca.artifacts.Root" }
Bellow is a list of supported artifact types
artifact-template-velocity
Represents an Apache Velocity template.
Apache Velocity allows you to insert logic (if / else / loops / etc) when processing the output of a template/text.
File must have .vtl extension.
A template can represent anything, such as device config, payload to interact with 3rd party systems, resource-accumulator template, etc...
Often a template will be parameterized, and each parameter needs to be defined within an mapping file.
Velocity reference document: http://velocity.apache.org/engine/1.7/user-guide.html
{ "description": " Velocity Template used for Configuration", "version": "1.0.0", "file_ext": [ "vtl" ], "derived_from": "tosca.artifacts.Implementation" }
artifact-mapping-resource
This type is meant to represent mapping files defining the contract of each resource to be resolved.
Each parameter in a template must have a corresponding mapping definition, modeled using datatype-resource-assignment.
Hence the mapping file is meant to be a list of entries defined using datatype-resource-assignment.
File must have .json extension.
{ "description": "Resource Mapping File used along with Configuration template", "version": "1.0.0", "file_ext": [ "json" ], "derived_from": "tosca.artifacts.Implementation" }
artifact-directed-graph
Represents a directed graph.
This is to represent a workflow logic.
File must have .json or .xml extension.
{ "description": "Directed Graph File", "version": "1.0.0", "file_ext": [ "json", "xml" ], "derived_from": "tosca.artifacts.Implementation" }
Node type
Represents a re-usable entity used to define a manageable software component.
TOSCA definition: http://docs.oasis-open.org/tosca/TOSCA-Simple-Profile-YAML/v1.2/csd01/TOSCA-Simple-Profile-YAML-v1.2-csd01.html#_Toc494454215
node_template
Represents an instance of the software component, identify by its node type.
Bellow is a list of supported node types
tosca.nodes.VNF
Identifies a VNF, can be used to correlate any type of VNF related information.
{ "description": "This is VNF Node Type", "version": "1.0.0", "derived_from": "tosca.nodes.Root" }
vnf-netconf-device
Represents the VNF information to establish a NETCONF communication.
{ "description": "This is VNF Device with Netconf Capability", "version": "1.0.0", "capabilities": { "netconf": { "type": "tosca.capabilities.Netconf", "properties": { "login-key": { "required": true, "type": "string", "default": "sdnc" }, "login-account": { "required": true, "type": "string", "default": "sdnc-tacacs" }, "source": { "required": false, "type": "string", "default": "npm" }, "target-ip-address": { "required": true, "type": "string" }, "port-number": { "required": true, "type": "integer", "default": 830 }, "connection-time-out": { "required": false, "type": "integer", "default": 30 } } } }, "derived_from": "tosca.nodes.Vnf" }
tosca.nodes.ResourceSource
Identifies a resource source, used during resource resolution.
{ "description": "TOSCA base type for Resource Sources", "version": "1.0.0", "derived_from": "tosca.nodes.Root" }
dg-generic
Identifies a Directed Graph used as imperative workflow.
{ "description": "This is Generic Directed Graph Type", "version": "1.0.0", "properties": { "content": { "required": true, "type": "string" }, "dependency-node-templates": { "required": true, "description": "Dependent Step Components NodeTemplate name.", "type": "list", "entry_schema": { "type": "string" } } }, "derived_from": "tosca.nodes.DG" }
tosca.nodes.Component
This is the root level for all component functionalities, represents a functionality.
{ "description": "This is default Component Node", "version": "1.0.0", "derived_from": "tosca.nodes.Root" }
Bellow is a list of supported components
component-jython-executor
Used to execute python scripts through Java.
See more about Jython: https://www.jython.org/
{ "description": "This is Jython Execution Component.", "version": "1.0.0", "capabilities": { "component-node": { "type": "tosca.capabilities.Node" } }, "interfaces": { "ComponentJythonExecutor": { "operations": { "process": { "inputs": { "instance-dependencies": { "required": true, "description": "Instance Names to Inject to Jython Script.", "type": "list", "entry_schema": { "type": "string" } }, "dynamic-properties": { "description": "Dynamic Json Content or DSL Json reference.", "required": false, "type": "json" } }, "outputs": { "response-data": { "description": "Execution Response Data in JSON format.", "required": false, "type": "string" }, "status": { "description": "Status of the Component Execution ( success or failure )", "required": true, "type": "string" } } } } } }, "derived_from": "tosca.nodes.component.Jython" }
component-resource-resolution
Used to perform resolution of resources.
Makes use of artifact-mapping-resource and artifact-template-velocity
Will store to resolution result as an attribute in the workflow context call assignment-params. Using the get_attribute expression, this attribute can be retrieve to be provide as workflow output (see Workflow).
{ "description": "This is Resource Assignment Component API", "version": "1.0.0", "attributes": { "assignment-params": { "required": true, "type": "string" } }, "capabilities": { "component-node": { "type": "tosca.capabilities.Node" } }, "interfaces": { "ResourceResolutionComponent": { "operations": { "process": { "inputs": { "resolution-key": { "description": "Key for service instance related correlation.", "required": false, "type": "string" }, "store-result": { "description": "Whether or not to store the output.", "required": false, "type": "boolean" }, "artifact-prefix-names": { "required": true, "description": "Template , Resource Assignment Artifact Prefix names", "type": "list", "entry_schema": { "type": "string" } }, "dynamic-properties": { "description": "Dynamic Json Content or DSL Json reference.", "required": false, "type": "json" } }, "outputs": { "resource-assignment-params": { "required": true, "type": "string" }, "status": { "required": true, "type": "string" } } } } } }, "derived_from": "tosca.nodes.Component" }
component-script-executor
Used to execute NETCONF, RESTCONF, SSH commands through a script.
{ "description": "This is Netconf Transaction Configuration Component API", "version": "1.0.0", "interfaces": { "ComponentScriptExecutor": { "operations": { "process": { "inputs": { "script-type": { "description": "Script type, kotlin type is supported", "required": true, "type": "string", "default": "internal", "constraints": [ { "valid_values": [ "kotlin", "jython", "internal" ] } ] }, "script-class-reference": { "description": "Kotlin Script class name or jython script name.", "required": true, "type": "string" }, "dynamic-properties": { "description": "Dynamic Json Content or DSL Json reference.", "required": false, "type": "json" } }, "outputs": { "response-data": { "description": "Execution Response Data in JSON format.", "required": false, "type": "string" }, "status": { "description": "Status of the Component Execution ( success or failure )", "required": true, "type": "string" } } } } } }, "derived_from": "tosca.nodes.Component" }
Workflow
Workflow Scope within CDS Framework
The workflow is within the scope of the micro provisioning and configuration management in controller domain and does NOT account for the MACRO service orchestration workflow which is covered by the SO Project.
A workflow defines an overall action to be taken on the service, hence is an entry-point for the run-time execution of the CBA package.
A workflow also defines inputs and outputs that will defined the payload contract of the request and response (see Dynamic API)
A workflow can be composed of one or multiple sub-actions to execute.
A CBA package can have as many workflows as needed.
Single action
The workflow is directly backed by a node_template of type tosca.nodes.Component
Multiple sub-actions
The workflow is backed by Directed Graph engine, node_template of type dg-generic
, and are imperative workflows.
A DG used as workflow for CDS is composed of multiple execute nodes; each individual execute node refers to a plugin, that is a node_template of type
.tosca.nodes.Component
Bellow the properties of a workflow:
Property | Description | |||||||
---|---|---|---|---|---|---|---|---|
workflow-name | Defines the name of the action that can be triggered by external system | |||||||
inputs |
They are two types of inputs, the dynamic ones, and the static one. Represents any input enforced by the designer at the workflow level The enrichment process will dynamically create a data-type for the workflow, capturing all the resources defined as part of the mapping definition file using the resource source input. The enrichment process will add an entry within the workflow input, with the name as follow:
Example for workflow named resource-assignment: dynamic input
"resource-assignment-properties": { "required": true, "type": "dt-resource-assignment-properties" } |
|||||||
outputs | Defines the outputs of the execution; there can be as many output as needed.
|
|||||||
steps | Defines the actual step to execute as part of the workflow
|
Example:
{ "workflow": { "resource-assignment": { <- workflow-name "inputs": { "vnf-id": { <- static inputs "required": true, "type": "string" }, "resource-assignment-properties": { <- dynamic inputs "required": true, "type": "dt-resource-assignment-properties" } }, "steps": { "call-resource-assignment": { <- step-name "description": "Resource Assignment Workflow", "target": "resource-assignment-process" <- node_template targeted by the step } }, "outputs": { "template-properties": { <- output "type": "json", <- complex type "value": { "get_attribute": [ <- uses expression to retrieve attribute from context "resource-assignment", "assignment-params" ] } } } } } }
TOSCA definition: http://docs.oasis-open.org/tosca/TOSCA-Simple-Profile-YAML/v1.2/csd01/TOSCA-Simple-Profile-YAML-v1.2-csd01.html#_Toc494454203
Template
A template is an artifact, and uses artifact-mapping-resource and artifact-template-velocity.
A template is parameterized and each parameter must be defined in a corresponding mapping file.
In order to know which mapping correlates to which template, the file name must start with an artifact-prefix
, serving as identifier to the overall template + mapping.
The requirement is as follows:
${artifact-prefix}-template
${artifact-prefix}-mapping
Resource accumulator
In order to resolve HEAT environment variables, resource accumulator templates are being in used in Dublin.
These templates are specific to the instantiation scenario, and relies on GR-API within SDNC.
It is composed of the following sections:
resource-accumulator-resolved-data: defines all the resources that can be resolved directly from the context. It expresses a direct mapping between the name of the resource and its value.
capability-data: defines what capability to use to create a specific resource, along with the ingredients required to invoke the capability and the output mapping.
Scripts
Library
NetconfClient
In order to facilitate NETCONF interaction within scripts, a python NetconfClient binded to our Kotlin implementation is made available. This NetconfClient can be used when using the component-netconf-executor
.
The client can be find here: https://github.com/onap/ccsdk-cds/blob/master/components/scripts/python/ccsdk_netconf/netconfclient.py
ResolutionHelper
When executing a component executor script, designer might want to perform resource resolution along with template meshing directly from the script itself.
The helper can be find here: https://github.com/onap/ccsdk-cds/blob/master/components/scripts/python/ccsdk_netconf/common.py
Resource Definition aka Data Dictionary
A data dictionary models the how a specific resource can be resolved.
A resource is a variable/parameter in the context of the service. It can be anything, but it should not be confused with SDC or Openstack resources.
A data dictionary can have multiple sources to handle resolution in different ways.
The main goal of data dictionary is to define re-usable entity that could be shared.
Creation of data dictionaries is a standalone activity, separated from the blueprint design.
As part of modelling a data dictionary entry, the following generic information should be provided:
Property | Description | Scope |
---|---|---|
name | Data dictionary name | Mandatory |
tags | Information related | Mandatory |
updated-by | The creator | Mandatory |
property | Defines type and description, as nested JSON | Mandatory |
sources | List of resource source instance (see resource source) | Mandatory |
Bellow are properties that all the resource source have will have
The modeling does allow for data translation between external capability and CDS for both input and output key mapping.
Property | Description | Scope |
input-key-mapping | map of resources required to perform the request/query. The left hand-side is what is used within the query/request, the right hand side refer to a data dictionary instance. | Optional |
output-key-mapping | name of the resource to be resolved mapped to the value resolved by the request/query. | Optional |
key-dependencies | list of data dictionary instances to be resolved prior the resolution of this specific resource. during run time execution the key dependencies are recursively sorted and resolved in batch processing using the acyclic graph algorithm. |
Optional |
Example:vf-module-model-customization-uuid
and vf-module-label
are two data dictionaries. A SQL table, VF_MODULE_MODEL, exist to correlate them.
Here is how input-key-mapping, output-key-mapping and key-dependencies can be used:
{ "name" : "vf-module-label", "tags" : "vf-module-label", "updated-by" : "adetalhouet", "property" : { "description" : "vf-module-label", "type" : "string" }, "sources" : { "primary-db" : { "type" : "source-primary-db", "properties" : { "type" : "SQL", "query" : "select sdnctl.VF_MODULE_MODEL.vf_module_label as vf_module_label from sdnctl.VF_MODULE_MODEL where sdnctl.VF_MODULE_MODEL.customization_uuid=:customizationid", "input-key-mapping" : { "customizationid" : "vf-module-model-customization-uuid" }, "output-key-mapping" : { "vf-module-label" : "vf_module_label" }, "key-dependencies" : [ "vf-module-model-customization-uuid" ] } } } }
Resource source
Defines the contract to resolve a resource.
A resource source is modeled, following TOSCA node_type definition, and derives from the tosca.nodes.ResourceSource
Input
Expects the value to be provided as input to the request.
{ "description": "This is Input Resource Source Node Type", "version": "1.0.0", "properties": {}, "derived_from": "tosca.nodes.ResourceSource" }
Default
Expects the value to be defaulted in the model itself.
{ "description": "This is Default Resource Source Node Type", "version": "1.0.0", "properties": {}, "derived_from": "tosca.nodes.ResourceSource" }
REST
Expects the URI along with the VERB and the payload, if needed.
CDS is currently deployed along the side of SDNC, hence the default rest connection provided by the framework is to SDNC MDSAL.
Property | Description | Scope |
---|---|---|
type | Expected output value, only JSON supported for now | Optional |
verb | HTTP verb for the request - default value is GET | Optional |
payload | Optional | |
endpoint-selector | Specify which REST system to connect to | Optional |
url-path |
URI | Mandatory |
path | JSON path to the value to fetch from the response | Mandatory |
expression-type |
Path expression type - default value is JSON_PATH | Optional |
{ "description": "This is Rest Resource Source Node Type", "version": "1.0.0", "properties": { "type": { "required": false, "type": "string", "default": "JSON", "constraints": [ { "valid_values": [ "JSON" ] } ] }, "verb": { "required": false, "type": "string", "default": "GET", "constraints": [ { "valid_values": [ "GET", "POST", "DELETE", "PUT" ] } ] }, "payload": { "required": false, "type": "string", "default": "" }, "endpoint-selector": { "required": false, "type": "string" }, "url-path": { "required": true, "type": "string" }, "path": { "required": true, "type": "string" }, "expression-type": { "required": false, "type": "string", "default": "JSON_PATH", "constraints": [ { "valid_values": [ "JSON_PATH", "JSON_POINTER" ] } ] }, "input-key-mapping": { "required": false, "type": "map", "entry_schema": { "type": "string" } }, "output-key-mapping": { "required": false, "type": "map", "entry_schema": { "type": "string" } }, "key-dependencies": { "required": true, "type": "list", "entry_schema": { "type": "string" } } }, "derived_from": "tosca.nodes.ResourceSource" }
Connection to a specific REST system can be expressed through the endpoint-selector
property, which refers to a macro defining the information about the REST system the connect to. Understand TOSCA Macro in the context of CDS.
Few ways are available to authenticate to the REST system:
-
token-auth Expand source
"dsl_definitions": { "dynamic-rest-source": { "type" : "token-auth", "url" : "http://localhost:32778", "token" : "Token 0123456789abcdef0123456789abcdef01234567" } }
-
basic-auth Expand source
"dsl_definitions": { "dynamic-rest-source": { "type" : "basic-auth", "url" : "http://localhost:32778", "username" : "bob", "password": "marley" } }
TBD - The trustore and the keystore can be provide as environment through the Environment folder in the CBA.
ssl-basic-auth Expand source"dsl_definitions": { "dynamic-rest-source": { "type" : "ssl-basic-auth", "url" : "http://localhost:32778", "keyStoreInstance": "JKS or PKCS12", "sslTrust": "trusture", "sslTrustPassword": "trustore password", "sslKey": "keystore", "sslKeyPassword: "keystore password" } }
SQL
Expects the SQL query to be modeled; that SQL query can be parameterized, and the parameters be other resources resolved through other means. If that's the case, this data dictionary definition will have to define key-dependencies
along with input-key-mapping
.
CDS is currently deployed along the side of SDNC, hence the primary database connection provided by the framework is to SDNC database.
Property | Description | Scope |
---|---|---|
type | Database type, only SQL supported for now | Mandatory |
endpoint-selector | Specify which database to connect to | Optional |
query | Statement to execute | Mandatory |
{ "description": "This is Database Resource Source Node Type", "version": "1.0.0", "properties": { "type": { "required": true, "type": "string", "constraints": [ { "valid_values": [ "SQL" ] } ] }, "endpoint-selector": { "required": false, "type": "string" }, "query": { "required": true, "type": "string" }, "input-key-mapping": { "required": false, "type": "map", "entry_schema": { "type": "string" } }, "output-key-mapping": { "required": false, "type": "map", "entry_schema": { "type": "string" } }, "key-dependencies": { "required": true, "type": "list", "entry_schema": { "type": "string" } } }, "derived_from": "tosca.nodes.ResourceSource" }
Connection to a specific database can be expressed through the endpoint-selector
property, which refers to a macro defining the information about the database the connect to. Understand TOSCA Macro in the context of CDS.
"dsl_definitions": { "dynamic-db-source": { "type": "maria-db", "url": "jdbc:mysql://localhost:3306/sdnctl", "username": "sdnctl", "password": "sdnctl" } }
Capability
Expects a script to be provided.
Property | Description | Scope |
---|---|---|
script-type | The type of the script - default value is Koltin | Optional |
script-class-reference |
The name of the class to use to create an instance of the script | Mandatory |
{ "description": "This is Component Resource Source Node Type", "version": "1.0.0", "properties": { "script-type": { "required": true, "type": "string", "default": "kotlin", "constraints": [ { "valid_values": [ "internal", "kotlin", "jython" ] } ] }, "script-class-reference": { "description": "Capability reference name for internal and kotlin, for jython script file path", "required": true, "type": "string" }, "key-dependencies": { "description": "Resource Resolution dependency dictionary names.", "required": true, "type": "list", "entry_schema": { "type": "string" } } }, "derived_from": "tosca.nodes.ResourceSource" }
- No labels