Note: Work in Progress
Requirement
Key Contacts - @Byung-Woo Jun (Ericsson), ...
Executive Summary - Provide ASD-based CNF orchestration support through SO and its sub-components, CNFM and CNF Adapter using K8S
- Support for provisioning ASD-based CNFs using an external K8s Manager
- Instantiate and optionally overwrite the values of an application
- Terminate an application and optionally remove all relevant Persistent Volume Claims and Persistent Volumes
- Change application details using the change package function, which allows to change the deployment version of the CNF thru an upgrade or rollback (stretch goal)
- Leverage and enhance SO capabilities and adding new capabilities for ASD- and Helm-based CNF orchestration
- Orchestrator shall support the capability to use the deployment parameters from ASD for the application or CNF deployment. These deployment parameter values shall correspond to the parameters defined in the “lifecycleParameters” section(s) of the ASD.
- Orchestrator shall support the capability to construct a values file from instance specific parameter values provided at deployment time, and default values supplied in the chart.
- Orchestrator shall support the capability to perform a chart render into concrete K8S resource descriptions.
- Container resource management for determining placement for CNF application on certain K8S cluster(s), orchestrator shall support the capability to parse the workload descriptors and extract those values.
Business Impact - Enables operators and service providers to orchestrate ASD-based CNFs services along with the VNFs and PNFs
Business Markets - All operators and service providers that are intended to use the ASD-based CNFs along with PNFs / VNFs
Funding/Financial Impacts - Reduction in the footprint of the ONAP for CNF support
Organization Mgmt, Sales Strategies - There is no additional organizational management or sales strategies for this requirement outside of a service providers "normal" ONAP deployment and its attendant organizational resources from a service provider
JIRA: - REQ-1043Getting issue details... STATUS
Epic
Epic | Description | JIRA | priority |
---|---|---|---|
SO and its sub-components, CNFM, CNF Adapter, shall support ASD-based CNF lifecycle orchestration | SO and its sub-components, CNFM, CNF Adapter, shall support ASD-based CNF lifecycle orchestration (deployment and configuration) | 1 | |
User Story
User Story | Affected Component | Description | JIRA | Priority |
---|---|---|---|---|
SO shall get the ASD-based CNF package from SDC and store its metadata to SO Catalog DB | SDC Controller, CatalogDB Adapter, CatalogDB | SO shall get the ASD-based CNF package (SDC Service CSAR) from SDC and store its metadata to SO Catalog DB. Pre Condition:
Post Condition:
| 1 | |
| 1 | |||
| 1 | |||
For ASD-based CNF provisioning, SO shall process model info and decide flows | API Handler, RequestDB Adapter, RequestDB, SO BPMN Infra, AAI | SO shall process ASD model info and decide ASD provisioning flows Pre Condition:
Post Condition:
| 2 | |
Task:
| 2.1 | |||
Task:
| 2.2 | |||
Task:
| 2.3 | |||
Task:
| 2.4 | |||
Task:
| 2.5 | |||
SO CNFM shall process ASD-based CNF Lifecycle orchestration | SO CNFM, ASD Repository, Helm Artifact Repository, OOF, AAI, SDNC Adapter, SDNC, CDS, CNF Adapter | SO CNFM shall process ASD-based CNF Lifecycle orchestration Pre Condition:
Post Condition:
Note: PoC, ASD external CPDs will not be handled (TBD) Note: ASD Repository, Helm Artifact Repository, Image Artifact Repository will be handled by another Epic, which is defined in Application Package Distribution | 3 | |
Task: Create SO CNFM and make it available in ONAP
| 3.1 | |||
Create SO CNFM REST API swagger, based on the ASD LCM Restful API, ASD LCM RESTful Protocols for SO CNF Manager and Swagger file (ASD LCM RESTful Protocols for SO CNF Manager) | 3.2 | |||
Task: Support for SO CNFM NBI API Handler
| 3.3 | |||
Task:
| 3.4 | |||
Task:
| 3.5 | |||
Task:
| 3.6 | |||
Task:
| 3.7 | |||
Task: Create SO CNFM Instance Database Management
| 3.8 | |||
Task:
| 3.9 | |||
Task:
| 3.10 | |||
Task:
| 3.11 | |||
Task:
| 3.12 | |||
Task:
| 3.13 | |||
Task:
| 3.14 | |||
SO Client shall send the A La Carte Request for ASD-based CNF orchestration (note: E2E support is out of PoC scope) | SO Client, SO, SO BPMN, CNFM CNF Adapter | SO Client shall send the A La Carte Request for ASD-based CNF orchestration Pre Condition:
Post Condition:
| 4 | |
Task:
| 4.1 | |||
Task:
| 4.2? |
Overall Process
- Pre-Conditions
- SDC accepts onboarding App packages, including ASD and DeploymentItems models, Helm Charts, Images and other artifacts, what allows to keep decomposition of Service instance
- SO subscribes and receives package notifications from SDC
- ASD-Based CNF LCM Orchestration
- Based on the notifications, SO ASDC Controller queries for the App packages from SDC, and stores models and artifacts to SO Catalog Database
- MACRO workflow in SO is used for orchestration
- ASD supports multiple Helm Charts in CNF packages.
- ASD instance will be decomposed to find its associated deployment item(s).
ASD-Based App Package Type
- The Resource VF(s) in the onboarded Service CSAR will have the following resource main manifest file
- application_name
- application_provider
- release_date_time
- entry_definition_type [ "asd"]
- SO distinguishes the App package based on the "entry_defintion_type" metadata.
- If it is "asd", SO will process the package as the ASD-based CNF.
- SO delegates the ASD-based CNF orchestration to the CNF Manager
LCM Architecture Overview
The following diagram depicts LCM Architecture for the ASD-based CNF.
SO Internal Architecture
The following diagram depicts SO internal architecture for the ASD-based orchestration:
ASD LCM Restful API
Please refer to the section, ASD LCM RESTful Protocols for SO CNF Manager .
- Swagger file (ASD LCM RESTful Protocols for SO CNF Manager)
ASD LCM Operations
Create an AS Identifier
In order for an application (CNF) to be eligible for instantiation, a CNF identifier must be created to identify the application service as a unique instance.
An AS instance supports multiple deployment items (helm charts). So, the AS instance id could be used for helm release names as the prefix.
Query AS Instance Identifiers
To list all AS instances, make a GET request to SO CNFM.
GET .../aslcm/v1/as_instances
Query an AS instance by Instance ID
To query a specific AS instance by its instance ID, make a GET request to SO CNFM.
GET .../aslcm/v1/as_instances/{asInstanceId}
Instantiate an AS
- To instantiate an AS, make a poST request to SO CNFM. The target cluster name is included in the requrest parameter.
POST .../as_instances/{asInstanceId}/instantiate
- if some of the parameters required for instantiation are complex and cannot fit into the additionalParams section of the JSON request body (TBD).
Upgrading an AS
This operation is referred to as the changing of an AS package.
POST .../as_instances/{asInstanceId}/change_aspkg
Terminate an AS Instance
This operation terminates an AS instance.
POST .../as_instances/{asInstnaceId}/terminate (TerminateAsRequest)
Note: for PoC, GRACEFUL termination type is not supported.
Delete an AS Identifier
This operation deletes an AS Identifier.
DELETE .../as_instances/{as_instanceId}
Note: optionally, Delete an AS Identifier could clean up resources of an AS Instance, e.g., Persistent Volumes (PVs)
Register K8S Clusters (checking if AAI can be used)
To instantiate an AS on an non-ONAP K8S cluster, a cluster configuration file that is specific to the cluster must be uploaded.
To add a cluster configuration file of a cluster, create a POST request .../aslcm/v1/clusterconfigs will be performed.
- CNFM receives the clusterconfigs info and creates a cluster configuration file (cluster name + "." + "config" to the ./kube directory.
The kubectl configuration file is used to the body of the request to define external K8S clusters.
The cluster configuration file for a particular cluster must be retrieved from the cluster administrator.
Should the cluster configuration file change for any reason, e.g., CA certificate rotation on the target cluster or client key expires, then the cluster file registered in SO CNFM/AAI shall need to be updated.
The target cluster server and port must be reachable from the SO CNFM.
- to verify the connectdion to the target cluster, run the following command from the ONAP K8S cluster
- kubectl --kubeconfig ${PATH_TO_TARGET_CLUSTER_CONFIGURATION_FILE} get namespaces
Deregister K8S Clusters (checking if AAI can be used)
To remove a cluster configruation file, create a DELETE request. .../aslcm/v1/clusterconfig/{configName}
CNFM will remove the "configName" + "." + "config" file from the .kube directory.
The command returns the HTTP status code 204 No Content
List Registered K8S Clusters (checking if AAI can be used)
To get details about registered clusters, create a GET request .../aslcm/v1/clusterconfigs
The API returns a paginated response, but if a customized response is needed, additional parameters for page, size, sor and filtering could be applied.
Helm Command Support
The Helm library uses the Kubernetes client library to communicate with Kubernetes. Currently, the library uses REST+JSON. It stores information in Secrets located inside of Kubernetes; i.e., it does not need its own database.
- Helm Install
- Helm Uninstall
- Helm Upgrade
Component Communication Security
The component communication security will be achieved by leveraging Service Mesh. The external Helm Artifact Repository secure communication is under discussion.
Design and Distribution of ASD Service CSAR - Day 0
The following sequence diagram depicts design and distribution of ASD service CSAR, based on CNFO. This sequence could be changed.
Instantiation of ASD Service CSAR - Day 1
The following sequence diagram depicts instantiation of ASD-based CNF.
Option A: use Helm Command:
Option B: use of CNF Adapter
Note: use of CDS is TBD.
AAI Data Model
AAI CNF Model - Overview
Note: the following overview came from the Direct CNFO AAI CNF Model. We will start it from here and will finalize the AAI model of ASD-based CNF.
- Currently no CNF Resources information is visible in ONAP AAI - agree
- Some interfaces are already implemented (Multicloud k8s Status/Query API) that
allow retrieval of detailed resources information - plan to leverage existing interfaces if they are appropriate - Initial implementation of CNF Model in AAI should be simple and allow user to
know about resources available and where to get their exact status from - plan to leverage the CNF model in AAI if they are appropriate - Long-Term solution should design appropriate CNF Resources in AAI, providing
only the most important data and relationships about them. - TBD
AAI CNF Model
Note: the following overview came from the Direct CNFO AAI CNF Model. We will start it from here and will finalize the AAI model of ASD-based CNF.
- Create additional AAI Object storing Any CNF Resource in K8s
- This new object (eg. k8sobject) should be related to generic-vnf/vf-module and cloud-region (subresource), using similar relationship matrix as vservers
- Data stored within AAI would be very generic, containing more or less only:
- Name [string; Primary Key]
- Group, Version, Kind [strings; Primary Key]
- Namespace [string; Primary Key]
- Labels [map[string]string or []string] (depending on AAI capabilities)
- Annotations [map[string]string or []string ] (depending on AAI capabilities)
- DetailsReflink [string or object]
- This field allows AAI Object consumer to specify query toward SO CNF Adapter to get full object data
Simple AAI CNF Model - Jakarta - Initial Model
Note: the following overview came from the Direct CNFO AAI CNF Model. We will start it from here and will finalize the AAI model of ASD-based CNF.
This model was drafted for reference and requires further tuning and agreement. This model needs to be discussed further at the Modeling Subcommittee
Main assumptions of this model:
- Update existing resources schema (Cloud-Region, Generic-VNF) and/or create CNF counterparts (VF-Module)
- Store Resources definitions about Controllers (eg. Deployments), Compute (Pods, Containers), Storage (PV/PVC), Network (Pod Interfaces, Services) and Configuration resources (eg. ConfigMaps, Secrets).
- Store unclassified resources like CRDs under “Uncategorized” - like object
SO ASDC Controller Handling
ASDC Controller Enhancement
SO Catalog DB Enhancement
VF-VF-Module-Helm Package
- During SDC onboarding, the ASD is mapped to VF, and the DeploymentItems is mapped to Vf-Module
- SO expects ASD VSP onboarding packages has the Native Helm, without Dummy Heat.
- VF is split into multiple vf-modules. each vf-module is dedicated to one Helm package.
- each Helm application after instantiation is visible to ONAP as a separate vf-module
Deployment Components
note: the current CNFO; will be changed for ASD-based CNF orchestration
ONAP Component | Description |
---|---|
AAI | Required for Inventory Cloud Owner, Customer, Owning Entity, Service, Generic VNF, VF-Module |
SDC | VSP, VF and Service Modeling of the ASD-based CNF |
SO | Required for Macro Orchestration using the generic building blocks |
CNF Adapter | |
SDNC | Provides GENERIC-RESOURCE-API for cloud instantiation orchestration via CDS |
Policy | used to store naming policy |
Portal | required to access SDC |
CDS | TBD |
CNFM | |
MultiCloud | TBD |
Runtime Catalog Mgr | |
Helm Artifact Repository | |
Image Artifact Repository |
SO CNF Manager Input Parameter Handling
Note: Work in Progress
Requirement
Key Contacts - @Byung-Woo Jun (Ericsson), ...
Executive Summary - Provide ASD-based CNF orchestration support through SO and its sub-components, CNFM and CNF Adapter using K8S
- Support for provisioning ASD-based CNFs using an external K8s Manager
- Leverage and enhance SO capabilities and adding new capabilities for ASD- and Helm-based CNF orchestration
- Orchestrator shall support the capability to use the deployment parameters from ASD for the application or CNF deployment. These deployment parameter values shall correspond to the parameters defined in the “lifecycleParameters” section(s) of the ASD.
- Orchestrator shall support the capability to construct a values file from instance specific parameter values provided at deployment time, and default values supplied in the chart.
- Orchestrator shall support the capability to perform a chart render into concrete K8S resource descriptions.
- Container resource management for determining placement for CNF application on certain K8S cluster(s), orchestrator shall support the capability to parse the workload descriptors and extract those values.
Business Impact - Enables operators and service providers to orchestrate ASD-based CNFs services along with the VNFs and PNFs
Business Markets - All operators and service providers that are intended to use the ASD-based CNFs along with PNFs / VNFs
Funding/Financial Impacts - Reduction in the footprint of the ONAP for CNF support
Organization Mgmt, Sales Strategies - There is no additional organizational management or sales strategies for this requirement outside of a service providers "normal" ONAP deployment and its attendant organizational resources from a service provider
JIRA: - REQ-1043Getting issue details... STATUS
Epic
Epic | Description | JIRA | priority |
---|---|---|---|
SO and its sub-components, CNFM, CNF Adapter, shall support ASD-based CNF lifecycle orchestration | SO and its sub-components, CNFM, CNF Adapter, shall support ASD-based CNF lifecycle orchestration (deployment and configuration) | 1 | |
User Story
User Story | Affected Component | Description | JIRA | Priority |
---|---|---|---|---|
SO shall get the ASD-based CNF package from SDC and store its metadata to SO Catalog DB | SDC Controller, CatalogDB Adapter, CatalogDB | SO shall get the ASD-based CNF package (SDC Service CSAR) from SDC and store its metadata to SO Catalog DB. Pre Condition:
Post Condition:
| 1 | |
| 1 | |||
| 1 | |||
For ASD-based CNF provisioning, SO shall process model info and decide flows | API Handler, RequestDB Adapter, RequestDB, SO BPMN Infra, AAI | SO shall process ASD model info and decide ASD provisioning flows Pre Condition:
Post Condition:
| 2 | |
Task:
| 2.1 | |||
Task:
| 2.2 | |||
Task:
| 2.3 | |||
Task:
| 2.4 | |||
Task:
| 2.5 | |||
SO CNFM shall process ASD-based CNF Lifecycle orchestration | SO CNFM, ASD Repository, Helm Artifact Repository, OOF, AAI, SDNC Adapter, SDNC, CDS, CNF Adapter | SO CNFM shall process ASD-based CNF Lifecycle orchestration Pre Condition:
Post Condition:
Note: PoC, ASD external CPDs will not be handled (TBD) Note: ASD Repository, Helm Artifact Repository, Image Artifact Repository will be handled by another Epic, which is defined in Application Package Distribution | 3 | |
Task: Create SO CNFM and make it available in ONAP
| 3.1 | |||
Task: Support for SO CNFM NBI API Handler
| 3.2 | |||
Task:
| 3.3 | |||
Task:
| 3.4 | |||
Task:
| 3.5 | |||
Task:
| 3.6 | |||
Task: Create SO CNFM Instance Database Management
| 3.7 | |||
Task:
| 3.8 | |||
Task:
| 3.9 | |||
Task:
| 3.10 | |||
Task:
| 3.11 | |||
Task:
| 3.12 | |||
Task:
| 3.13 | |||
SO Client shall send the A La Carte Request for ASD-based CNF orchestration (note: E2E support is out of PoC scope) | SO Client, SO, SO BPMN, CNFM CNF Adapter | SO Client shall send the A La Carte Request for ASD-based CNF orchestration Pre Condition:
Post Condition:
| 4 | |
Task:
| 4.1 | |||
Task:
| 4.2? |
Overall Process
- Pre-Conditions
- SDC accepts onboarding App packages, including ASD and DeploymentItems models, Helm Charts, Images and other artifacts, what allows to keep decomposition of Service instance
- SO subscribes and receives package notifications from SDC
- ASD-Based CNF LCM Orchestration
- Based on the notifications, SO ASDC Controller queries for the App packages from SDC, and stores models and artifacts to SO Catalog Database
- MACRO workflow in SO is used for orchestration
ASD-Based App Package Type
- The Resource VF(s) in the onboarded Service CSAR will have the following resource main manifest file
- oran_application_name
- oran_application_provider
- oran_release_date_time
- oran_entry_definition_type [ "asd"]
- SO distinguishes the App package based on the "oran_entry_defintion_type" metadata.
- If it is "asd", SO will process the package as the ASD-based CNF.
- SO delegates the ASD-based CNF orchestration to the CNF Manager
LCM Architecture Overview
The following diagram depicts LCM Architecture for the ASD-based CNF.
SO Internal Architecture
The following diagram depicts SO internal architecture for the ASD-based orchestration:
Design and Distribution of ASD Service CSAR - Day 0
The following sequence diagram depicts design and distribution of ASD service CSAR.
Instantiation of ASD Service CSAR - Day 1
The following sequence diagram depicts instantiation of ASD-based CNF.
Note: use of CDS is TBD.
AAI Data Model
AAI CNF Model - Overview
Note: the following overview came from the Direct CNFO AAI CNF Model. We will start it from here and will finalize the AAI model of ASD-based CNF.
- Currently no CNF Resources information is visible in ONAP AAI - agree
- Some interfaces are already implemented (Multicloud k8s Status/Query API) that
allow retrieval of detailed resources information - plan to leverage existing interfaces if they are appropriate - Initial implementation of CNF Model in AAI should be simple and allow user to
know about resources available and where to get their exact status from - plan to leverage the CNF model in AAI if they are appropriate - Long-Term solution should design appropriate CNF Resources in AAI, providing
only the most important data and relationships about them. - TBD
AAI CNF Model
Note: the following overview came from the Direct CNFO AAI CNF Model. We will start it from here and will finalize the AAI model of ASD-based CNF.
- Create additional AAI Object storing Any CNF Resource in K8s
- This new object (eg. k8sobject) should be related to generic-vnf/vf-module and cloud-region (subresource), using similar relationship matrix as vservers
- Data stored within AAI would be very generic, containing more or less only:
- Name [string; Primary Key]
- Group, Version, Kind [strings; Primary Key]
- Namespace [string; Primary Key]
- Labels [map[string]string or []string] (depending on AAI capabilities)
- Annotations [map[string]string or []string ] (depending on AAI capabilities)
- DetailsReflink [string or object]
- This field allows AAI Object consumer to specify query toward SO CNF Adapter to get full object data
Simple AAI CNF Model - Jakarta - Initial Model
Note: the following overview came from the Direct CNFO AAI CNF Model. We will start it from here and will finalize the AAI model of ASD-based CNF.
This model was drafted for reference and requires further tuning and agreement. This model needs to be discussed further at the Modeling Subcommittee
Main assumptions of this model:
- Update existing resources schema (Cloud-Region, Generic-VNF) and/or create CNF counterparts (VF-Module)
- Store Resources definitions about Controllers (eg. Deployments), Compute (Pods, Containers), Storage (PV/PVC), Network (Pod Interfaces, Services) and Configuration resources (eg. ConfigMaps, Secrets).
- Store unclassified resources like CRDs under “Uncategorized” - like object
SO ASDC Controller Handling
ASDC Controller Enhancement
SO Catalog DB Enhancement
VF-VF-Module-Helm Package
- During SDC onboarding, the ASD is mapped to VF, and the DeploymentItems is mapped to Vf-Module
- SO expects ASD VSP onboarding packages has the Native Helm, without Dummy Heat.
- VF is split into multiple vf-modules. each vf-module is dedicated to one Helm package.
- each Helm application after instantiation is visible to ONAP as a separate vf-module
Deployment Components
note: the current CNFO; will be changed for ASD-based CNF orchestration
ONAP Component | Description |
---|---|
AAI | Required for Inventory Cloud Owner, Customer, Owning Entity, Service, Generic VNF, VF-Module |
SDC | VSP, VF and Service Modeling of the ASD-based CNF |
SO | Required for Macro Orchestration using the generic building blocks |
CNF Adapter | |
SDNC | Provides GENERIC-RESOURCE-API for cloud instantiation orchestration via CDS |
Policy | used to store naming policy |
Portal | required to access SDC |
CDS | TBD |
CNFM | |
MultiCloud | TBD |
Runtime Catalog Mgr | |
Helm Artifact Repository | |
Image Artifact Repository |
SO CNF Manager Input Parameter Handling