Summary: Edge Scoping
Table of Contents |
---|
Distributed Edge Cloud Infrastructure Object Hierarchy (Stretch Goal)
Value:
- Fine grained resource management & analytics for Distributed Edge Clouds
References:
- Infrastructure Modelling: ONAP R3+ Cloud Infrastructure Modeling; Cloud Infrastructure Aggregate Representation Classes
Jira | ||||||
---|---|---|---|---|---|---|
|
ONAP Component | Life Cycle Phase | Enhancements |
---|---|---|
Multi-Cloud | Deploy | Support Distributed Cloud Infrastructure Capability Discovery (Note 1, Note 2) |
A&AI | Deploy | Support Standardized Distributed Cloud Infrastructure Object Hierarchy & Capability Database (Ref. 1)
|
OOF | Deploy | Execute Distributed Cloud Infrastructure Placement Policies for Optimized Service/VNF Placement across Cloud Regions (Note 3, Note 4) |
SO | Deploy | Extend SO ↔ OOF API to support data opaque to SO (Note 5) Extend SO ↔ MC API to support data opaque to SO (Note 6) |
Assumption for Policy, SO, OOF:
- This uses the current Generic VNF workflow in SO
Note 1:
- Configured Capacity and Utilized (or Currently Used) Capacity are managed by the specific cloud.
Note 2:
- Cloud SW Capability example
- Cloud region "x" with SR-IOV, GPU, Min-guarantee support
- Cloud region "y" with SR-IOV support
- Cloud HW Capability example
- Resource cluster "xa" in Cloud region "x" with SR-IOV and GPU support
- Resource cluster "xb" in Cloud region "x" with GPU support
- Resource cluster "ya" in Cloud region "y" with SR-IOV support
Note 3:
- 5G Service/VNF placement example
- Constraints used by Optimization Framework (OOF)
5G CU-UP VNF location to be fixed to a specific physical DC based on 5G DU, bounded by a max distance from 5G DU
- Optimization Policy used by OOF
Choose optimized cloud region (or instance) for the placement of 5G CU UP for subscriber group based on the above constraints
- Constraints used by Optimization Framework (OOF)
Note 4:
- For the 5G Service/VNF placement example in Note 3
- 5G CU-UP VNF preferably maps to a specific Cloud region & Physical DC End Point
Note 5:
- For the 5G Service/VNF placement example in Note 3
- OOF will pass the Physical DC End Point to SO as a opaque data
Note 6:
- For the 5G Service/VNF placement example in Note 3
- SO passes the Physical DC End Point to Multi-Cloud as a opaque data, besides the Cloud Region
Cloud-agnostic Placement/Networking & Homing Policies (Phase 1 - Casablanca MVP, Phase 2 - Stretch Goal)
End-to-end use case Applicability:
All (especially the data plane VNFs with fine-grained VNF placement and high performance networking requirements)
Value:
Improve "workload deployability" by avoiding exposure of "cloud specific" capabilities to several ONAP components and addressing "separation of concerns"
Support capacity check (besides capability check) for HPA resources
Applicable to all workloads - VM-based or Container-based
Jira server ONAP JIRA serverId 425b2b0a-557c-3c0c-b515-579789cceedb key MULTICLOUD-272
Phase 1 (Casablanca MVP) Summary:
- Multi-Cloud Policy Framework
- Assist OOF in target cloud region selection for VNF placement (aka homing) by summarizing cloud-specific capability, capacity & cost metrics (e.g. VMs could have different cost in different clouds, Infra High Availability (HA) for VMs in a VNF could have different cost in different clouds)
Cloud Agnostic Intent (Policy) Execution Workflow - Steps 1- 6
- Dynamically modify the cloud specific VNF deployment template based on cloud-specific realization of the specified intent (e.g. Infra HA for VMs within a VNF could have different realizations across different clouds)
Cloud Agnostic Intent (Policy) Execution Workflow - Step 7
- Assist OOF in target cloud region selection for VNF placement (aka homing) by summarizing cloud-specific capability, capacity & cost metrics (e.g. VMs could have different cost in different clouds, Infra High Availability (HA) for VMs in a VNF could have different cost in different clouds)
Intent Support
Single realization option per Cloud Region for the specified Intent
- Impact Projects:
- Multi-Cloud (Highest), OOF
- End-to-end use case demonstration:
- vCPE, vDNS
Phase 2 (Casablanca Stretch Goal) Summary (Build on Phase 1 Work):
- Multi-Cloud Policy Framework
- Dynamically modify the cloud specific VNF deployment template based on cloud-specific realization of the specified intent – Impact to VNF configuration
- E.g. High performance Intra-DC data plane networking with several realization choices
- Dynamically modify the cloud specific VNF deployment template based on cloud-specific realization of the specified intent – Impact to VNF configuration
- Intent Support
- Multiple realization options per Cloud Region for the specified Intent
- Major Impact Projects:
- Multi-Cloud
- Minor Impact Projects:
- SO, OOF, GNF Controller
- Wiki Link:
References:
The sequence diagram below expands "Multi-Cloud/VNFM Deploy Apps" in Edge Scoping Sequence Diagram
Cloud Agnostic Intent (Policy) Workflow Summary (Phase 1 - Casablanca MVP):
Gliffy Diagram size 1200 name Cloud Agnostic Intent Execution Workflow pagePin 40
Cloud Agnostic Intent (Policy) Workflow Details (Phase 1 - Casablanca MVP):
Private Cloud Setup - OpenStack-based
- Pre-defined (including custom) flavors map to Instance types in Public Clouds
- Pre-defined flavors are created by the Cloud Admin before the Cloud is used by ONAP for workload deployment
- VMware VIO Configuration for Min Guarantee feature
- Create necessary tenants per <cloud owner, cloud region>
- Mapping of VNFC, VNF (VF module), Service (e.g. vCPE) to the corresponding tenant happens in the respective Multi-Cloud plugin.
VNFC to Instance Type Mapping
- One or more VNFCs (e.g. vCPE VGW) could map to an Instance Type
- Use Case: Residential Broadband vCPE (Approved)
- OpenStack-based Clouds
- Instance type maps to pre-defined Flavors
- Microsoft Azure
- Pre-defined Instance Types
Step 1. SO → OOF - Get Target <Cloud Owner, Cloud Region> for the Service Instances
Step 2. OOF → Policy - Fetch Enhanced Capacity Check & Cloud Selection Policy for Homing
2a) OOF Processing - the fetched Policy (example below) is stored in a local data structure and is available for further use (need OOF code changes).
Code Block | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||
// //Example 1: vCPE, Burstable QoS //vCPE: Infrastructure Resource Isolation for VNF with Burstable QoS // { "service": "cloudSelectionPolicy", "policyName": "oofMulti-cloudCasablanca.cloudSelectionPolicy_vCPE_VNF", "description": "Cloud Selection Policy for vCPE VNFs", "templateVersion": "0.0.1", "version": "oofMulti-cloudCasablanca", "priority": "3", "riskType": "test", "riskLevel": "2", "guard": "False", "content": { //Note on cloudOwner, cloudRegion usage: e.g. //"cloudOwner": "xyz", //can be a specific cloud owner such as Azure, VMware VIO, Wind River Titanium Cloud etc. //"cloudRegion": "1", //can be a specific cloud region for a cloud owner //If no cloudOwner is specified, policy applies to all cloudOwners //If no cloudRegion is specified, policy applies to all cloudRegions {//new in R3 "cloudOwner": "Azure", "cost-intent": { "dollarCostEvaluationVM-Type": "TRUE", //evaluate dollar cost per VM type if operator has configured a policy "dollarCostEvaluationVM-FeatureGroup": "TRUE" //evaluate dollar per group of features if operator has configured a policy //deployment-intent will be mapped to a cloud-specific realization which will be mapped to a feature group. } }, {//new in R3 "cloudOwner": "VMware VIO", //can be a specific cloud owner such as Azure, VMware VIO, Wind River Titanium Cloud etc. "deployment-intent": { "name": "Infrastructure Resource Isolation for VNF", // realization possible without dedicating CPU and Memory, refer to section on "Cloud Resource Partitioning for Differentiated QoS" // on how this can help in offering tiered services "qosProperty": { {"Burstable QoS": "TRUE", "Burstable QoS Oversubscription Percentage": "25"} } } }, } "resources": ["vgw", "vgmux"], //"vgw" is also interchangeably used as "vg" "applicableResources": "any", "identity": "cloud-atrributes", "policyScope": ["vCPE", "US", "INTERNATIONAL", "ip", "vgw", "vgmux"], "policyType": "AllPolicy" } // //Example 2: //vCPE: Infrastructure Resource Isolation for VNF with Guaranteed QoS // { "service": "cloudSelectionPolicy", "policyName": "oofMulti-cloudCasablanca.cloudSelectionPolicy_vCPE_VNF", "description": "Cloud Selection Policy for vCPE VNFs", "templateVersion": "0.0.1", "version": "oofMulti-cloudCasablanca", "priority": "3", "riskType": "test", "riskLevel": "2", "guard": "False", "content": { //Note on cloudOwner, cloudRegion usage: e.g. //"cloudOwner": "xyz", //can be a specific cloud owner such as Azure, VMware VIO, Wind River Titanium Cloud etc. //"cloudRegion": "1", //can be a specific cloud region for a cloud owner //If no cloudOwner is specified, policy applies to all cloudOwners //If no cloudRegion is specified, policy applies to all cloudRegions {//new in R3 "cloudOwner": "Azure", "cost-intent": { "dollarCostEvaluationVM-Type": "TRUE", //evaluate dollar cost per VM type if operator has configured a policy "dollarCostEvaluationVM-FeatureGroup": "TRUE" //evaluate dollar per group of features if operator has configured a policy //deployment-intent will be mapped to a cloud-specific realization which will be mapped to a feature group. } }, {//new in R3 "deployment-intent": { "name": "Infrastructure Resource Isolation for VNF", // realization possible without dedicating CPU and Memory, refer to section on "Cloud Resource Partitioning for Differentiated QoS" // on how this can help in offering tiered services "qosProperty": { {"Guaranteed QoS": "TRUE"} } } } } "resources": ["vgw", "vgmux"], //"vgw" is also interchangeably used as "vg" "applicableResources": "any", "identity": "cloud-atrributes", "policyScope": ["vCPE", "US", "INTERNATIONAL", "ip", "vgw", "vgmux"], "policyType": "AllPolicy" } // //Example 3: //vDNS: Infrastructure HA for VMs in a VNF & Infrastructure Resource Isolation for VNF with Burstable QoS // { "service": "cloudSelectionPolicy", "policyName": "oofMulti-cloudCasablanca.cloudSelectionPolicy_vDNS_VNF", "description": "Cloud Selection Policy for vDNS VNFs", "templateVersion": "0.0.1", "version": "oofMulti-cloudCasablanca", "priority": "3", "riskType": "test", "riskLevel": "2", "guard": "False", "content": { //Note on cloudOwner, cloudRegion usage: e.g. //"cloudOwner": "xyz", //can be a specific cloud owner such as Azure, VMware VIO, Wind River Titanium Cloud etc. //"cloudRegion": "1", //can be a specific cloud region for a cloud owner //If no cloudOwner is specified, policy applies to all cloudOwners //If no cloudRegion is specified, policy applies to all cloudRegions {//new in R3 "cloudOwner": "Azure", "cost-intent": { "dollarCostEvaluationVM-Type": "TRUE", //evaluate dollar cost per VM type if operator has configured a policy "dollarCostEvaluationVM-FeatureGroup": "TRUE" //evaluate dollar per group of features if operator has configured a policy //deployment-intent will be mapped to a cloud-specific realization which will be mapped to a feature group. } }, {//new in R3 "cloudOwner": "VMware VIO", //can be a specific cloud owner such as Azure, VMware VIO, Wind River Titanium Cloud etc. "deployment-intent": { "name": "Infrastructure Resource Isolation for VNF", // realization possible without dedicating CPU and Memory, refer to section on "Cloud Resource Partitioning for Differentiated QoS" // on how this can help in offering tiered services "qosProperty": { {"Burstable QoS": "TRUE", "Burstable QoS Oversubscription Percentage": "25"} } } }, {//new in R3 "deployment-intent": { "name": "Infrastructure High Availability (HA) for VNF", } }, } "resources": ["vDNS"], "applicableResources": "any", "identity": "cloud-atrributes", "policyScope": ["vDNS", "US", "INTERNATIONAL", "vDNS"], "policyType": "AllPolicy" } // //Example 4: //vDNS: Infrastructure HA for VMs in a VNF & Infrastructure Resource Isolation for VNF with Guaranteed QoS // { "service": "cloudSelectionPolicy", "policyName": "oofMulti-cloudCasablanca.cloudSelectionPolicy_vDNS_VNF", "description": "Cloud Selection Policy for vDNS VNFs", "templateVersion": "0.0.1", "version": "oofMulti-cloudCasablanca", "priority": "3", "riskType": "test", "riskLevel": "2", "guard": "False", "content": { //Note on cloudOwner, cloudRegion usage: e.g. //"cloudOwner": "xyz", //can be a specific cloud owner such as Azure, VMware VIO, Wind River Titanium Cloud etc. //"cloudRegion": "1", //can be a specific cloud region for a cloud owner //If no cloudOwner is specified, policy applies to all cloudOwners //If no cloudRegion is specified, policy applies to all cloudRegions {//new in R3 "cloudOwner": "Azure", "cost-intent": { "dollarCostEvaluationVM-Type": "TRUE", //evaluate dollar cost per VM type if operator has configured a policy "dollarCostEvaluationVM-FeatureGroup": "TRUE" //evaluate dollar per group of features if operator has configured a policy //deployment-intent will be mapped to a cloud-specific realization which will be mapped to a feature group. } }, {//new in R3 "deployment-intent": { "name": "Infrastructure High Availability (HA) for VNF", } }, {//new in R3 "deployment-intent": { "name": "Infrastructure Resource Isolation for VNF", // realization possible without dedicating CPU and Memory, refer to section on "Cloud Resource Partitioning for Differentiated QoS" // on how this can help in offering tiered services "qosProperty": { {"Guaranteed QoS": "TRUE"} } } } } "resources": ["vDNS"], "applicableResources": "any", "identity": "cloud-atrributes", "policyScope": ["vDNS", "US", "INTERNATIONAL", "ip", "vDNS"], "policyType": "AllPolicy" } |
Step 3. OOF → A&AI - Fetch Cloud-Agnostic (Standardized) Capabilities for the Service Instance
3a) OOF Processing - Perform Cloud Agnostic Capability check for each <cloud owner, cloud region>. OOF will prune any <cloud owner, cloud region> which is not satisfying the standardized capabilities.
Step 4. OOF → MC - Push Cloud Agnostic Policy for the Service Instance
4a) OOF Processing
The enhanced OOF ↔ MC capacity check API, described below, is filled based on the enhanced Capacity Check & Cloud Selection Policy for Homing retrieved in step 2) – need OOF code changes.
Code Block | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||
//flexibility of having cloud owner and region in the new API provides fine grained control, addresses capacity/cost differences across different cloud owners/regions and ensures backward compatibility OOF -> MC { "VNFC": "vgw", //"vgw" is also interchangeably used as "vg" //new in R3 {//new in R3 "cloudOwner": "Azure", "cost-intent": { "dollarCostEvaluationVM-Type": "TRUE", //evaluate dollar cost per VM type if operator has configured a policy "dollarCostEvaluationVM-FeatureGroup": "TRUE" //evaluate dollar per feature/group of features if operator has configured a policy //evaluate dollar per group of features if operator has configured a policy //deployment-intent will be mapped to a cloud-specific realization which will be mapped to a feature group. } }, {//new in R3 "cloudOwner": "VMware VIO", //can be a specific cloud owner such as Azure, VMware VIO, Wind River Titanium Cloud etc. "deployment-intent": { "name": "Infrastructure Resource Isolation for VNF", // realization possible without dedicating CPU and Memory, refer to section on "Cloud Resource Partitioning for Differentiated QoS" // on how this can help in offering tiered services "qosProperty": { {"Burstable QoS": "TRUE", "Burstable QoS Oversubscription Percentage": "25"} } } }, {//new in R3 + R2 "capacityProperty": //same as R2, presence of this means capacity check needs to be done for the <cloud owner, cloud region> { "request": "{\"vCPU\": {\"quantity\": {\"get_param\": \"REQUIRED_VCPU\"}, \"Memory\": {\"quantity\": {\"get_param\": \"REQUIRED_MEM\"}, \"unit\": \"GB\"}, \"Storage\": {\"quantity\": {\"get_param\": \"REQUIRED_DISK\"}, \"unit\": \"GB\"}}" } }, } //return netValue per <cloud owner, cloud region> //cloud regions which fail capacity check are not in this list MC -> OOF { { "cloudOwner": "OpenStack", "cloudRegion": "1", "netValue": "99" }, { "cloudOwner": "VIO", "cloudRegion": "5", "netValue": "100" }, { "cloudOwner": "Azure", "cloudRegion": "3", "netValue": "101" }, } |
Code Block | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||
// //Policy relevant to MC Azure Plugin // //<Service, VNFC> to instanceType Mapping { { "Service": "vCPE" "VNFC": "vgw", //"vgw" is also intechangeably used as "vg" "cloudOwner": "Azure", "instanceType": "x1", "AvailabilitySetFeatureGroup": "G1" //availability set is a realization of the Intent "Infra High Availability (HA) for VMs in a VNF" } } //workload deployment cost for instance type per cloud region { { "cloudRegion": "5", "instanceType": "x1", "featureGroup": "G1", "workloadDeploymentBaseVMCost": "100", "workloadDeploymentFeatureGroupCost": "0" }, { "cloudRegion": "10", "instanceType": "x1", "featureGroup": "G1", "workloadDeploymentBaseVMCost": "120" "workloadDeploymentFeatureGroupCost": "0" }, } // //Policy Relevant to VIO plugin // //fixed workload deployment cost { "workloadDeploymentCost": "100", } |
5a) MC Processing (need MC code changes)
For each cloud owner
- Parse OOF → MC Policy (Intent) API
- If a Cloud owner does not support a specific "deployment-intent"
- Drop all the cloud regions for the cloud owner from the candidate list
- For each cloud region // Public cloud could have different costs in different geographic locations
- Compute net_value based on cost
- net_value = net_value + workload_deployment_cost
- If Plugin of cloud owner supports cost based on "dollarCostEvaluationVM-Type" and/or "dollarCostEvaluationVM-FeatureGroup"
- The workload deployment cost is computed per <instance type, cloud region> based on workload deployment cost policy described in Step 5b).
- Instance Type is derived from <Service, VNFC, cloud owner>
- More details are in 5b)
- Implementation Notes:
- It is not mandatory for all plugins to implement this feature since the OOF → MC API has the flexibility of turning on this feature per <cloud owner, cloud region>
- The workload deployment cost is computed per <instance type, cloud region> based on workload deployment cost policy described in Step 5b).
- Else
- The workload deployment cost is computed as a fixed cost per plugin
- If Plugin of cloud owner supports cost based on "dollarCostEvaluationVM-Type" and/or "dollarCostEvaluationVM-FeatureGroup"
- net_value = net_value + workload_deployment_cost
- Capacity Check
- Private Clouds (OpenStack based)
- Perform capacity check per specified Tenant (OpenStack Project)
- The tenant is not currently passed in the OOF → MC API; this is figured out from simple mapping function; A simple mapping would be a tenant per <cloud owner, cloud region> as part of Multi-VIM plugin configuration.
- If Capacity check fails, drop the cloud region out of the candidate list
- Perform capacity check per specified Tenant (OpenStack Project)
- Public Clouds or Other Clouds
- Capacity check always succeeds //assumption: public cloud has infinite capacity
- Private Clouds (OpenStack based)
- Compute net_value based on cost
5b) Workload Deployment Cost Policy - Configured by the Operator
The operator/service provider who uses ONAP will choose which VIMs to use and include the appropriate MC plugins in his ONAP deployment. For example, let’s assume they pick private Openstack, private VMWare, and public Azure as the platform to run their services on.
For R3, Workload Deployment Cost Policy can be stored in the form of configuration file(s) in the OOM K8S Persistent Volumes visible to the relevant MC plugin to simplify implementation. Beyond R3, this could be moved to the Policy DB. The details of the configuration are described below.
- Plugin of cloud owner can support cost based on "dollarCostEvaluationVM-Type" and/or "dollarCostEvaluationVM-FeatureGroup"
- Where workload deployment cost includes dollar cost of VM Instance Type (based on <Service, VNFC, cloud owner>) and dollar cost (or discount) of other cloud-specific feature groups corresponding to the intent expressed under the deployment-intent keyword in the OOF → MC API
- As an example, with respect to the deployment-intent, "Infrastructure Resource Isolation for VNF" with "Burstable QoS" can yield potential cost savings as compared to "Guaranteed QoS" by allowing smart over-subscription while still guaranteeing isolation
- Where workload deployment cost includes dollar cost of VM Instance Type (based on <Service, VNFC, cloud owner>) and dollar cost (or discount) of other cloud-specific feature groups corresponding to the intent expressed under the deployment-intent keyword in the OOF → MC API
- Note that the operator is free to choose the method of calculating the cost which includes initial cost, support cost & operational cost.
- Note that the operator is free to choose what time duration the cost metric is specified for each of the MultiVIM plugins (e.g., cost per hour, cost per month) since they will do it consistently for each of the VIMs.
"Workload Deployment Cost Policy Example" depicted above has an exemplary description of this.
Step 5. MC → OOF – Return a net value for each <cloud owner, cloud region> if the capacity check succeeds
6a) OOF Processing - cloud_net_value input in Multi-objective Optimization (need OOF code changes)
Each service specifies an service-specific objective function that is stored as part of the service-specific policy and is used by OOF to evaluate the candidate <cloud owner, cloud region>. For simplicity of the example, let’s consider service that consists only of one VNF instance. The objective function has two components:
- distance from customer location to the VNF - the service designed assigns a weight for the distance: wd
- the cost of deploying the VNF in a location - the service designer assigns a weight for the cost: wc
OOF optimization function: min (wd*distance + wc*cloud_net_value)
If the service does not care about the cost at all, it would set wc = 0. If the service designer wants to minimize cost, he could set wd=0. Note that candidates that are too far can be eliminated by a distance constraint even before the optimization. For example, if the service has a distance constraint of at most 100 kilometers, then only those <cloud owner, cloud region> within 100 kilometers to the customer location would be considered in the objective function evaluation.
If the service designer wants to trade off between distance and cost, for example, they might set wd = 1, wc = 2. This would mean that one $1 increase in price is as valuable as 2 kilometers in distance.
<cloud owner, cloud region> Candidate 1: $100, 100 kilometers => value: 300
<cloud owner, cloud region> Candidate 2: $150, 80 kilometers => value: 380
<cloud owner, cloud region> Candidate 3: $50, 190 kilometers => value: 290 <- pick this one
Step 6. OOF → SO - Return the target <cloud owner, cloud region> for the Service Instance
Step 7. SO → MC - Deploy VNF template in the target <cloud owner, cloud region> for the Service Instance
7a) MC Processing (need MC code changes)
- Parse Template (e.g. OpenStack Heat Template)
- For each VNFC, instance type in the template
- Fetch Cloud-Agnostic Workload Deployment Policy (Intent) based on <Service (e.g. vCPE), VNFC (e.g. vGW)>
- Value/Content: <Policy JSON>
- Note: The policy details are in Section 7b).
- Parse Policy JSON
- Modify template according to Intent
- Intent examples of interest for R3
- "Infrastructure High Availability (HA) for VNF"
- "Infrastructure Resource Isolation for VNF"
- "Burstable QoS"
- "Infrastructure Resource Isolation for VNF"
- "Guaranteed QoS"
- Intent examples of interest for R3
- Fetch Cloud-Agnostic Workload Deployment Policy (Intent) based on <Service (e.g. vCPE), VNFC (e.g. vGW)>
- For each VNFC, instance type in the template
Policy (Intent) Realization
- "Infrastructure High Availability (HA) for VNF"
- OpenStack-based Cloud realization
- For R3, Host-based anti-affinity using server groups //Beyond R3, Support other anti-affinity models at availability zone level etc.
- Implementation Notes:
- Instance "count" in heat template specifies VNFC scale out factor
- While dynamic injection of server group into heat template is ideal, a simple starting point could be just switching to an alternate heat template which is identical to the deployment template and additionally has server group
- Azure realization
- Availability Set?
- OpenStack-based Cloud realization
"Infrastructure Resource Isolation for VNF" – { "qosProperty": { {"Burstable QoS": "TRUE", "Burstable QoS Oversubscription Percentage": "25"} } }
OpenStack-based VMware VIO Cloud realization
- This can be achieved through min guarantee -- Max or limit (upper bound) & Min or Reservation (guarantee) are part of OpenStack flavor metadata
- Example
- VNFC with "Guaranteed QoS"
- "flavor-xyz-no-oversubscription"
- vCPU (Min/Max) - 16, Mem (Min/Max) - 32GB
- Same VNFC with "Burstable QoS", 25% over-subscription
- "flavor-xyz-25-percent-oversubscription"
- vCPU (Min) - 16, Mem (Min) - 32GB
- vCPU (Max) - 20, Mem (Max) - 40GB
- VNFC with "Guaranteed QoS"
- Only certain pre-defined over-subscription values are allowed to simplify implementation
- Implementation Notes:
- While dynamic injection of limit/reservation into flavor is ideal, a simple starting would be to be to switch to a pre-defined flavor in the environment file
- For aforementioned example
- Original flavor - "flavor-xyz-no-oversubscription"
- Modified flavor based on Policy - "flavor-xyz-25-percent-oversubscription"
- For aforementioned example
- While dynamic injection of limit/reservation into flavor is ideal, a simple starting would be to be to switch to a pre-defined flavor in the environment file
- Example
- Implementation Notes:
- From an implementation stand point, MC could be exposing a Workload Deployment Policy (Intent) API
- Input : deployment-intent, cloud owner, cloud region, deployment template, deployment environment file, ...
- Output : Success or Failure with reason, modified deployment template, modified deployment environment file, ...
- From an implementation stand point, MC could be exposing a Workload Deployment Policy (Intent) API
- "Infrastructure High Availability (HA) for VNF"
7b) Cloud-Agnostic Workload Deployment Policy (Intent)
- For R3, Cloud-Agnostic Workload Deployment Policy (Intent) can be stored in the form of configuration file(s) in the OOM K8S Persistent Volumes to simplify implementation.
- For R4 and beyond, this policy (optionally with specific realization options) will be passed from SO → MC. This is captured in the R4 and beyond workflow (Edge Scoping - Casablanca Stretch Goal/Beyond Casablanca).
- This policy is exactly the same as the policies with "deployment-intent" in the Enhanced Capacity Check & Cloud Selection Policy for Homing described in Section 2.
- An exemplary policy is depicted below.
Code Block | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||
//Policy Example 1 - VNFC VGW which is part of vCPE service { "Service": "vCPE" "VNFC": "vgw", //"vgw" is also intechangeably used as "vg" { "cloudOwner": "VMware VIO", //can be a specific cloud owner such as Azure, VMware VIO, Wind River Titanium Cloud etc. "deployment-intent": { "name": "Infrastructure Resource Isolation for VNF", // realization possible without dedicating CPU and Memory, refer to section on "Cloud Resource Partitioning for Differentiated QoS" // on how this can help in offering tiered services "qosProperty": { {"Burstable QoS": "TRUE", "Burstable QoS Oversubscription Percentage": "25"} } } } } //Policy Example 2 - VNFC vDNS which is part of vLoadBalancer/vDNS service { "Service": "vDNS", "VNFC": "vDNS", //By default, Policy (Intent) is applicable to all cloud owners/regions unless specified. { "deployment-intent": { "name": "Infrastructure High Availability (HA) for VNF", } }, { "deployment-intent": { "name": "Infrastructure Resource Isolation for VNF", // realization possible without dedicating CPU and Memory, refer to section on "Cloud Resource Partitioning for Differentiated QoS" // on how this can help in offering tiered services "qosProperty": { {"Guaranteed QoS": "TRUE"} } } } } |
Follow ups:
- Use Cases for Integration testing
- vCPE
- In the current state, this use case cannot support the intent "Infra HA for VMs in a VNF"
- vDNS
- Can support intent "Infra HA for VMs in a VNF" and "Infrastructure Resource Isolation for VNF"
- Nothing additional needed in OOF or MC
- Changes needed in SO to call OOF API
- Matti to follow up with Shankar
- vCPE
- Policy DB – is there any restriction on the type of json objects that can be stored?
- Matti to follow up with Ankit
Implementation trade offs for Casablanca (R3) and potential Dublin (R4) plan:
- Deployment-Intent
- 1. "Infrastructure Resource Isolation for VNF" – { "qosProperty": { {"Burstable QoS": "TRUE", "Burstable QoS Oversubscription Percentage": "25"} } }
- Casablanca Plan
- Only certain pre-defined over-subscription values are allowed to reflect practical deployment and simplify implementation
- Dublin & Beyond Potential Plan
- Creating instance types on demand for private clouds - to study
- Casablanca Plan
- 2. Cloud-agnostic Workload Deployment Policy (Intent) is not passed in the OOF → SO API & SO → MC API
- Casablanca Plan
- Cloud-Agnostic Workload Deployment Policy (Intent) can be stored in the form of Multi-Cloud configuration file(s) in OOM K8S Persistent Volumes to simplify implementation.
- Dublin & Beyond Potential Plan
- Pass this policy (optionally with specific realization options) from OOF → SO. SO passes the policy and associated information transparently from SO → MC. This is captured in the R4 and beyond workflow (Edge Scoping - Casablanca Stretch Goal/Beyond Casablanca).
- Casablanca Plan
- 1. "Infrastructure Resource Isolation for VNF" – { "qosProperty": { {"Burstable QoS": "TRUE", "Burstable QoS Oversubscription Percentage": "25"} } }
- Policy-based capacity check & cloud-selection
- 3. Tenant Information is not passed in the OOF → MC API
- Casablanca Plan
- The tenant information is derived from a simple mapping function per <cloud owner, cloud region>
- A simple mapping would be a tenant per <cloud owner, cloud region> as part of Multi-VIM plugin configuration.
- Need to make sure that this scheme is synchronous with the SO → MC API path
- The tenant information is derived from a simple mapping function per <cloud owner, cloud region>
- Dublin & Beyond Potential Plan
- Pass Tenant Information per <cloud owner, cloud region> in the OOF → MC API
- Casablanca Plan
- 4. Capacity Check for Public Clouds is not supported
- Casablanca Plan
- Public Clouds always pass the capacity check
- Dublin & Beyond Potential Plan
- Quota in Public Clouds - to study
- Casablanca Plan
5. VM Instance type/VM Feature Group dollar-cost-based cloud selection
Casablanca Plan
VM Instance type/VM Feature Group dollar-cost-based cloud selection is not mandatory for all Multi-Cloud Plugins
If a Multi-Cloud Plugin does not support cost based on "dollarCostEvaluationVM-Type" and/or "dollarCostEvaluationVM-FeatureGroup"
- The workload deployment cost is computed as a fixed cost per plugin
Dublin & Beyond Potential Plan
- Deep dive further on dollar-cost-based cloud selection models/implementation for public/private clouds
6. The workload instance type model (Note 1 below) is not passed from OOF → MC. Rationale below.
Currently, the VNFC model supports only compute/memory/local storage. A comprehensive VNFC model needs to include HW generation (Intel Broadwell/Haswell, ARM etc.), HPA details (SR-IOV etc.) and more.
Casablanca Plan
Since the VNFC model standardization is in progress, VNFC mapping to VM Instance Type (OpenStack Flavor, Azure M4 etc.) happens in the Multi-Cloud plugin through configuration.
- Note 1: An exception to the rule is a simple capacity check based on VM compute/memory/local storage which can be supported by the current instance type model.
- Dublin & Beyond Potential Plan
- Align with the modeling work and make progress
- 3. Tenant Information is not passed in the OOF → MC API
Cloud Resource Partitioning for Differentiated QoS (Combined with Previous)
Value:
- Applicable to all use cases
- Casablanca Targets:
- vCPE (Enable Tiered service offering); 5G Network Slicing (Stretch Goal)
References:
Edge Automation Requirement:
Support three types of slices in the Cloud Infrastructure (Definition Reference: https://kubernetes.io/docs/tasks/configure-pod-container/quality-service-pod/)
- Guaranteed Resource Slice (hard isolation) for various infra Resources (CPU/Memory/Network)
- Max (limit), Min (request) are the same; resource guarantee is "Max"
- Maps to 5G Applications such as Connected Car which fall in the category of ultra-reliable machine-type communications (ref. 1)
- Burstable Resource Slice (soft isolation) for various infra Resources
- Min (request) <= Max (limit); resource guarantee is "Min"
- Maps to Burstable Network Slice such > 1Gbps broadband which fall in the category of extreme mobile broadband (ref. 1)
- Best Effort Resource Slice (no isolation) for various infra Resources
- No Min (request) ; resource guarantee is "None"
- Maps to 5G Applications such as IoT which fall in the category of massive machine-type communications (ref. 1)
Implementation:
- Leverage current HPA framework with appropriate extensions
References:
- https://metis-ii.5g-ppp.eu/wp-content/uploads/white_papers/5G-RAN-Architecture-and-Functional-Design.pdf
Driving Superior Isolation for Tiered Services using Resource Reservation -- Optimization Policies for Residential vCPE
-https://jira.onap.org/browse/OPTFRA-240
Note:
- Any VMs/Containers which are part of a resource slice will adhere to the specs of the resource slice
ONAP Component | Life Cycle Phase | Enhancements |
---|---|---|
Policy | Design | Configuration Policies for Guaranteed, Burstable & Best Effort Cloud Infrastructure Resource Slices (this will apply to VMs/Containers also) Placement Policies for Resource Slices
|
Multi-Cloud | Deploy | Resource Slice Capability Discovery |
A&AI | Deploy | Resource Slice Capability per Cloud Region
Resource Slice Type
|
OOF | Deploy | Execute Resource Slice Placement Policies for Optimized Service/VNF Placement across Cloud Regions |
Aggregated Infrastructure Telemetry Streams (Aligns with HPA requirements, Combining efforts with HPA)
Value
Edge Infrastructure Analytics complementing 5G VNF Analytics
Jira server ONAP JIRA serverId 425b2b0a-557c-3c0c-b515-579789cceedb key MULTICLOUD-254
ONAP, as in R2, collects the statistics/alarms/events from workloads (VMs) and take any close loop control actions such as Heal a process, scale-out, restart etc.. In R3, infrastructure related statistics/alarms/events will be collected, generate actionable insights and take life cycle actions on the workloads. Infrastructure statistics normally include performance counters, NIC counters, IPMI information on per physical server node basis. To reduce the load on the ONAP, it is necessary that aggregated (summarized) information is sent to the ONAP from edge-clouds.
As part of this activity, intention is to create aggregation micro-service that collects the data from physical nodes (over collected and other mechanisms), aggregate the information (time based aggregation, threshold based aggregation, silencing etc.,..) based on the configurable rules and export the aggregate data to DCAE. This micro service can be instantiated by ONAP itself - one or more instances for edge-clouds at the ONAP-central itself using OOM, it could be instantiated at the edge-cloud using their own deployment tools or it could be deployed edge service providers at the regional site level.
Impacted projects (development activities)
ONAP Component | Enhancements |
---|---|
Overall |
|
Multi-Cloud |
|
AAI & ESR |
|
PORTAL | ESR portal related changes to take information about the edge-cloud (CA Cert and UN/PWD information) |
DCAE & DMAPP | None expected?? |
Life Cycle stages related functions
ONAP Component | Life cycle phase | Activities |
---|---|---|
AAI and ESR | Deploy & Run time |
|
AAI and ESR | Run time |
|
Multi-Cloud | Run time |
|
High level architecture slides:
View file | ||||
---|---|---|---|---|
|
ONAP Edge Analytics with DCAE/DMaaP independent of closed loop (Beyond Casablanca)
Value
- 5G Analytics
ONAP Component | Life cycle phase | Enhancements |
---|---|---|
OOM - ONAP Central | Deploy |
|
Multi-Cloud Deployment in Edge Cloud (Stretch Goal)
Jira server ONAP JIRA serverId 425b2b0a-557c-3c0c-b515-579789cceedb key MULTICLOUD-262
Value:
- Multi-Cloud service to assist in central A&AI scaling by caching A&AI data locally and syncing up with A&AI periodically