Versions Compared

Old Version 149

changes.mady.by.user ramki krishnan

Saved on

compared with

New Version 150

changes.mady.by.user ramki krishnan

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Follow ups:

  • Policy DB – is there any restriction on json objects store? 
    • Matti to follow up with Ankit

Implementation trade offs for Casablanca (R3) and potential Dublin (R4) plan:

  • Deployment-Intent 
    • 1. "Infrastructure Resource Isolation for VNF" – { "qosProperty": { {"Burstable QoS": "TRUE", "Burstable QoS Oversubscription Percentage": "25"} } }
      • Casablanca Plan
        • Only certain pre-defined over-subscription values are allowed to reflect practical deployment and simplify implementation 
      • Dublin & Beyond Potential Plan
        • Creating instance types on demand for private clouds - to study
    • 2. Cloud-agnostic Workload Deployment Policy (Intent) is not passed in the SO → MC API
      • Casablanca Plan
        • Cloud-Agnostic Workload Deployment Policy (Intent) can be stored in the form of Multi-Cloud configuration file(s) in OOM K8S Persistent Volumes to simplify implementation.
      • Dublin & Beyond Potential Plan
        • Pass this policy (optionally with specific realization options) from SO → MC. This is captured in the R4 and beyond workflow (Edge Scoping - Beyond Casablanca). 
  • Policy-based capacity check & cloud-selection
    • 3. Tenant Information is not passed in the OOF → MC API
      • Casablanca Plan
        • The tenant information is derived from a simple mapping function per <cloud owner, cloud region>
          • A simple mapping would be a tenant per <cloud owner, cloud region> as part of Multi-VIM plugin configuration.
          • Need to make sure that this scheme is synchronous with the SO → MC API path
      • Dublin & Beyond Potential Plan
        • Pass Tenant Information per <cloud owner, cloud region> in the OOF → MC API
    • 4. Capacity Check for Public Clouds is not supported
      • Casablanca Plan
        • Public Clouds always pass the capacity check
      • Dublin & Beyond Potential Plan
        • Quota in Public Clouds - to study

    • 5.  The workload instance type model (Note 1 below) is not passed from OOF → MC. Rationale below.

      • Currently, the VNFC model supports only compute/memory/local storage. A comprehensive VNFC model needs to include HW generation (Intel Broadwell/Haswell, ARM etc.), HPA details (SR-IOV etc.) and more.  

      • Casablanca Plan

        • Since the VNFC model standardization is in progress, VNFC mapping to VM Instance Type (OpenStack Flavor, Azure M4 etc.) happens in the Multi-Cloud plugin through configuration. 

        • Note 1: An exception to the rule is a simple capacity check based on VM compute/memory/local storage which can be supported by the current instance type model.
      • Dublin & Beyond Potential Plan
        • Align with the modelling work and make progress

1. SO → OOF - Get Target <Cloud Owner, Cloud Region> for the Service Instances

2. OOF → Policy - Fetch Enhanced Capacity Check & Cloud Selection Policy for Homing 

2a) OOF Processing - the fetched Policy (example below) is stored in a local data structure and is available for further use (need OOF code changes).

Step 1. SO → OOF - Get Target <Cloud Owner, Cloud Region> for the Service Instances

Step 2. OOF → Policy - Fetch Enhanced Capacity Check & Cloud Selection Policy for Homing 

2a) OOF Processing - the fetched Policy (example below) is stored in a local data structure and is available for further use (need OOF code changes).

Code Block
languagepy
themeEmacs
titleOOF Homing Enhanced Capacity Check & Cloud Selection Policy Example (Step 2a)
linenumberstrue
collapsetrue
{

"service": "cloudSelectionPolicy",
"policyName": "oofMulti-cloudCasablanca.cloudSelectionPolicy_vCPE_VNF",
"description": "Cloud Selection Policy for vCPE VNFs",
"templateVersion": "0.0.1",
"version": "oofMulti-cloudCasablanca",
"priority": "3",
"riskType": "test",
"riskLevel": "2",
"guard": "False",

"content": 
{
	{//new in R3
		"cloudOwner": "All", //can be a specific cloud owner such as Azure, VMware VIO, Wind River Titanium Cloud etc. 
		"cloudRegion": "All", //can be a specific cloud region for a cloud owner
		"cost-intent":
		{
			"dollarCostEvaluationVM-Type": "TRUE", //evaluate dollar cost per VM type if operator has configured a policy
			"dollarCostEvaluationVM-FeatureGroup": "TRUE" 
				//evaluate dollar per group of features if operator has configured a policy
				//deployment-intent will be mapped to a cloud-specific realization which will be mapped to a feature group. 
		}
	},			 			  	
  	
	{//new in R3
		"cloudOwner": "VMware VIO", //can be a specific cloud owner such as Azure, VMware VIO, Wind River Titanium Cloud etc.
		"cloudRegion": "All", //can be a specific cloud region for a cloud owner
		"deployment-intent": 
		{
			"name": "Infrastructure Resource Isolation for VNF", 
				// realization possible without dedicating CPU and Memory, refer to section on "Cloud Resource Partitioning for Differentiated QoS" 
				// on how this can help in offering tiered services
			"qosProperty": 
			{
				{"Burstable QoS": "TRUE", "Burstable QoS Oversubscription Percentage": "25"}
			}
		}
	},
Code Block
languagepy
themeEmacs
titleOOF Homing Enhanced Capacity Check & Cloud Selection Policy Example (Step 2a)
linenumberstrue
collapsetrue
{

"service": "cloudSelectionPolicy",
"policyName": "oofMulti-cloudCasablanca.cloudSelectionPolicy_vCPE_VNF",
"description": "Cloud Selection Policy for vCPE VNFs",
"templateVersion": "0.0.1",
"version": "oofMulti-cloudCasablanca",
"priority": "3",
"riskType": "test",
"riskLevel": "2",
"guard": "False",

"content": 
{
	{//new in R3
		"cloudOwner": "All", //can be a specific cloud owner such as Azure, VMware VIO, Wind River Titanium Cloud etc. 
		"cloudRegion": "All", //can be a specific cloud region for a cloud owner
		"costdeployment-intent": 
		{
			"dollarCostEvaluationVM-Typename": "TRUE", //evaluate dollar cost per VM type if operator has configured a policy
			"dollarCostEvaluationVM-FeatureGroup": "TRUE" 
				//evaluate dollar per group of features if operator has configured a policy
				//deployment-intent will be mapped to a cloud-specific realization which will be mapped to a feature group. 
		}
	},			 			  	
  	
	{//new in R3
		"cloudOwner": "VMware VIO", //can be a specific cloud owner such as Azure, VMware VIO, Wind River Titanium Cloud etc.
		"cloudRegion": "All", //can be a specific cloud region for a cloud owner
		"deployment-intent": 
		{
			"name": "Infrastructure Resource Isolation for VNF", 
				// realization possible without dedicating CPU and Memory, refer to section on "Cloud Resource Partitioning for Differentiated QoS" 
				// on how this can help in offering tiered services
			"qosProperty": 
			{
				{"Burstable QoS": "TRUE", "Burstable QoS Oversubscription Percentage": "25"}
			}
		}
	},
	
	{//new in R3
		"cloudOwner": "All"Infrastructure High Availability (HA) for VNF", 
		}
	}, 		  	
}

"resources": ["vgw", "vgmux"], //"vgw" is also interchangeably used as "vg"
"applicableResources": "any",
"identity": "cloud-atrributes",
"policyScope": ["vCPE", "US", "INTERNATIONAL", "ip", "vgw", "vgmux"],
"policyType": "AllPolicy"

}

Step 3. OOF → A&AI - Fetch Cloud-Agnostic (Standardized) Capabilities for the Service Instance

3a) OOF Processing - Perform Cloud Agnostic Capability check for each <cloud owner, cloud region>. OOF will prune any <cloud owner, cloud region> which is not satisfying the standardized capabilities.

Step 4. OOF → MC - Push Cloud Agnostic Policy for the Service Instance 

4a) OOF Processing

The enhanced OOF ↔ MC capacity check API, described below, is filled based on the enhanced Capacity Check & Cloud Selection Policy for Homing retrieved in step 2) – need OOF code changes.

Code Block
languagepy
themeEmacs
titleOOF <-> MC API Examples (Step 4a)
linenumberstrue
collapsetrue
//flexibility of having cloud owner and region in the new API provides fine grained control, addresses capacity/cost differences across different //cloud owners/regions and ensures backward compatibility

OOF -> MC
{
	"VNFC": "vgw", //"vgw" is also intechangeably used as "vg" //new in R3

	{//new in R3
		"cloudOwner": "All", //can be a specific cloud owner such as Azure, VMware VIO, Wind River Titanium Cloud etc. 
		"cloudRegion": "All", //can be a specific cloud region for a cloud owner
		"deploymentcost-intent": 
		{
			"namedollarCostEvaluationVM-Type": "TRUE"Infrastructure, High//evaluate Availabilitydollar (HA)cost for VNF", 
		}
	}, 		  	
}

"resources": ["vgw", "vgmux"], //"vgw" is also interchangeably used as "vg"
"applicableResources": "any",
"identity": "cloud-atrributes",
"policyScope": ["vCPE", "US", "INTERNATIONAL", "ip", "vgw", "vgmux"],
"policyType": "AllPolicy"

}

3. OOF → A&AI - Fetch Cloud-Agnostic (Standardized) Capabilities for the Service Instance

3a) OOF Processing - Perform Cloud Agnostic Capability check for each <cloud owner, cloud region>. OOF will prune any <cloud owner, cloud region> which is not satisfying the standardized capabilities.

4. OOF → MC - Push Cloud Agnostic Policy for the Service Instance 

4a) OOF Processing

The enhanced OOF ↔ MC capacity check API, described below, is filled based on the enhanced Capacity Check & Cloud Selection Policy for Homing retrieved in step 2) – need OOF code changes.

Code Block
languagepy
themeEmacs
titleOOF <-> MC API Examples (Step 4a)
linenumberstrue
collapsetrue
//flexibility of having cloud owner and region in the new API provides fine grained control, addresses capacity/cost differences across different //cloud owners/regions and ensures backward compatibility

OOF -> MC
{
	"VNFC": "vgw", //"vgw" is also intechangeably used as "vg" //new in R3

	{//new in R3
		"cloudOwner": "All", //can be a specific cloud owner such as Azure, VMware VIO, Wind River Titanium Cloud etc. 
		"cloudRegion": "All", //can be a specific cloud region for a cloud owner
		"cost-intent":
		{
			"dollarCostEvaluationVM-Typeper VM type if operator has configured a policy
			"dollarCostEvaluationVM-FeatureGroup": "TRUE" //evaluate dollar per feature/group of features if operator has configured a policy
				//evaluate dollar per group of features if operator has configured a policy
				//deployment-intent will be mapped to a cloud-specific realization which will be mapped to a feature group. 
		}
	},			 			  	
  	
	{//new in R3
		"cloudOwner": "VMware VIO", //can be a specific cloud owner such as Azure, VMware VIO, Wind River Titanium Cloud etc.
		"cloudRegion": "All", //can be a specific cloud region for a cloud owner
		"deployment-intent": 
		{
			"name": "Infrastructure Resource Isolation for VNF", 
				// realization possible without dedicating CPU and Memory, refer to section on "Cloud Resource Partitioning for Differentiated QoS" 
				// on how this can help in offering tiered services
			"qosProperty": 
			{
				{"Burstable QoS": "TRUE", //evaluate dollar cost per VM type if operator has configured a policy
			"dollarCostEvaluationVM-FeatureGroup": "TRUE" //evaluate dollar per feature/group of features if operator has configured a policy
				//evaluate dollar per group of features if operator has configured a policy
				//deployment-intent will be mapped to a cloud-specific realization which will be mapped to a feature group. 
		}
	},			 			  	
  "Burstable QoS Oversubscription Percentage": "25"}
			}
		}
	},
	
	{//new in R3
		"cloudOwner": "All", //can be a specific cloud owner such as Azure, VMware VIO, Wind River Titanium Cloud etc. 
		"cloudRegion": "All", //can be a specific cloud region for a cloud owner
		"deployment-intent": 
		{
			"name": "Infrastructure High Availability (HA) for VNF", 
		}
	},
	
	{//new in R3 + R2
		"cloudOwner": "VMware VIOOpenStack", 	//can benew ain specific cloud owner such as Azure, VMware VIO, Wind River Titanium Cloud etc.R3, 
								   	// different cloud owners may need different capacities for the same VNFC because of implementation differences
		"cloudRegion": "All", 		//can benew a specific cloud region for a cloud ownerin R3, 
		"deployment-intent": 
		{
			"name": "Infrastructure Resource Isolation for VNF", 
				// realizationdifferent possiblecloud withoutregions dedicatingfor CPUa andcloud Memory,owner refermay toneed sectiondifferent oncapacities "Cloudfor Resourcethe Partitioningsame VNFC fordue Differentiatedto QoS"different SW
							 		// on how this can help in offering tiered services
			"qosProperty": 
			{ versions and HW configuration	

		"capacityProperty": //same as R2, presence of this means capacity check needs to be done for the <cloud owner, cloud region>
		{ 		 		
			"request": 
				"{"Burstable QoS\"vCPU\": {\"quantity\": {\"get_param\": \"TRUEREQUIRED_VCPU\"}, \"Memory\"Burstable QoS Oversubscription Percentage: {\"quantity\": {\"get_param\": \"25REQUIRED_MEM\"}
	, 		}
		}
	},
	
	{//new in R3
		"cloudOwner	\"unit\": 	\"AllGB\"}, //can be a specific cloud owner such as Azure, VMware VIO, Wind River Titanium Cloud etc. 
		"cloudRegion": "All", //can be a specific cloud region for a cloud owner
		"deployment-intent": 
		{
			"name": "Infrastructure High Availability (HA) for VNF", 
		}
	},
	
	{//new in R3 + R2\"Storage\": {\"quantity\": {\"get_param\": \"REQUIRED_DISK\"}, \"unit\": \"GB\"}}"
		}		
	},
}

//return netValue per <cloud owner, cloud region>
//cloud regions which fail capacity check are not in this list
MC -> OOF
{
	{
		"cloudOwner": "OpenStack",
		"cloudRegion": "1",
		"netValue": "99"
	},
	{
		"cloudOwner": "OpenStackVIO", 	// new in R3, 
								   	// different cloud owners may need different capacities for the same VNFC because of implementation differences
		"cloudRegion": "5",
		"netValue": "100"
	},
	{
		"cloudOwner": "Azure",
		"cloudRegion": "All3", 
		// new in R3, 
							 		// different cloud regions for a cloud owner may need different capacities for the same VNFC due to different SW
							 		// versions and HW configuration	

		"capacityProperty": //same as R2, presence of this means capacity check needs to be done for the <cloud owner, cloud region>
		{ 		 		
			"request": 
				"{\"vCPU\": {\"quantity\": {\"get_param\": \"REQUIRED_VCPU\"}, \"Memory\": {\"quantity\": {\"get_param\": \"REQUIRED_MEM\"}, 	
				\"unit\": 	\"GB\"}, \"Storage\": {\"quantity\": {\"get_param\": \"REQUIRED_DISK\"}, \"unit\": \"GB\"}}"
		}		
	},
}

//return netValue per <cloud owner, cloud region>
//cloud regions which fail capacity check are not in this list
MC -> OOF
{
	{
		"cloudOwner": "OpenStack",
		"cloudRegion": "1",
		"netValue": "99"
	},
	{
		"cloudOwner": "VIO",
		"cloudRegion": "5",
		"netValue": "100"
	},
	{
		"cloudOwner": "Azure",
		"cloudRegion": "3",
		"netValue": "101"
	},
}
Code Block
languagepy
themeEmacs
titleWorkload Deployment Cost Policy Example (Step 5b)
linenumberstrue
collapsetrue
//
//Policy relevant to MC Azure Plugin
//

//<Service, VNFC> to instanceType Mapping
{
	{
		"Service": "vCPE"
		"VNFC": "vgw", //"vgw" is also intechangeably used as "vg"	
		"cloudOwner": "Azure",
		"instanceType": "x1",
		"AvailabilitySetFeatureGroup": "G1" //availability set is a realization of the Intent "Infra High Availability (HA) for VMs in a VNF"
	}
}

//workload deployment cost for instance type per cloud region 
{
	{
		"cloudRegion": "5",
		"instanceType": "x1",
		"featureGroup": "G1",
		"workloadDeploymentBaseVMCost": "100",
		"workloadDeploymentFeatureGroupCost": "0"
	}, 
	{
		"cloudRegion": "10",
		"instanceType": "x1",
		"featureGroup": "G1",
		"workloadDeploymentBaseVMCost": "120"
		"workloadDeploymentFeatureGroupCost": "0"
	}, 
}

5a) MC Processing (need MC code changes)

For each cloud owner

  • Parse OOF → MC Policy (Intent) API 
  • For each cloud region // Public cloud could have different costs in different geographic locations
    • Compute net_value based on cost
      • net_value = net_value + workload_deployment_cost 
        • The workload deployment cost is computed per <instance type, cloud region> based on workload deployment cost policy described in Step 5b).
          • Instance Type is derived from <Service, VNFC, cloud owner>
          • More details are in 5b)
        • Implementation Notes:
          • It is not mandatory for all plugins to implement this feature since the OOF → MC API has the flexibility of turning on this feature per <cloud owner, cloud region>
    • Capacity Check 
      • Private Clouds (OpenStack based)
        • Perform capacity check per specified Tenant (OpenStack Project)
          • The tenant is not currently passed in the OOF → MC API; this is figured out from simple mapping function; A simple mapping would be a tenant per <cloud owner, cloud region> as part of Multi-VIM plugin configuration.
        • If Capacity check fails, drop the cloud region out of the candidate list
      • Public Clouds or Other Clouds
        • Capacity check always succeeds //assumption: public cloud has infinite capacity

5b) Workload Deployment Cost Policy - Configured by the Operator

The operator/service provider who uses ONAP will choose which VIMs to use and include the appropriate MC plugins in his ONAP deployment. For example, let’s assume they pick private Openstack, private VMWare, and public Azure as the platform to run their services on.

For R3, Workload Deployment Cost Policy can be stored in the form of configuration file(s) in the OOM K8S Persistent Volumes visible to the relevant MC plugin to simplify implementation.  Beyond R3, this could be moved to the Policy DB. The details of the configuration are described below.

  • Workload deployment cost includes dollar cost of Instance Type (based on <Service, VNFC, cloud owner>) and dollar cost (or discount) of other cloud-specific feature groups corresponding to the intent expressed under the deployment-intent keyword in the OOF → MC API
    • As an example, with respect to the deployment-intent, "Infrastructure Resource Isolation for VNF" with "Burstable QoS" can yield potential cost savings as compared to "Guaranteed QoS" by allowing smart over-subscription while still guaranteeing isolation
  • Note that the operator is free to choose the method of calculating the cost which includes initial cost, support cost & operational cost. 
  • Note that the operator is free to choose what time duration the cost metric is specified for each of the MultiVIM plugins (e.g., cost per hour, cost per month) since they will do it consistently for each of the VIMs. 

"Workload Deployment Cost Policy Example" depicted above has an exemplary description of this.

5. MC →  OOF – Return a net value for each <cloud owner, cloud region> if the capacity check succeeds

6a) OOF Processing - cloud_net_value input in Multi-objective Optimization (need OOF code changes)

Each service specifies an service-specific objective function that is stored as part of the service-specific policy and is used by OOF to evaluate the candidate <cloud owner, cloud region>. For simplicity of the example, let’s consider service that consists only of one VNF instance. The objective function has two components:

- distance from customer location to the VNF - the service designed assigns a weight for the distance: wd

- the cost of deploying the VNF in a location - the service designer assigns a weight for the cost: wc

OOF optimization function: min (wd*distance + wc*cloud_net_value)

If the service does not care about the cost at all, it would set wc = 0. If the service designer wants to minimize cost, he could set wd=0. Note that candidates that are too far can be eliminated by a distance constraint even before the optimization. For example, if the service has a distance constraint of at most 100 kilometers, then only those <cloud owner, cloud region> within 100 kilometers to the customer location would be considered in the objective function evaluation.

If the service designer wants to trade off between distance and cost, for example, they might set wd = 1, wc = 2. This would mean that one $1 increase in price is as valuable as 2 kilometers in distance.

<cloud owner, cloud region> Candidate 1: $100, 100 kilometers => value: 300

<cloud owner, cloud region> Candidate 2: $150, 80 kilometers => value: 380

<cloud owner, cloud region> Candidate 3: $50, 190 kilometers => value: 290  <- pick this one

6. OOF → SO - Return the target <cloud owner, cloud region> for the Service Instance

7. SO → MC - Deploy VNF template in the target <cloud owner, cloud region> for the Service Instance

7a) MC Processing (need MC code changes)

  • Parse Template (e.g. OpenStack Heat Template)
    • For each VNFC, instance type in the template
      • Fetch Cloud-Agnostic Workload Deployment Policy (Intent) based on <Service (e.g. vCPE), VNFC (e.g. vGW)>
        • Value/Content: <Policy JSON> 
        • Note: The policy details are in Section 7b).
      • Parse Policy JSON
      • Modify template according to Intent 
        • Intent examples of interest for R3 
          • "Infrastructure High Availability (HA) for VNF" 
          • "Infrastructure Resource Isolation for VNF"   
            • "Burstable QoS"
          • "Infrastructure Resource Isolation for VNF"   
            • "Guaranteed QoS"

  • Policy (Intent) Realization

    • "Infrastructure High Availability (HA) for VNF"
      • OpenStack-based Cloud realization 
        • For R3, Host-based anti-affinity using server groups //Beyond R3, Support other anti-affinity models at availability zone level etc. 
        • Implementation Notes: 
          • Instance "count" in heat template specifies VNFC scale out factor 
          • While dynamic injection of server group into heat template is ideal, a simple starting point could be just switching to an alternate heat template which is identical to the deployment template and additionally has server group
      • Azure realization 
        • Availability Set?

    • "Infrastructure Resource Isolation for VNF" – { "qosProperty": { {"Burstable QoS": "TRUE", "Burstable QoS Oversubscription Percentage": "25"} } }

        • Example 
          • VNFC with "Guaranteed QoS" 
            • "flavor-xyz-no-oversubscription"
            • vCPU (Min/Max) - 16, Mem (Min/Max) - 32GB 
          • Same VNFC with "Burstable QoS", 25% over-subscription 
            • "flavor-xyz-25-percent-oversubscription"
            • vCPU (Min) - 16, Mem (Min) - 32GB 
            • vCPU (Max) - 20, Mem (Max) - 40GB  
        • Only certain pre-defined over-subscription values are allowed to simplify implementation
        • Implementation Notes:
          • While dynamic injection of limit/reservation into flavor is ideal, a simple starting would be to be to switch to a pre-defined flavor in the environment file
            • For aforementioned example
              • Original flavor - "flavor-xyz-no-oversubscription"
              • Modified flavor based on Policy - "flavor-xyz-25-percent-oversubscription" 
    • Implementation Notes:
      • From an implementation stand point, MC could be exposing a Workload Deployment Policy (Intent) API
        • Input : deployment-intent, cloud owner, cloud region, deployment template, deployment environment file, ...
        • Output : Success or Failure with reason, modified deployment template, modified deployment environment file, ...

7b) Cloud-Agnostic Workload Deployment Policy (Intent)

  • For R3, Cloud-Agnostic Workload Deployment Policy (Intent) can be stored in the form of configuration file(s) in the OOM K8S Persistent Volumes to simplify implementation.
    • For R4 and beyond, this policy (optionally with specific realization options) will be passed from SO → MC. This is captured in the R4 and beyond workflow (Edge Scoping - Beyond Casablanca).
  • This policy is exactly the same as the policies with "deployment-intent" in the Enhanced Capacity Check & Cloud Selection Policy for Homing described in Section 2.
  • An exemplary policy is depicted below.

...

languagepy
themeEmacs
titleCloud-Agnostic Workload Deployment Policy (Step 7b)
linenumberstrue
collapsetrue

...

"netValue": "101"
	},
}
Code Block
languagepy
themeEmacs
titleWorkload Deployment Cost Policy Example (Step 5b)
linenumberstrue
collapsetrue
//
//Policy relevant to MC Azure Plugin
//

//<Service, VNFC> to instanceType Mapping
{
	{
		"Service": "vCPE"
		"VNFC": "vgw", //"vgw" is also intechangeably used as "vg"	
		"cloudOwner": "Azure",
		"instanceType": "x1",
		"AvailabilitySetFeatureGroup": "G1" //availability set is a realization of the Intent "Infra High Availability (HA) for VMs in a VNF"
	}
}

//workload deployment cost for instance type per cloud region 
{
	{
		"cloudRegion": "5",
		"instanceType": "x1",
		"featureGroup": "G1",
		"workloadDeploymentBaseVMCost": "100",
		"workloadDeploymentFeatureGroupCost": "0"
	}, 
	{
		"cloudRegion": "10",
		"instanceType": "x1",
		"featureGroup": "G1",
		"workloadDeploymentBaseVMCost": "120"
		"workloadDeploymentFeatureGroupCost": "0"
	}, 
}

5a) MC Processing (need MC code changes)

For each cloud owner

  • Parse OOF → MC Policy (Intent) API 
  • For each cloud region // Public cloud could have different costs in different geographic locations
    • Compute net_value based on cost
      • net_value = net_value + workload_deployment_cost 
        • The workload deployment cost is computed per <instance type, cloud region> based on workload deployment cost policy described in Step 5b).
          • Instance Type is derived from <Service, VNFC, cloud owner>
          • More details are in 5b)
        • Implementation Notes:
          • It is not mandatory for all plugins to implement this feature since the OOF → MC API has the flexibility of turning on this feature per <cloud owner, cloud region>
    • Capacity Check 
      • Private Clouds (OpenStack based)
        • Perform capacity check per specified Tenant (OpenStack Project)
          • The tenant is not currently passed in the OOF → MC API; this is figured out from simple mapping function; A simple mapping would be a tenant per <cloud owner, cloud region> as part of Multi-VIM plugin configuration.
        • If Capacity check fails, drop the cloud region out of the candidate list
      • Public Clouds or Other Clouds
        • Capacity check always succeeds //assumption: public cloud has infinite capacity

5b) Workload Deployment Cost Policy - Configured by the Operator

The operator/service provider who uses ONAP will choose which VIMs to use and include the appropriate MC plugins in his ONAP deployment. For example, let’s assume they pick private Openstack, private VMWare, and public Azure as the platform to run their services on.

For R3, Workload Deployment Cost Policy can be stored in the form of configuration file(s) in the OOM K8S Persistent Volumes visible to the relevant MC plugin to simplify implementation.  Beyond R3, this could be moved to the Policy DB. The details of the configuration are described below.

  • Workload deployment cost includes dollar cost of Instance Type (based on <Service, VNFC, cloud owner>) and dollar cost (or discount) of other cloud-specific feature groups corresponding to the intent expressed under the deployment-intent keyword in the OOF → MC API
    • As an example, with respect to the deployment-intent, "Infrastructure Resource Isolation for VNF" with "Burstable QoS" can yield potential cost savings as compared to "Guaranteed QoS" by allowing smart over-subscription while still guaranteeing isolation
  • Note that the operator is free to choose the method of calculating the cost which includes initial cost, support cost & operational cost. 
  • Note that the operator is free to choose what time duration the cost metric is specified for each of the MultiVIM plugins (e.g., cost per hour, cost per month) since they will do it consistently for each of the VIMs. 

"Workload Deployment Cost Policy Example" depicted above has an exemplary description of this.

Step 5. MC →  OOF – Return a net value for each <cloud owner, cloud region> if the capacity check succeeds

6a) OOF Processing - cloud_net_value input in Multi-objective Optimization (need OOF code changes)

Each service specifies an service-specific objective function that is stored as part of the service-specific policy and is used by OOF to evaluate the candidate <cloud owner, cloud region>. For simplicity of the example, let’s consider service that consists only of one VNF instance. The objective function has two components:

- distance from customer location to the VNF - the service designed assigns a weight for the distance: wd

- the cost of deploying the VNF in a location - the service designer assigns a weight for the cost: wc

OOF optimization function: min (wd*distance + wc*cloud_net_value)

If the service does not care about the cost at all, it would set wc = 0. If the service designer wants to minimize cost, he could set wd=0. Note that candidates that are too far can be eliminated by a distance constraint even before the optimization. For example, if the service has a distance constraint of at most 100 kilometers, then only those <cloud owner, cloud region> within 100 kilometers to the customer location would be considered in the objective function evaluation.

If the service designer wants to trade off between distance and cost, for example, they might set wd = 1, wc = 2. This would mean that one $1 increase in price is as valuable as 2 kilometers in distance.

<cloud owner, cloud region> Candidate 1: $100, 100 kilometers => value: 300

<cloud owner, cloud region> Candidate 2: $150, 80 kilometers => value: 380

<cloud owner, cloud region> Candidate 3: $50, 190 kilometers => value: 290  <- pick this one

Step 6. OOF → SO - Return the target <cloud owner, cloud region> for the Service Instance

Step 7. SO → MC - Deploy VNF template in the target <cloud owner, cloud region> for the Service Instance

7a) MC Processing (need MC code changes)

  • Parse Template (e.g. OpenStack Heat Template)
    • For each VNFC, instance type in the template
      • Fetch Cloud-Agnostic Workload Deployment Policy (Intent) based on <Service (e.g. vCPE), VNFC (e.g. vGW)>
        • Value/Content: <Policy JSON> 
        • Note: The policy details are in Section 7b).
      • Parse Policy JSON
      • Modify template according to Intent 
        • Intent examples of interest for R3 
          • "Infrastructure High Availability (HA) for VNF" 
          • "Infrastructure Resource Isolation for VNF"   
            • "Burstable QoS"
          • "Infrastructure Resource Isolation for VNF"   
            • "Guaranteed QoS"

  • Policy (Intent) Realization

    • "Infrastructure High Availability (HA) for VNF"
      • OpenStack-based Cloud realization 
        • For R3, Host-based anti-affinity using server groups //Beyond R3, Support other anti-affinity models at availability zone level etc. 
        • Implementation Notes: 
          • Instance "count" in heat template specifies VNFC scale out factor 
          • While dynamic injection of server group into heat template is ideal, a simple starting point could be just switching to an alternate heat template which is identical to the deployment template and additionally has server group
      • Azure realization 
        • Availability Set?

    • "Infrastructure Resource Isolation for VNF" – { "qosProperty": { {"Burstable QoS": "TRUE", "Burstable QoS Oversubscription Percentage": "25"} } }

        • Example 
          • VNFC with "Guaranteed QoS" 
            • "flavor-xyz-no-oversubscription"
            • vCPU (Min/Max) - 16, Mem (Min/Max) - 32GB 
          • Same VNFC with "Burstable QoS", 25% over-subscription 
            • "flavor-xyz-25-percent-oversubscription"
            • vCPU (Min) - 16, Mem (Min) - 32GB 
            • vCPU (Max) - 20, Mem (Max) - 40GB  
        • Only certain pre-defined over-subscription values are allowed to simplify implementation
        • Implementation Notes:
          • While dynamic injection of limit/reservation into flavor is ideal, a simple starting would be to be to switch to a pre-defined flavor in the environment file
            • For aforementioned example
              • Original flavor - "flavor-xyz-no-oversubscription"
              • Modified flavor based on Policy - "flavor-xyz-25-percent-oversubscription" 
    • Implementation Notes:
      • From an implementation stand point, MC could be exposing a Workload Deployment Policy (Intent) API
        • Input : deployment-intent, cloud owner, cloud region, deployment template, deployment environment file, ...
        • Output : Success or Failure with reason, modified deployment template, modified deployment environment file, ...

7b) Cloud-Agnostic Workload Deployment Policy (Intent)

  • For R3, Cloud-Agnostic Workload Deployment Policy (Intent) can be stored in the form of configuration file(s) in the OOM K8S Persistent Volumes to simplify implementation.
    • For R4 and beyond, this policy (optionally with specific realization options) will be passed from SO → MC. This is captured in the R4 and beyond workflow (Edge Scoping - Beyond Casablanca).
  • This policy is exactly the same as the policies with "deployment-intent" in the Enhanced Capacity Check & Cloud Selection Policy for Homing described in Section 2.
  • An exemplary policy is depicted below.
Code Block
languagepy
themeEmacs
titleCloud-Agnostic Workload Deployment Policy (Step 7b)
linenumberstrue
collapsetrue
//Policy Example 1 - VNFC VGW which is part of vCPE service
{
	"Service": "vCPE"
	"VNFC": "vgw", //"vgw" is also intechangeably used as "vg"	
	
	{
		"cloudOwner": "VMware VIO", //can be a specific cloud owner such as Azure, VMware VIO, Wind River Titanium Cloud etc.
		"cloudRegion": "All", //can be a specific cloud region for a cloud owner
		"deployment-intent": 
		{
			"name": "Infrastructure Resource Isolation for VNF", 
				// realization possible without dedicating CPU and Memory, refer to section on "Cloud Resource Partitioning for Differentiated QoS" 
				// on how this can help in offering tiered services
			"qosProperty": 
			{
				{"Burstable QoS": "TRUE", "Burstable QoS Oversubscription Percentage": "25"}

			}
		}
	}
}

//Policy Example 2 - VNFC vDNS which is part of vLoadBalancer/vDNS service
{
	"Service": "vDNS",
	"VNFC": "vDNS", 
	
	//By default, Policy (Intent) is applicable to all cloud owners/regions unless specified.
	{		
		"cloudOwner": "All", //can be a specific cloud owner such as Azure, VMware VIO, Wind River Titanium Cloud etc. 
		"cloudRegion": "All", //can be a specific cloud region for a cloud owner
		"deployment-intent": 
		{
			"name": "Infrastructure High Availability (HA) for VNF", 
		}
	},
	
	{
		"cloudOwner": "All", //can be a specific cloud owner such as Azure, VMware VIO, Wind River Titanium Cloud etc. 
		"cloudRegion": "All", //can be a specific cloud region for a cloud owner
		"deployment-intent": 
		{
			"name": "Infrastructure Resource Isolation for VNF", 
				// realization possible without dedicating CPU and Memory, refer to section on "Cloud Resource Partitioning for Differentiated QoS" 
				// on how this can help in offering tiered services
			"qosProperty": 
			{		
				{"Guaranteed QoS": "TRUE"}
			}
		}
	}
}


Follow ups:


  • Policy DB – is there any restriction on json objects store? 
    • Matti to follow up with Ankit


Implementation trade offs for Casablanca (R3) and potential Dublin (R4) plan:


  • Deployment-Intent 
    • 1. "Infrastructure Resource Isolation for VNF" – { "qosProperty": { {"Burstable QoS": "TRUE", "Burstable QoS Oversubscription Percentage": "25"} } }
      • Casablanca Plan
        • Only certain pre-defined over-subscription values are allowed to reflect practical deployment and simplify implementation 
      • Dublin & Beyond Potential Plan
        • Creating instance types on demand for private clouds - to study
    • 2. Cloud-agnostic Workload Deployment Policy (Intent) is not passed in the SO → MC API
      • Casablanca Plan
        • Cloud-Agnostic Workload Deployment Policy (Intent) can be stored in the form of Multi-Cloud configuration file(s) in OOM K8S Persistent Volumes to simplify implementation.
      • Dublin & Beyond Potential Plan
        • Pass this policy (optionally with specific realization options) from SO → MC. This is captured in the R4 and beyond workflow (Edge Scoping - Beyond Casablanca). 
  • Policy-based capacity check & cloud-selection
    • 3. Tenant Information is not passed in the OOF → MC API
      • Casablanca Plan
        • The tenant information is derived from a simple mapping function per <cloud owner, cloud region>
          • A simple mapping would be a tenant per <cloud owner, cloud region> as part of Multi-VIM plugin configuration.
          • Need to make sure that this scheme is synchronous with the SO → MC API path
      • Dublin & Beyond Potential Plan
        • Pass Tenant Information per <cloud owner, cloud region> in the OOF → MC API
    • 4. Capacity Check for Public Clouds is not supported
      • Casablanca Plan
        • Public Clouds always pass the capacity check
      • Dublin & Beyond Potential Plan
        • Quota in Public Clouds - to study

    • 5.  The workload instance type model (Note 1 below) is not passed from OOF → MC. Rationale below.

      • Currently, the VNFC model supports only compute/memory/local storage. A comprehensive VNFC model needs to include HW generation (Intel Broadwell/Haswell, ARM etc.), HPA details (SR-IOV etc.) and more.  

      • Casablanca Plan

        • Since the VNFC model standardization is in progress, VNFC mapping to VM Instance Type (OpenStack Flavor, Azure M4 etc.) happens in the Multi-Cloud plugin through configuration. 

        • Note 1: An exception to the rule is a simple capacity check based on VM compute/memory/local storage which can be supported by the current instance type model.
      • Dublin & Beyond Potential Plan
        • Align with the modelling work and make progress


Cloud Resource Partitioning for Differentiated QoS (Combined with Previous)

...