See also: Hardware Platform Enablement In ONAP
capability_types:
tosca.capabilities.Compute:
derived_from: tosca.capabilities.Root
properties:
name:
type: string
required: false
num_cpus:
type: integer
required: false
constraints:
- greater_or_equal: 1
cpu_frequency:
type: scalar-unit.frequency
required: false
constraints:
- greater_or_equal: 0.1 GHz
disk_size:
type: scalar-unit.size
required: false
constraints:
- greater_or_equal: 0 MB
mem_size:
type: scalar-unit.size
required: false
constraints:
- greater_or_equal: 0 MB
tosca.capabilities.Container:
derived_from: tosca.capabilities.Compute
tosca.capabilities.Storage:
derived_from: tosca.capabilities.Root
properties:
name:
type: string
required: false
tosca.relationships.HostedOn:
derived_from: tosca.relationships.Root
valid_target_types: [ tosca.capabilities.Container ]
# tosca.relationships.HostedOn:
# - impacts declarative workflow
# - based on tosca.capabilities.Container
Critics of TOSCA Spec Normatives:
- tosca.capabilities.Container inherits from Compute!!!
- tosca.capabilities.Compute, property 'name' - we don't actually need it?
- tosca.capabilities.Storage is not similar to the tasca Compute - it only includes the 'name' property, does nor specify any storage quantifiers like size etc.
- in TOSCA Specs, Storage is not an infrastructure-level requirement; it is rather a kind of application-level (attachable, with an initial image), required by (attached to) other application nodes
- Tosca Compute capability is a mix of CPU+Mem. Don't we want to keep them apart for greater flexibility?
- TOSCA Specs does not have requirements for I/O
- TOSCA Specs normatives do not provide all information items required by the ETSI and IM
- tosca.capabilities.Compute and *.Storage bring with them unnecessary inheritance, coupling with relationship types, may affect the TOSCA declarative workflows
- Model the basic HPA requirements with TOSCA capability types. VDUs will have requirements of these capability types. These HPA requirements will never be satisfied within the VNF and Service models, the orchestrator will do that in run-time
- Abstain from using tosca.capability.Compute as it does not fully match the requirements
- Define special ONAP capability types, separate for each of these categories: CPU, Memory, Storage, I/O, networking. These capability types express the most basic hardware characteristics. They also should be simple, flat bags of strictly named properties, all properties of primitive data types with clear restrictions.
- Derive from the basic capability types an additional level of capabilities, with advanced (HPA) details. As detailed as they seem, these capabilities are still generic, with strict definitions of properties that are shared by the major hardware vendors. In addition to the strictly-typed properties, the HPA-level capabilities will also have a json-formatted property in order to allow for even greater customization flexibility.
- Allow for further customization of the HPA-level capabilities into vendor-specific capabilities. These vendor-specific customized capability types may extend their HPA-level generic base by adding new properties and providing new constraints for the existing properties. In addition to the refinement of the strictly defined properties, the vendor-specific capabilities may provide their own validation schema for the json-formatted "flexible" property.
####### Basic specifications of hadware capabilities ####
capability_types:
onap.capabilities.infrastructure.CPU:
derived_from: tosca.capabilities.Root
description: basic processor capabilities
properties:
num_cpus:
type: integer
required: false
constraints:
- greater_or_equal: 1
cpu_frequency:
type: scalar-unit.frequency
required: false
constraints:
- greater_or_equal: 0.1 GHz
onap.capabilities.infrastructure.Memory:
derived_from: tosca.capabilities.Root
description: basic memory capabilities
properties:
mem_size:
type: scalar-unit.size
required: false
constraints:
- greater_or_equal: 0 MB
onap.capabilities.infrastructure.Storage:
derived_from: tosca.capabilities.Root
description: basic storage specifications
properties:
storage_size:
type: scalar-unit.size
required: false
constraints:
- greater_or_equal: 0 MB
onap.capabilities.infrastructure.IO:
derived_from: tosca.capabilities.Root
description: basic IO specifications
onap.capabilities.infrastructure.NIC:
derived_from: tosca.capabilities.Root
description: basic networking interface characteristics
####### Advanced hardware capabilities, with more details ####
capability_types:
onap.capabilities.infrastructure.hpa.CPU:
derived_from: onap.capabilities.infrastructure.CPU
description: detailed processor capabilities for hardware-aware VNF vendors
properties:
schema_selector:
description: vendor+architecture, for example, Intel64
type: string
required: true
schema_version:
type: version
required: false
custom_features:
description: additional features, formatted as JSON, validated against a schema
type: json
constraints:
- schema: http://schema.url
required: false
simultaneousMultiThreading:
type: boolean
description: |
The use of Simultaneous Multi-Threading HW is an efficient way to
increase the compute capacity of a platform. SMT HW threads share
some CPU core resources. In some VDU implementations, it may be
necessary to very explicitly control the HW thread allocation on
a platform. This could be to help ensure locality in data caches
or as a mechanism to enhance determinism
required: false
logicalCpuPinningPolicy:
type: string
constraints:
- valid_values: [Dedicated, Shared]
required: false
logicalCpuThreadPinningPolicy:
type: string
constraints:
- valid_values:
- Isolate # Allocate on different execution units.
- Prefer # co-location of vCPUs to physical execution units
- Require # co-location of vCPUs to physical execution units
required: false
instructionSetExtensions:
type: string
constraints:
- valid_values:
- Isolate # Allocate on different execution units.
- Prefer # co-location of vCPUs to physical execution units
- Require # co-location of vCPUs to physical execution units
required: false
hypervisorConfiguration:
type: string
required: false
computeRas:
description: Reliability, Availability, Serviceability (RAS)
type: string
required: false
onap.capabilities.infrastructure.hpa.Memory:
derived_from: onap.capabilities.infrastructure.Memory
description: HPA-level memory capabilities
properties:
schema_selector:
description: vendor+architecture, for example, Intel64
type: string
required: true
schema_version:
type: version
required: false
custom_features:
description: additional features, formatted as JSON, validated against a schema
type: json
required: false
memoryPageSize:
type: scalar-unit.size
required: false
memoryAllocationPolicy:
type: string
constraints:
- valid_values:
- StrictLocal
- PreferredLocal
required: false
memoryBandwidth:
description: Agreed unit of memory bandwidth
type: scalar-unit.size
required: false
processorCacheAllocation:
description: Agreed unit of processor cache
type: string
required: false
memoryType:
description: Type of memory
type: string
required: false
memorySpeed:
description: Agreed unit of memory speed
type: string
required: false
memoryRas:
description:
type: string
required: false
localNumaMemory:
type: boolean
required: false
onap.capabilities.infrastructure.hpa.Storage:
derived_from: onap.capabilities.infrastructure.Storage
description: HPA-level storage specifications
properties:
schema_selector:
description: vendor+architecture, for example, Intel64
type: string
required: true
schema_version:
type: version
required: false
custom_features:
description: additional features, formatted as JSON, validated against a schema
type: json
required: false
storageIops:
type: integer
required: false
constraints:
- greater_or_equal: 0
storageResilencyMechanism:
type: string
required: false
description: Erasure code based back-end, triple replication
onap.capabilities.infrastructure.hpa.IO:
derived_from: onap.capabilities.infrastructure.IO
description: HPA-level IO characteristics
properties:
schema_selector:
description: vendor+architecture, for example, Intel64
type: string
required: true
schema_version:
type: version
required: false
custom_features:
description: additional features, formatted as JSON, validated against a schema
type: json
required: false
pciVendorId:
description: PCI-SIG vendor ID for the device
type: string
required: false
pciDeviceId:
description: PCI-SIG device ID for the device
type: string
required: false
pciNumDevices:
description: Number of PCI devices required.
type: string
required: false
pciAddress:
description: Geographic location of the PCI device via the standard PCI-SIG addressing model of Domain:Bus:device:function
type: string
required: false
pciDeviceLocalToNumaNode Boolean
type: string
required: false
onap.capabilities.infrastructure.hpa.NIC:
derived_from: onap.capabilities.infrastructure.NIC
description: HPA-level networking interface characteristics
properties:
schema_selector:
description: vendor+architecture, for example, Intel64
type: string
required: true
schema_version:
type: version
required: false
custom_features:
description: additional features, formatted as JSON, validated against a schema
type: json
required: false
nicFeature:
description: Long list of NIC related items such as LSO, LRO, RSS, RDMA, etc.
type: map
entry_schema: string
required: false
dataProcessingAccelerationLibray:
description: Name and version of the data processing acceleration library required. Orchestration can match any NIC that is known to be compatible with the specified library.
type: string
required: false
interfaceType:
description: Virtio, PCI-Passthrough, SR-IOV, E1000, RTL8139, PCNET, etc.
type: string
required: false
capability_types:
com.intel.capabilities.hpa.CPU:
derived_from: onap.capabilities.infrastructure.hpa.CPU
schema_selector:
type: string
required: true
constraints: # fixed value for this vendor
- equal_to: Intel64
schema_version:
type: version
required: false
constraints:
- equal_to: 2.0
instructionSetExtensions:
type: string
constraints: # changes property definitions
- valid_values: [aes, sse, avx, cat, cmt, mbm]
required: false
a_well_known_property_of_this_vendor: # adds new strictly typed properties
type: integer
required: false
node_types:
onap.nodes.VDU:
derived_from: onap.nodes.Resource
capabilities:
host:
type: onap.capabilities.Container
requirements:
- cpu:
capability: onap.capabilities.infrastructure.CPU:
occurrences: [0..UNBOUNDED]
- memory:
capability: onap.capabilities.infrastructure.Memory:
occurrences: [0..UNBOUNDED]
- storage:
capability: onap.capabilities.infrastructure.Storage:
occurrences: [0..UNBOUNDED]
- io:
capability: onap.capabilities.infrastructure.IO:
occurrences: [0..UNBOUNDED]
- nic:
capability: onap.capabilities.infrastructure.NIC:
occurrences: [0..UNBOUNDED]
node_templates:
vdu_123:
type: onap.nodes.VDU
capabilities:
host:
requirements:
- memory:
node_filter:
capabilities:
- onap.capabilities.infrastructure.Memory:
properties:
- mem_size: {greater_or_equal: 2MB}
- cpu:
node_filter:
capabilities:
- onap.capabilities.infrastructure.hpa.CPU:
properties:
- schema_selector:
constraints: # fixed value for this vendor
- equal_to: Intel64
- schema_version:
constraints:
- greater_or_equal: 2.0
- custom_features:
constraints:
- equals_to: {,,.sigjigigirgigmirig}
- schema: http://json.schema.url
- Compliance with TOSCA 1.2 for using JSON properties
- Support for the 'node_filter' construct
- New languages to understand: JSON Schema, XML Schema, etc
24 Comments
Srinivasa Addepalli
This is pretty good.
One comments:
Thanks
Srini
Anatoly Katzman
I am now thinking of a way to tell whether a requirement is mandatory or not (not yet done with it). Do you think that a binary switch would not be enough, and that we may need a more advanced scoring scale?
Srinivasa Addepalli
Yes. I saw that you have one attribute called 'required'. If VNFC indicates that feature is optional, it is good to have score. OOF can add up scores of all optional features across various flavors and whichever flavor provides highest score, OOF will choose that flavor . Yes, I think that more scoring would help quite a bit. K8S has this functionality and it was very helpful and hence looking for similar capability in ONAP VNFC model too.
Thanks
Srini
Anatoly Katzman
Interesting. I will take this into account.
Srinivasa Addepalli
Thanks Anatoly
Anatoly Katzman
By the way, that 'required' attribute is not what you think it is. It is just a TOSCA way to say that a template without this property assigned is a bad syntax.
The score information you are asking for has to do with run-time lookup for the best infrastructure match..
Alexander Vul
Hi Srini,
AFAIC, there is no need for a score. From the VNF developer perspective, a certain hardware capability is either required or not. There is no "may be" in a form of a score. Just "yes" or "no". We had meetings with lots of VNF suppliers and we never heard any requirements to have a score. I would suggest that we keep things simple for now. If there is market evidence in the future that a score is required we can add something in to support it.
Observe, that operators can still affect all of this by creating operational policies that interpret the "strength" of requirements based based on additional factors, such as who the tenant is, or what service plan is in effect, or the relative importance of a given requirement, etc...
Alex
Srinivasa Addepalli
Hi Alex,
Score provides flexibility. If an operator or VNF vendor believe that all features are required, then he/she can make every feature specified in the template as mandatory.
Thanks
Srini
maopeng zhang
Hi Anatoly
using node_filter method seperately in CPU and memory points may meet some problems. We just want to filter nodes satisfy both memory and cpu requests, not separately.
please consider HPA feature NUMA, I have comment in the "Focus on VDU" page.
Thanks
Anatoly Katzman
I have just replied to your comment in Splitting VDU: VFC + Container, let's discuss it there.
Srinivasa Addepalli
I am trying to understand on how VDU looks like for this following scenario. Please validate the understanding ( capability definition needs to be updated to have need and score)
Say, I want to bring up VNFC with following
node_templates:
vdu_123:
type: onap.nodes.VDU
capabilities:
host:
requirements:
- vcpus:
node_filter:
capabilities:
- onap.capabilities.infrastructure.CPU
properties
- need : required
- num_cpus : {equal : 2}
- cpufreq:
node_filter:
capabilities:
- onap.capabilities.infrastructure.CPU
properties
- need : preferred
- score : 20
- cpu_frequency : {greater_or_equal : 2GHZ}
- memory:
node_filter:
capabilities:
- onap.capabilities.infrastructure.Memory:
properties:
- need : required
- mem_size: {greater_or_equal: 4GB}
- cpuNuma:
node_filter:
capabilities:
- onap.capabilities.infrastructure.hpa.CPU:
properties:
- schema_selector: {equal : Intel64}
- need: preferred
- score : 10
- numa_node_to_cpu_map: {0:1, 1:1} #Numa Node 0 to have 1 core and Numa Node 1 to have 1 core
- pageSize:
node_filter:
capabilities:
- onap.capabilities.infrastructure.hpa.memory:
properties:
- schema_selector: {equal : Intel64}
- need: required
- memoryPageSize : {equal : 2MB }
- memoryNuma:
node_filter:
capabilities:
- onap.capabilities.infrastructure.hpa.memory:
properties:
- schema_selector: {equal : Intel64}
- need: preferred
- score : 10
- numa_node_to_memory_map: {0:1GB, 1:3GB} #Numa Node 0 to have 1GB and Numa Node 1 to have 3GB
- lookAsideCrypto:
node_filter:
capabilities:
- onap.capabilities.infrastructure.hpa.IO:
properties:
- schema_selector: {equal : Intel64}
- need: required
- pciVendorId : XYZ
- pciDeviceId : ABC
- pciNumDevices: 1
- ovsDPDK:
node_filter:
capabilities:
- onap.capabilities.infrastructure.hpa.NIC:
properties:
- schema_selector: {equal : Intel64}
- need: required
- vSwitchAcceleration : {equal : ovsdpdk}
- avx2:
node_filter:
capabilities:
- com.intel.capabilities.hpa.CPU:
properties:
- schema_selector: {equal : Intel64}
- need: required
- instructionSetExtensions : {pattern : avx2}
- aesNi:
node_filter:
capabilities:
- com.intel.capabilities.hpa.CPU:
properties:
- schema_selector: {equal : Intel64}
- need: required
- instructionSetExtensions : {pattern : aes}
- cmt: #Cache Monitoring
node_filter:
capabilities:
- com.intel.capabilities.hpa.CPU:
properties:
- schema_selector: {equal : Intel64}
- need: preferred
- score : 5
- instructionSetExtensions : {pattern : cmt}
- mbm: # Memory Bandwidth Monitoring
node_filter:
capabilities:
- com.intel.capabilities.hpa.CPU:
properties:
- schema_selector: {equal : Intel64}
- need: preferred
- score: 5
- instructionSetExtensions : {pattern : mbm}
- cacheAllocation: # Memory Bandwidth Monitoring
node_filter:
capabilities:
- com.intel.capabilities.hpa.CPU:
properties:
- schema_selector: {equal : Intel64}
- need: required
- l3_cache : {greater_or_equal : 2MB}
Anatoly Katzman
Ouch, that was one long remark...
I would think of something like this:
Few points to highlight:
What do you think?
Srinivasa Addepalli
Reason, I had separated various features into different buckets in the template is to provide flexibility of choosing the need_level (required at run time vs Preferred at run time) and score. Based on the way capabilities are defined, is that acceptable way of template definition?
Anatoly Katzman
Srinivasa Addepalli
>>>More than one requirements under the same name, and the position does matter. The first is mandatory, all next are less and less important (credits to shitao li for the idea)
I guess you mean if there is any duplicate, right? That is, if memoryPageSize property is used multiple times, then the 1st one is important and others are lesser important. That seems okay to me. But, if there are different properties say 'memoryPageSize' and 'numa_node_to_memory_map', then having 'need' and 'score' for each property help as this is deterministic way of representing the desire.
Anatoly Katzman
What you want is to have the "isMandatory" and "score" qualifiers per property in the requirement assignments. It's a good idea, but implementing it in TOSCA is a pain. I have considered few possible solutions, and all came out either over-complicated, or too verbose (yes, even more verbose than mine), or, worse, non-standard. The multi-layer requirement solution proposed here is a sensible compromise,
Srinivasa Addepalli
Hi Anatoly,
I just started to learn the capabilities of TOSCA. So, please use this input as food for thought.
It is good if we could satisfy following requirements:
In K8S, following example describes (borrowed from (https://github.com/davidkbainbridge/demo-affinity/blob/ffa03468935acfbc40bc11d13dd47506084aae61/ansible/roles/test/files/vnf/must_have_accel_prefers_perst.yml.j2)
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: hw.accel
operator: In
values:
- "true"
preferredDuringSchedulingIgnoredDuringExecution:
- weight: {{WEIGHT}}
preference:
matchExpressions:
- key: hw.perst
operator: In
values:
- "true"
Here, there are two arbitrary HPA features are used hw.accel and hw.perst. These two are feature labels understood to be normalized feature names. K8S grammar seems to satisfy our requirements. But, TOSCA has some different grammar. in TOSCA, each parameter is expected to be defined with the type before that parameter is used in templates.
Based on my understanding of HPA features, there are few categories of HPA features:
Keeping above in mind and TOSCA grammar restriction in mind, I have defined following. Again, as I said, I am a TOSCA beginner. So, use this information as food for thought.
maopeng zhang
Allow me to join the discussion.
1. how to check the value in value provided.
2. TOSCA property definition is also a standard to define the same thing you want to do.
You can regard the property whole as KVP and each property is a some kind of key.
I see now the parameters you provided are all supported by the TOSCA properties, such as name,value, greater than,....
please see:
http://docs.oasis-open.org/tosca/TOSCA-Simple-Profile-YAML/v1.2/csprd01/TOSCA-Simple-Profile-YAML-v1.2-csprd01.html 3.6.9 Property definition
If do so, It will be a new wheel or grammar. Do not need to create a new wheel in my mind.
It is also not simple than the TOSCA properties definition,
If wrong, please correct me.
Srinivasa Addepalli
Hi Maopeng
Sorry that I am repeating myself here.
"
These are HPA representation requirements:
"
That is why, we went with defining feature and attributes themselves as values to TOSCA properties. One level of indirection would help in achieving above requirements.
Srini
Srinivasa Addepalli
What is the purpose of schema_selector and schema_version?
Anatoly Katzman
The names and the idea of these properties are all taken from Key-Value Pair Registries.docx (the HPA information model proposed by Intel). schema_selector = "architecture", for example "Intel64". Using only this property, a model can express a wish for a hardware architecture in general, without getting into too many details. When combined with the "detail" properties, the schema_selector serves as an additional qualifier.
Srinivasa Addepalli
As you know, there is a discovery phase of HPA "feature sets" supported by underlying VIMs/Clouds. In case of Openstack based VIMs, they have concept called flavors where each flavor represents set of HPA features and values the infrastructure supports. In case of Azure clouds, they have concept called "VM Sizes" and AWS CLoud has concept called "Instance types". Multi-Cloud service is expected to discover these sets and HPA features of each set and populate in AAI database. As part of populating them, the idea is to normalize the HPA features in each set to some agreed names. Idea is that these normalized names are the ones which are also used to identify the feature requirementsin VNFC template. Due to this normalization, it becomes easy for OOF (which is expected to choose right feature set record as part of VM instantiation) to compare feature requirements of VNFC with feature-sets available in AAF. I am wondering whether the schema_selector is supposed to be represented with the feature name? If that is not the intent, which part of the VNFC template is supposed to represent the actual uniform feature name? I hope I am clear. Let me know, if I am not and I will try to give an example.
Anatoly Katzman
No, schema_selector is not the same as a flavor name. This and other property names are taken from this Intel document, and I assumed that these names were agreed and uniform across the hardware vendors. If this is not the case - well, we can always rename them into something more standard..
maopeng zhang
Hi Anatoly
The VDU defined here is different from the Quick Start Page. Which one is the final definition? thanks