Page History

...

This epic (

Flow Diagram for HPA for K8s

Image Added

Discovery and Labeling of Node Features

Discovery of HPA features will be done using the node feature discovery [NFD] addon (https://github.com/kubernetes-sigs/node-feature-discovery) for kubernetes.  NFD will be deployed as a daemonSet, with pods on each node and will discover and label hardware features on each kubernetes node with features labels (https://github.com/kubernetes-sigs/node-feature-discovery#feature-labels).

Registering of K8s Clusters

Registration of K8s Cluster will happen at the ESR UI. ESR will then send a POST request to 

http://{{MSB_IP}}:{{MSB_PORT}}/api/multicloud-k8s/v1/{{cloud-owner}}/{{cloud-region-id}}/registry

Multicloud K8s Plugin registrationHandler will receive said POST and register the new Kubernetes cluster with the K8s Plugin using the connectivity code/API.

The registrationHandler will then query the k8s cluster under registration to get a list of nodes and the labels for each node.

Reporting of Cluster Info to AA&I

Reporting of HPA features to AA&I

...

is done by the registrationHandler.

...

Data will be

...

munged to cloud-region/tenant/flavors schema for

...

that cluster and reported to AA&I. registrationHandler is planning on using AA&I module under development for Frankfurt to report to AA&I. In this case we will send a request to the AA&I module instructing it to update the cloud-region, create a tenant and create flavors that match the node feature labels in said cloud-region.

Update of CNF Artifacts to Include nodeSelector

CNF and VNF helm charts will be updated with nodeSelector requirements & recommendations in line with node labels created by NFD during feature discovery.
Example Node Features: 

# Hardware Feature Labels:
feature.node.kubernetes.io/cpu-cpuid.AESNI=true,
feature.node.kubernetes.io/cpu-cpuid.AVX2=true,
feature.node.kubernetes.io/cpu-cpuid.AVX=true,
feature.node.kubernetes.io/cpu-cpuid.FMA3=true,
feature.node.kubernetes.io/cpu-cpuid.IBPB=true,
feature.node.kubernetes.io/cpu-cpuid.STIBP=true,
feature.node.kubernetes.io/cpu-hardware_multithreading=true,
feature.node.kubernetes.io/cpu-pstate.turbo=true,
feature.node.kubernetes.io/cpu-rdt.RDTCMT=true,
feature.node.kubernetes.io/cpu-rdt.RDTMON=true,
feature.node.kubernetes.io/memory-numa=true,
feature.node.kubernetes.io/network-sriov.capable=true,
feature.node.kubernetes.io/pci-0300_102b.present=true,
feature.node.kubernetes.io/storage-nonrotationaldisk=true

# Software Feature Labels:
feature.node.kubernetes.io/kernel-config.NO_HZ=true,
feature.node.kubernetes.io/kernel-config.NO_HZ_FULL=true,
feature.node.kubernetes.io/kernel-version.full=3.10.0-957.el7.x86_64,
feature.node.kubernetes.io/kernel-version.major=3,
feature.node.kubernetes.io/kernel-version.minor=10,
feature.node.kubernetes.io/kernel-version.revision=0,
feature.node.kubernetes.io/system-os_release.ID=centos,
feature.node.kubernetes.io/system-os_release.VERSION_ID.major=7,
feature.node.kubernetes.io/system-os_release.VERSION_ID.minor=,
feature.node.kubernetes.io/system-os_release.VERSION_ID=7

Instantiation of K8s HPA

Policy and OOF can then will use the same path to discover capabilities of K8s nodes and use that as are done for OpenStack since we are using the same data model from AAI. OOF will use AA&I information to home particular CNF workloads to nodes which can accelerate their function. Further Multicloud will use labels when instantiating CNF workload to ensure kubernetes places the workloads on the correct nodecloud-regions which contain requested features. When K8s plugin instantiates said workloads Kubernetes will read nodeSelector preferences and place each workload on nodes which contain needed features to accelerate their function.

High Level Task Overview

• Ensure NFD (Node feature discovery - K8S CNCF project) deploy to Multicloud + K8s Plugin is working and labeling Kubernetes nodes
• Add registrationHandler to K8s Plugin and expand K8s Plugin API to work with ESR VIM registration of K8s Clusters
  • Use POST/DELETE/GET of http://{{MSB_IP}}:{{MSB_PORT}}/api/multicloud-k8s/v1/{{cloud-owner}}/{{cloud-region-id}}/registry 
  • Ensure registrationHandler can get K8s feature labels - (labels discovered and placed by NFD representing hardware features)
  • Ensure registrationHandler can populate A&AI cloud-region/tenant/compute flavors based on cluster under registration
  • Register K8s cluster internally with the K8s Plugin
• Add Node App in K8s Plugin to retrieve node labels from a cluster
• Identify and Address any gaps in policy and OOF to do match making of CNFs/VNFs using K8s Plugin

...