Kubernetes based Cloud-region support
Code Name: K8S
This page tracks the related materials/discussions/etc related to cloud regions that are controlled by Kubernetes.
As starting point, this effort has started as small subgroup of multicloud as task force.As the efforts evolve, logistics would be revised. Maybe this task force would be promoted to a independent group or an independent project.
Meetings
- [k8s] Team ONAP5, China(Wed)2:00am / UTC(Wed) 18:00PM / ET(Tue) 13:00pm / PT(Tue) 10:00am from Jan 16/17, 2018. weekly meeting
Slides/links
- Slides presented to architecture subcommittee to start this project as PoC :
Project template for project proposal
Project Name:
Project name: K8S based cloud-region support
code name: K8S
Repository name: multicloud/k8s
Background:
As of ONAP R2, ONAP can only orchestrate VNFs across cloud regions that have Openstack and its variations as VIM. Supported Multi-Cloud plugins include VMWare VIO, Windriver Titanium, Openstack Newton and Openstack Ocata. VMWare VIO and Windriver Titanium are vendor supported Openstack projects with additional features.
As of ONAP R2, ONAP can only deploy VM workload types.
Need for containers are felt for following reasons
- Resource constrained Edges : Power and space constraints limit number of compute nodes that can be deployed in edges.
- Ubiquity of container support : Many cloud providers are now supporting containerized workload using Kubernetes.
- Micro Service Architecture based solutions: Containers are considered to be ideal solution to implement micro-services.
Kubernetes is one of the popular container orchestrators. K8S is supported by GCE, AZURE and AWS and will be supported by Akraino Edge stack that enable edge clouds.
K8S is also being enhanced to support VM workload types too. This helps cloud-regions that need to support VMs while moving some workloads to containers. For security reason, cloud-regions may continue to support VM types for security reasons and hence there is need for VIM that support both VMs and containers.
Since same K8S instance can orchestrate both VM and container workload types, same compute nodes can be leveraged for both VMs and containers.
Telco and CSPs are seeing similar need to deploy networking applications as containers
There are few differences between containers for Enterprise applications and networking applications. Networking applications are of three types - Management applications, Control plane applications and data plane applications. Management and control plane applications are similar to Enterprise applications, but data plane applications different in following aspects:
- Multiple virtual network interfaces
- Multiple IP addresses
- SRIOV networking support
- Programmable virtual switch (for service function chaining, to tap the traffic for visibility etc..)
Since, both VMs and container workloads are used for networking applications, there would be need for
- Sharing the networks across VMs and containers.
- Sharing the volumes across VMs and containers.
Project Description:
The effort will investigate/drive a way to allow ONAP to deploy/manage Network Services/VNFs over cloud regions that support K8S orchestrator.
Increasing VNF density on each node and latency requirements are driving container based VNFs. This project enhances ONAP to support VNFs as containers in addition to VNFs as VMs.
- Support for multiple container orchestration technologies as cloud infrastructure - allow VNFs to run within container technology and also allow closed feedback loop same to VM based VIM. e.g. openstack
- Support for co-existence of VNF VMs and VNF containers - Add container orchestration technology in addition to the traditional VM-based VIM or environment managed by ONAP. So that VNFs can be deployed within container by ONAP.
- Support for uniform network connectivity among VMs and containers.
Goal and scope
the first target of container/COE is k8s. but other container/COE technology, e.g. docker swarm, is not precluded. If volunteers steps up for it, it would be also addressed.
- Have ONAP take advantage of container/COE technology for cloud native era
- Utilizing of industry momentum/direction for container/COE
Influence/feedback the related technologies(e.g. TOSCA, container/COE)
Teach ONAP container/COE instead of openstack so that VNFs can be deployed/run over container/COE in cloud native way
At the same time it's important to keep ONAP working, not break them.
- Don’t change the existing components/work flow with (mostly) zero impact.
- Leverage the existing interfaces and the integration points where possible
Functionality
- Allow VNFs within container through container orchestration managed by ONAP same to VM based VIM.
- Allow closed loop feedback and policy
- Allow container based VNFs to be designed
- Allow container based VNFs be configured and monitored.
- Kubernetes cloud infrastructure as initial container orchestration technology under Multi-Cloud project.
API/Interfaces
the following table summarizes the impact on other projects
component | comment |
modelling | New names of Data model to describe k8s node/COE instead of compute/openstack. Already modeling for k8s is being discussed. |
OOF | New policy to use COE, to run VNF in container |
A&AI/ESR | Schema extensions to represent k8s data. (kay value pairs) |
Multicloud | New plugin for COE/k8s. (depending on the community discussion, ARIA and helm support needs to be considered. But this is contained within multicloud project.) |
First target for first release
the scope of Beijing is
Scope for Beijing
First baby step to support containers in a Kubernetes cluster via a Multicloud SBI / Plugin
Minimal implementation with zero impact on MVP of Multicloud Beijing work
Use Cases
Sample VNFs(vFW and vDNS)
integration scenario
Register/unregister k8s cluster instance which is already deployed. (dynamic deployment of k8s is out of scope)
onboard VNFD/NSD to use container
Instantiate / de-instantiate containerized VNFs through K8S Plugin in K8S cluster
Vnf configuration with sample VNFs(vFW, vDNS)
Target for later release
- Installer/test/integration
- More container orchestration technology
- More than sample VNFs
- delegating functionalities to CoE/K8S
Non-Goal/out of scope
The followings are not goal or out-of-scope of this proposal.
- Not installer/deployment. ONAP running over container
- OOM project ONAP on kubernetes
- https://wiki.onap.org/pages/viewpage.action?pageId=3247305
- https://wiki.onap.org/display/DW/ONAP+Operations+Manager+Project
- Self hosting/management might be possible. But it would be further phase.
- container/COE deployment
- On-demand Installing container/coe on public cloud/VMs/baremetal as cloud deployment
- This is also out of scope for now.
- For ease of use/deployment, this will be further phase.
Architecture Alignment.
How does this project fit into the rest of the ONAP Architecture?
The architecture (will be)is designed to enhancement to some existing project.
It doesn’t introduce new dependency
How does this align with external standards/specifications?
Convert TOSCA model to each container northbound APIs in some ONAP component. To be discussed.
Are there dependencies with other open source projects?
Kubernetes pod API or other container northbound AP
CNCF(Cloud Native Computing Foundation), OCI(Open Container Initiative), CNI(Container Networking interface)
UseCases
- sample VNF(vFW and vDNS): In Beijing only deploying those VNF over CoE
- other potential usecases(vCPE) are addressed after Beijing release.
the work flow to register k8s instance is depicted as follows
the work flow to deploy VNF into pod is as follows
Other Information:
link to seed code (if applicable) N/A
Vendor Neutral
if the proposal is coming from an existing proprietary codebase, have you ensured that all proprietary trademarks, logos, product names, etc., have been removed?
Meets Board policy (including IPR)
Use the above information to create a key project facts section on your project page
Key Project Facts:
This project will be subproject of Multicloud project. Isaku will lead this effort under the umbrella of multicloud project.
NOTE: if this effort is sub-project of multicloud as ARC committee recommended, this will be same to multicloud's.
Facts | Info |
---|---|
PTL (first and last name) | same to multicloud project |
Jira Project Name | same to multicloud project |
Jira Key | same to multicloud project |
Project ID | same to multicloud project |
Link to Wiki Space | Support for K8S (Kubernetes) based Cloud regions |
Release Components Name:
Note: refer to existing project for details on how to fill out this table
Components Name | Components Repository name | Maven Group ID | Components Description |
---|---|---|---|
container | multicloud/container | org.onap.multicloud.container | container orchestration engine cloud infrastructure |
Resources committed to the Release:
Note 1: No more than 5 committers per project. Balance the committers list and avoid members representing only one company.
Note 2: It is critical to complete all the information requested, that will help to fast forward the onboarding process.
Role | First Name Last Name | Linux Foundation ID | Email Address | Location |
---|---|---|---|---|
committer | Isaku Yamahata | yamahata | isaku.yamahata@gmail.com | PT(pacific time zone) |
contributors | Munish Agarwal | Munish.Agarwal@ericsson.com | ||
Bin Hu | bh526r | bh526r@att.com | ||
Manjeet S. Bhatia | manjeets | |||
Phuoc Hoang | hoangphuocbk | phuoc.hc@dcn.ssu.ac.kr | ||
Mohamed ElSerngawy | melserngawy | mohamed.elserngawy@kontron.com | EST | |
Interested (will attend my first on 20180206) - part of oom and logging projects | michaelobrien | frank.obrien@amdocs.com | EST (GMT-5) |