Background
ONAP is a set of projects. It is unlikely that ONAP will be deployed in its entirety by customers. It is envisaged that there could be multiple profiles, with each profile satisfying a set of deployments. A given ONAP profile may not require all projects and also it may not require all Microservices in the chosen projects.
In this document, we are defining one profile - ONAP4K8S.
Why ONAP4K8S?
Akraino ICN BP family (https://wiki.akraino.org/pages/viewpage.action?pageId=11995140) has chosen ONAP for orchestrating workloads across multiple edge locations. ICN BP addresses the edge locations with K8S resource orchestrator, that is ICN BP is not used in cases where Openstack or non-K8S based resource orchestrators are used. Keeping this in mind, ICN is requesting bare-minimum ONAP that is required to deploy workloads in K8S regions.
Many Enterprises are adopting K8S to deploy workloads in their local data centers/edge locations. Many feel that ONAP asis can be a challenging to install and maintain. Also concerned about security challenges associated with the code that is unused. The second challenge is the amount of memory and CPU power it requires to run the entire ONAP. Since, many components of ONAP are not required, it is felt that a profile of ONAP would be good. Hence ONAP4K8S.
ONAP4K8S requirements
Based on ICN requirements and talking to few Enterprise customers, following are the requirements for ONAP4K8S
- ONAP4K8S to contain Microservies that are required for K8S based workload deployments.
- ONAP4K8S shall not have Microservices that are not used.
- ONAP4K8S shall provide a way to onboard resource bundles, applications consisting of multiple resource bundles.
- ONAP4K8S shall use cloud native open source projects for infrastructure
- fluentD for logging.
- Jaeger for tracing
- ISTIO/Envoy for service mesh
- ONAP4K8S shall maintain security of passwords and private keys.
- ONAP4K8S shall provide 'Role Based Access Control' for all operations.
- ONAP4K8S shall scale-out.
- ONAP4K8S shall use distributed databases
- ONAP4K8S should have simple UI to onboard, instantiate, terminate and provide Day2 configuration
ONAP4K8S package
Keeping the above requirements in mind, current thinking is to create ONAP4K8S package with the following containers
From ONAP:
- Multi-Cloud K8S Service
- CSM MicroServices
- SMS
- SMS Microservice
- SMS Quorumclient Service
- Vault Service
- Consul Service
- CA key distribution services dist-center service
- TPM Services - abrmd Service and testca service AI - Kiran Kamineni, Pramod Raghavendra Jayathirth and Manjunath Ranganathaiah
- SMS
- UI Service (to be developed).
- Multi-Cluster scheduler micro-service (to be developed)
- ....
From CNCF and other open source projects:
- Vault
- Vault is the only binary that runs in there.
- ConsulD Cluster
- Consul Server and N number of Consul Agents
- MongoDB
- Mongo is the only binary that runs in the POD.
- etcD
- List down Microservices Ritu Sood
- ISTIO
- istio-citadel
- istio-egressgateway
- istio-galley
- istio-ingressgateway
- istio-pilot
- istio-policy
- istio-sidecar-injector
- istio-telemetry
- Logging (Fluentd, Elastic Search and Kibana)
- List down Microservices Kiran Kamineni
- Jaeger
- jaeger-agent
- jaeger-collector
- jaeger-query
ONAP4K8S's Projects and Repositories
| Micro-Service/Container (Project) | Source code repository and Directories | Comments |
|---|---|---|
1.SMS Microservice (SMS) | https://github.com/onap/aaf-sms | |
| 2. Vaut/db (SMS) | https://github.com/onap/aaf-sms | |
| 3. Abrmd (SSHSM) | https://gerrit.onap.org/r/gitweb?p=aaf/sshsm.git;a=tree | |
4. dist-center(SSHSM) | https://gerrit.onap.org/r/gitweb?p=aaf/sshsm.git;a=tree | |
| 5. testca (SSHSM) | https://gerrit.onap.org/r/gitweb?p=aaf/sshsm.git;a=tree | |
| 6. etcd (Multicloud/k8s Plugin) | https://github.com/onap/multicloud-k8s | |
7. Multicloud/k8s Plugin service (Multicloud/k8s Plugin) 8. Mongo (Multicloud/k8s Plugin) |
Deliverables:
- Create ONAP4K8S helm chart (Figure out how OOM can be used to do this. If not, may need to go with its own set of helm charts)
- Ensure that ONAP4K8S package can be used to instantiate vFW and EdgeX applications.
12 Comments
Srinivasa Addepalli
ramki krishnan
Please provide your input
Srini
ramki krishnan
This is an extremely valuable effort especially from Edge perspective.
Thanks,
Ramki
Prakash R.
I reviewed and trying to understand how this fits with scheme for OOM for platform extension (per site/edge) vs workload PODs/cluster's infrastructure at the Edge. May need more effort to comment yet. Certainly a god beginning.
Srinivasa Addepalli
Hi Prakash,
This page is defining subset of ONAP that is required to deploy workloads in multiple Edge or cloud K8S clusters.
Prakash R.
Following up yes ONAP4K8S is part of a Region defined as per Akraino ICN Blue Print. Refer Deliverables 4 above.
The Casablanca OOM shows OOM does installs part of the containerized modules
Again with plans for Prometheus for service assurance and Health check modules, makes me think this is getting way complex than a simple deployment with k8s controller. Can you consider using current CRD with kubebuilder instead of helm operator. Will need better understanding what option will work better and simplify in long term.
Sylvain Desbureaux
Hello,
Although I'm on favour of using as many CNCF projects as possible, I'm a bit puzzled on this "ONAP" profile.
Only 2 components (actually 2 subcomponents) of ONAP are used and the core of ONAP (which is SO/A&AI/SDC) is not present.
Stephen Terrill , Catherine Lefevre , Eric Debeau , is this an "official" ONAP profile?
Srinivasa Addepalli , on a more technical part, what's is the value add of SMS and multicloud-k8s?
I'm not an expert of the first so I cannot pronounce but it seems that helm may be sufficient compared to multicloud-k8s no?
Eric Debeau
It is a proposal from MultiCloud project. This is not an official ONAP Profile agreed by TSC. We have not defined "ONAP Profile".
Srinivasa Addepalli
Currently, this profile is limited to what Akraino ICN project requires.
Yes, today, it has only MC components and security components from ONAP in this profile.
In future, we see some part of SDC to be present as it is required to onboard NFs, create services, approve services kind of workflow.
On the profiles in general: I believe companies are not taking ONAP in its entirety. I think people are creating their own ONAP profiles for their own purposes. This profile is also for similar purpose. In this case the purpose is Akraino ICN.
SMS: SMS intends to be used to store kubeconfig data as it has secret information such as private keys, tokens and passwords to access remote K8S cluster API servers.
Viswanath Kumar Skand Priya
Srinivasa Addepalli - If the intent is to bring up part of ONAP ( not entire ONAP ) for a given environment, couldn't we create an override profile in OOM and use it to deploy ONAP ?
Or is it a fork of ONAP for edge world?
Srinivasa Addepalli
We initially tried this and it required too many changes and we felt that those changes don't look so generic. We also felt that it would be a maintenance challenge in the OOM. Moreover, we are understanding that the OOM helm charts will be moving to individual projects and hence don't want to invest time on making changes, just to find that these changes are not valid any more in future. Hence, we had parallel helm chart for ONAP4K8S.
Viswanath Kumar Skand Priya
Another generic question I have is - why we intend to always equate containers & K8S for edge world? Today we are deploying containerized core functions such as mme, epc etc in our core datacenter. We will end up creating a service that spans across xNFs. What is the solution for those requirements ? Should we have to treat the container orchestration in both core & edge separate except for the fact that they both are different VIM regions?
Srinivasa Addepalli
No, The title of the feature is "K8S based cloud region support" and "ONAP4K8S". Whatever we do is nothing specific to Edge. It would work with any K8S environment whether it is in the Edge or in the public cloud or core networks.
But, Edge bring up more requirements such as
So, if you make your solution work for Edge/K8S, it will work with Clouds-K8S (EKS, AKS, AKE) for sure.
Srini