The official documentation for installation of ONAP with OOM / Kubernetes is located in readthedocs here (TBD). The supported versions of Kubernetes is as follows:
Software Requirements
| Release | Kubernetes | Helm | Kubectl | Docker |
|---|---|---|---|---|
| amsterdam | 1.7.x | 2.3.x | 1.7.x | 1.12.x |
| beijing/master | 1.8.5- OOM-813 - Getting issue details... STATUS | 2.6.x | 1.8.5- | 20180321: use 1.12 for now |
Hardware Requirements
| Release | RAM | HD | vCores | Ports | |
|---|---|---|---|---|---|
| any | 63G at startup 70G after 48h Recommend 128G class VM
| 75G at startup 100G after a week Recommend 120-160G HD | 4 min 60 peak at startup recommended 16-32 vCores  | see list on PortProfile Recommend 0.0.0.0/0 (all open) |
Quickstart
Beijing ONAP on Azure ARM template to full OOM install - see ONAP on Kubernetes on Microsoft Azure#Quickstart
Amsterdam ONAP on Intel Openstack to full OOM install - see ONAP on Kubernetes on OpenStack
Installation Flow
1) To start - pick your infrastructure - ONAP supports deployment to all the major public clouds such as:
2) On your VM in the above infrastructure - create a Kubernetes environment (a one time activity) follow one of these guides:
- Setting up Kubernetes with Rancher (recommended)
- Setting up Kubernetes with Kubeadm
- Setting up Kubernetes with Cloudify
3) Install ONAP - use the cd.sh script provided as a patch in review
Known Issues
Beijing/master branch
These are current failures for individual ONAP compoents that have workarounds or can be ignored - as a service to everyone who asks - last update 20180314
AAF has a failing container - AAF-170 - Getting issue details... STATUS
SDC has an internal dependency issue and will need a coordinated restart - SDC-739 - Getting issue details... STATUS
SDNC has a timing issue and will need a restart - OOM-543 - Getting issue details... STATUS
(vCPE/vVOLTE use case only) VFC has a failing container - VFC-818 - Getting issue details... STATUS
(vCPE/vVOLTE use case only) VNFSDK has a failing container - VNFSDK-226 - Getting issue details... STATUS
Details
Installation templates for Kubernetes and deploying ONAP via OOM for the master and amsterdam branches are in the following locations
OOM-710 - Getting issue details... STATUS
Kubernetes devops / workarounds / troubleshooting content restored from version 378 - into the page ONAP on Kubernetes on Rancher
Example End to End Kubernetes based ONAP install and deployment
Scenario: installing Rancher on clean Ubuntu 16.04 64g VM (single collocated server/host) and the master branch of onap via OOM deployment (2 scripts)
1 hour video of automated installation on an AWS EC2 spot instance
oom_rancher_cd_full_auto_install_on_aws_ec2_20180227.mp4
Note: amsterdam will require a different onap-parameters.yaml
Either (the rancher install is kicked in by the entrypoint script) git clone https://gerrit.onap.org/r/oom cd oom git pull https://gerrit.onap.org/r/oom refs/changes/19/32019/6 cd install/rancher ./oom_rancher_setup.sh -b master -s cd.onap.info -e onap or just run the WIP oom_entrypoint.sh script (And swap out your own onap-parameters.yaml) use the latest on https://jira.onap.org/browse/OOM-710 wget https://jira.onap.org/secure/attachment/LATEST_ID/oom_entrypoint.sh chmod 777 oom_entrypoint.sh ./oom_entrypoint.sh -b master -s your_ip -e onap # refer to this script that runs oom_rancher_setup.sh and cd.sh in sequence
clone continuous deployment script - until it is merged
wget https://jira.onap.org/secure/attachment/ID/cd.sh chmod 777 cd.sh wget https://jira.onap.org/secure/attachment/ID/aaiapisimpledemoopenecomporg.cer wget https://jira.onap.org/secure/attachment/ID/onap-parameters.yaml wget https://jira.onap.org/secure/attachment/ID/aai-cloud-region-put.json ./cd.sh -b master -e onap # wait about 25-60 min depending on the speed of your network pulling docker images
Results
obrienbiometrics:azure michaelobrien$ ssh ubuntu@cd.onap.info Welcome to Ubuntu 16.04.3 LTS (GNU/Linux 4.4.0-1041-aws x86_64) ubuntu@ip-172-31-95-122:~$ sudo su - root@ip-172-31-95-122:~# wget https://jira.onap.org/secure/attachment/ID/oom_rancher_setup.sh --2018-02-27 15:51:09-- https://jira.onap.org/secure/attachment/ID/oom_rancher_setup.sh Saving to: ‘oom_rancher_setup.sh’ oom_rancher_setup.sh 100%[=========================================================================================>] 6.68K --.-KB/s in 0s root@ip-172-31-95-122:~# chmod 777 oom_rancher_setup.sh root@ip-172-31-95-122:~# wget https://jira.onap.org/secure/attachment/ID/cd.sh root@ip-172-31-95-122:~# wget https://jira.onap.org/secure/attachment/ID/aaiapisimpledemoopenecomporg.cer root@ip-172-31-95-122:~# wget https://jira.onap.org/secure/attachment/ID/onap-parameters.yaml root@ip-172-31-95-122:~# wget https://jira.onap.org/secure/attachment/ID/aai-cloud-region-put.json root@ip-172-31-95-122:~# ./oom_rancher_setup.sh -b master -s cd.onap.info -e onap Installing on cd.onap.info for master: Rancher: 1.6.14 Kubectl: 1.8.6 Helm: 2.6.1 Docker: 17.03 + sh -c docker version Client: Version: 17.03.2-ce Status: Downloaded newer image for rancher/server:v1.6.14 Saving to: ‘helm-v2.6.1-linux-amd64.tar.gz’ --2018-02-27 15:52:46-- https://releases.rancher.com/cli/v0.6.7/rancher-linux-amd64-v0.6.7.tar.gz Get:2 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/universe amd64 jq amd64 1.5+dfsg-1 [144 kB] Setting up jq (1.5+dfsg-1) ... publicValue: 896EC0CC76D9A171BFF0 secretValue: wH1x1hf5oJe9FSPLex1rgTkk5oabtRMrJGb9MJ6x ID NAME ORCHESTRATION STATE CREATED 1a5 Default cattle active 2018-02-27T15:53:04Z Creating kubernetes environment named onap env id: 1a7 you should see an additional kubernetes environment ID NAME ORCHESTRATION STATE CREATED 1a5 Default cattle active 2018-02-27T15:53:04Z 1a7 onap kubernetes active 2018-02-27T15:53:48Z Registering host for image: rancher/agent:v1.2.9 url: http://127.0.0.1:8880/v2-beta/scripts/9E6292FC1B3C65923DD2:1514678400000:Ri3dGEnzBp03z2DVtDrZ4nCQ0XE registrationToken: 9E6292FC1B3C65923DD2:1514678400000:Ri3dGEnzBp03z2DVtDrZ4nCQ0XE Status: Downloaded newer image for rancher/agent:v1.2.9 INFO: Running Agent Registration Process, CATTLE_URL=http://cd.onap.info:8880/v1 INFO: Attempting to connect to: http://cd.onap.info:8880/v1 INFO: http://cd.onap.info:8880/v1 is accessible INFO: Launched Rancher Agent: 2a4a71b60657b964701019de36c6045f842d7844057433362f715e201751cf6f waiting 7 min for host registration to finish KUBECTL_TOKEN base64 encoded: QmFzaWMgT0RrMlJVTXdRME0zTmtRNVFURTNNVUpHUmpBNmQwZ3hlREZvWmpWdlNtVTVSbE5RVEdWNE1YSm5WR3RyTlc5aFluUlNUWEpLUjJJNVRVbzJlQT09 Verify all pods up on the kubernetes system - will return localhost:8080 until a host is added kubectl get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE kube-system heapster-76b8cd7b5-bjckj 1/1 Running 0 6m kube-system kube-dns-5d7b4487c9-zz9cp 3/3 Running 0 6m kube-system kubernetes-dashboard-f9577fffd-tcxld 1/1 Running 0 6m kube-system monitoring-grafana-997796fcf-n4sw5 1/1 Running 0 6m kube-system monitoring-influxdb-56fdcd96b-h2jr8 1/1 Running 0 6m kube-system tiller-deploy-cc96d4f6b-6nlgh 1/1 Running 0 6m 172-31-95-122:~# nohup ./cd.sh -b master & [1] 21891 root@ip-172-31-95-122:~# nohup: ignoring input and appending output to 'nohup.out' root@ip-172-31-95-122:~# tail -f nohup.out Tue Feb 27 16:43:00 UTC 2018 verify onap-config is 0/1 not 1/1 - as in completed - an error pod - means you are missing onap-parameters.yaml or values are not set in it. waiting for config pod to complete pre pull docker images - 15+ min docker login -u docker -p docker nexus3.onap.org:10001 Login Succeeded docker pull oomk8s/readiness-check:1.1.0 Status: Downloaded newer image for nexus3.onap.org:10001/openecomp/appc-image:v1.2.0 waiting for last pull start onap pods ********** Creating deployments for policy ********** Creating namespace ********** namespace "onap-policy" created Creating service account ********** clusterrolebinding "onap-policy-admin-binding" created Creating registry secret ********** secret "onap-docker-registry-key" created Creating deployments and services ********** NAME: onap-policy LAST DEPLOYED: Tue Feb 27 17:20:31 2018 NAMESPACE: onap STATUS: DEPLOYED ==> v1/ConfigMap NAME DATA AGE policy-dep-pe-configmap 13 1s policy-dep-drools-settings-configmap 1 1s policy-sdk-log-configmap 1 1s policy-filebeat-configmap 1 1s policy-pdp-log-configmap 1 1s policy-drools-log-configmap 1 1s policy-pap-log-configmap 1 1s .... 8 pending > 0 at the 88th 15 sec interval onap-aaf aaf-6c64db8fdd-glxxv 0/1 Running 0 25m onap-sdnc dmaap-listener-579cdfd5cb-wd9gd 0/1 Init:0/1 2 26m onap-sdnc sdnc-0 0/2 Init:0/1 2 26m onap-sdnc sdnc-dbhost-0 0/2 Pending 0 26m onap-sdnc sdnc-dgbuilder-794d686f78-tgq5t 0/1 Init:0/1 2 26m onap-sdnc sdnc-portal-69b79b6646-g7zz2 0/1 Init:0/1 2 26m onap-sdnc ueb-listener-6ff56545b6-754v7 0/1 Init:0/1 2 26m onap-vfc vfc-ztevnfmdriver-585d8db4f7-q24z6 0/1 ImagePullBackOff 0 25m Basic Policy Health Check | PASS | root@ip-172-31-95-122:~# kubectl get pods --all-namespaces | grep onap-policy onap-policy brmsgw-869dc8888c-htkkz 1/1 Running 0 1h onap-policy drools-5f568559b-vgtd9 2/2 Running 0 1h onap-policy mariadb-7c66956bf-tkszv 1/1 Running 0 1h onap-policy nexus-69f7497db5-rnmt9 1/1 Running 0 1h onap-policy pap-7fbff7db64-fpqgj 2/2 Running 0 1h onap-policy pdp-55c454589-nzfcf 2/2 Running 0 1h root@ip-172-31-95-122:~# kubectl get services --all-namespaces | grep onap-policy onap-policy brmsgw NodePort 10.43.5.29 <none> 9989:30216/TCP 1h onap-policy drools NodePort 10.43.96.150 <none> 6969:30217/TCP,9696:30221/TCP 1h onap-policy mariadb ClusterIP None <none> 3306/TCP 1h onap-policy nexus ClusterIP None <none> 8081/TCP 1h onap-policy pap NodePort 10.43.125.137 <none> 8443:30219/TCP,9091:30218/TCP 1h onap-policy pdp NodePort 10.43.130.155 <none> 8081:30220/TCP 1h root@ip-172-31-95-122:~# curl http://cd.onap.info:30220/pdp/swagger-ui.html#/policy-engine-services <div id="swagger-ui-container" class="swagger-ui-wrap"></div> root@ip-172-31-95-122:~# # will post a curl REST get/post in a minute
Manual Quickstart Installation - Please use script instead
(Manual instructions) - try to use the script at Setting up Kubernetes with Rancher via https://gerrit.onap.org/r/#/c/32019
ONAP Minimum R1 Installation Helm Apps
oom/kubernetes/oneclick/setenv.bash maybe updated to the following reduce app set.
|
1) install rancher, clone oom, run config-init pod, run one or all onap components
***************** Note: uninstall docker if already installed - as Kubernetes 1.8 under rancher supports 17.03.2 as of 20180124
***************** |
|---|
Install Rancher
ONAP deployment in kubernetes is modelled in the oom project as a 1:1 set of service:pod sets (1 pod per docker container). The fastest way to get ONAP Kubernetes up is via Rancher on any bare metal or VM that supports a clean Ubuntu 16.04 install and more than 60G ram.
TODO: REMOVE from table cell - wrapping is not working
(on each host) add to your /etc/hosts to point your ip to your hostname (add your hostname to the end). Add entries for all other hosts in your cluster. For example on openlab - you will need to add the name of your host before you install docker - to avoid the error below sudo: unable to resolve host onap-oom
Open Ports On most hosts like openstack or EC2 you can open all the ports or they are open by default - on some environments like Rackspace VM's you need to open them
Fix virtual memory allocation (to allow onap-log:elasticsearch to come up under Rancher 1.6.11+)
clone oom (scp your onap_rsa private key first - or clone anon - Ideally you get a full gerrit account and join the community) see ssh/http/http access links below https://gerrit.onap.org/r/#/admin/projects/oom
(on each host (server and client(s) which may be the same machine)) Install only the 17.03.2 version of Docker (the only version that works with Kubernetes in Rancher 1.6.13+) Install Docker
Pre pull docker images the first time you install onap. Currently the pre-pull will take 16-180 min depending on your network. Pre pulling the images will allow the entire ONAP to start in 3-8 min instead of up to 3 hours.
Use script above in oom/kubernetes/config once it is merged. https://git.onap.org/oom/tree/kubernetes/config/prepull_docker.sh
To monitor when prepull is finished see section: Prepulldockerimages. It is advised to wait until pre pull has finished before continuing. (on the master only) Install rancher (Optional: use 8880 instead of 8080 if there is a conflict) - note there may be issues with the dns pod in Rancher after a reboot or when running clustered hosts - a clean system will be OK -
In Rancher UI - dont use (http://127.0.0.1:8880) - use the real IP address - so the client configs are populated correctly with callbacks You must deactivate the default CATTLE environment - by adding a KUBERNETES environment - and Deactivating the older default CATTLE one - your added hosts will attach to the default
Register your hostRegister your host(s) - run following on each host (including the master if you are collocating the master/host on a single machine/vm) For each host, In Rancher > Infrastructure > Hosts. Select "Add Host" The first time you add a host - you will be presented with a screen containing the routable IP - hit save only on a routable IP. Enter IP of host: (if you launched racher with 127.0.0.1/localhost - otherwise keep it empty - it will autopopulate the registration with the real IP)
Copy command to register host with Rancher,
Execute command on each host, for example:
wait for kubernetes menu to populate with the CLI Install KubectlThe following will install kubectl (for Kubernetes 1.8.6 ) https://github.com/kubernetes/kubernetes/issues/57528 on a linux host. Once configured, this client tool will provide management of a Kubernetes cluster.
Paste kubectl config from Rancher (you will see the CLI menu in Rancher / Kubernetes after the k8s pods are up on your host) Click on "Generate Config" to get your content to add into .kube/config Verify that Kubernetes config is good
Install HelmThe following will install Helm - currently 2.8.0 on a linux host. Helm is used by OOM for package and configuration management. https://lists.onap.org/pipermail/onap-discuss/2018-January/007674.html Prerequisite: Install Kubectl (2.6.1 comes with Rancher 1.6.14 - for master)
NOTE: If helm version takes long time and eventually errors out, this is most likely because incoming access to port 10250 (exposed by kubelet) is blocked by firewall. Make sure to configure firewall accordingly |
