References
OOM-1
-
Getting issue details...
STATUS
The OOM (ONAP Operation Manager) project has pushed Kubernetes based deployment code to the oom repository. This page details on getting ONAP running on Kubernetes for various environments.
Undercloud Installation
We need a kubernetes installation with the proper architecture components running. This architecture can be provided by vendors like Redhat or Rancher
https://kubernetes.io/docs/concepts/overview/components/
Ubuntu 16.04 Install Session
Install Rancher
http://rancher.com/docs/rancher/v1.5/en/quick-start-guide/
http://rancher.com/docs/rancher/v1.6/en/installing-rancher/installing-server/#single-container
Install a docker version that Rancher and Kubernetes support which is currently 1.12.3
http://rancher.com/docs/rancher/v1.5/en/hosts/#supported-docker-versions
Verify your Rancher admin console is up on the external port you configured above
Wait for the docker container to finish DB startup
Having issues registering a combined single VM (controller + host) - moving on to using 2 VM's
http://rancher.com/docs/rancher/v1.6/en/hosts/
ONAP Installation
Clone
Install the latest version of the OOM (ONAP Operations Manager) project repo - specifically the ONAP on Kubernetes work just uploaded June 2017
https://gerrit.onap.org/r/gitweb?p=oom.git
Provision
Manually
Start a service
In this case robot - to check your Kubernetes installation.
[root@obrien-mbp oneclick]# ./createAll.bash -n onap -a robot ********** Creating up ONAP: robot Creating namespaces ********** namespace "onap-robot" created Creating services ********** service "robot" created ********** Creating deployments for robot ********** Robot.... deployment "robot" created **** Done ****[root@obrien-mbp oneclick]# kubectl cluster-info Kubernetes master is running at http://localhost:8080 To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
|
---|
Kubernetes Rest api
http://localhost:8080
{ "paths": [ "/api", "/api/v1", "/apis", "/apis/apps", "/apis/apps/v1beta1", "/apis/authentication.k8s.io", "/apis/authentication.k8s.io/v1beta1", "/apis/authorization.k8s.io", "/apis/authorization.k8s.io/v1beta1", "/apis/autoscaling", "/apis/autoscaling/v1", "/apis/batch", "/apis/batch/v1", "/apis/batch/v2alpha1", "/apis/certificates.k8s.io", "/apis/certificates.k8s.io/v1alpha1", "/apis/extensions", "/apis/extensions/v1beta1", "/apis/policy", "/apis/policy/v1beta1", "/apis/rbac.authorization.k8s.io", "/apis/rbac.authorization.k8s.io/v1alpha1", "/apis/storage.k8s.io", "/apis/storage.k8s.io/v1beta1", "/healthz", "/healthz/ping", "/healthz/poststarthook/bootstrap-controller", "/healthz/poststarthook/extensions/third-party-resources", "/healthz/poststarthook/rbac/bootstrap-roles", "/logs", "/metrics", "/swaggerapi/", "/ui/", "/version" ] } |
---|
Pod List
In verification
[root@obrien-mbp oneclick]# ./createAll.bash -n onap
[root@obrien-mbp oneclick]# kubectl get service --all-namespaces=true
NAMESPACE NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default kubernetes 10.254.0.1 <none> 443/TCP 5h
onap-aai aai-service 10.254.172.161 <nodes> 8443:30233/TCP,8080:30232/TCP 1m
onap-aai hbase None <none> 8020/TCP 1m
onap-aai model-loader-service 10.254.138.128 <nodes> 8443:30229/TCP,8080:30210/TCP 1m
onap-appc dbhost None <none> 3306/TCP 1m
onap-appc dgbuilder 10.254.89.166 <nodes> 3000:30228/TCP 1m
onap-appc sdnctldb01 None <none> 3306/TCP 1m
onap-appc sdnctldb02 None <none> 3306/TCP 1m
onap-appc sdnhost 10.254.68.235 <nodes> 8282:30230/TCP,1830:30231/TCP 1m
onap-message-router dmaap 10.254.149.48 <nodes> 3904:30227/TCP,3905:30226/TCP 1m
onap-message-router global-kafka None <none> 9092/TCP 1m
onap-message-router zookeeper None <none> 2181/TCP 1m
onap-mso mariadb 10.254.254.189 <nodes> 3306:30252/TCP 1m
onap-mso mso 10.254.32.65 <nodes> 8080:30223/TCP,3904:30225/TCP,3905:30224/TCP,9990:30222/TCP,8787:30250/TCP 1m
onap-policy brmsgw 10.254.207.78 <nodes> 9989:30216/TCP 1m
onap-policy drools 10.254.114.125 <nodes> 6969:30217/TCP 1m
onap-policy mariadb None <none> 3306/TCP 1m
onap-policy nexus None <none> 8081/TCP 1m
onap-policy pap 10.254.126.231 <nodes> 8443:30219/TCP,9091:30218/TCP 1m
onap-policy pdp 10.254.244.112 <nodes> 8081:30220/TCP 1m
onap-policy pypdp 10.254.176.200 <nodes> 8480:30221/TCP 1m
onap-portal portalapps 10.254.198.12 <nodes> 8006:30213/TCP,8010:30214/TCP,8989:30215/TCP 1m
onap-portal portaldb None <none> 3306/TCP 1m
onap-portal vnc-portal 10.254.0.129 <nodes> 6080:30211/TCP,5900:30212/TCP 1m
onap-robot robot 10.254.255.97 <nodes> 88:30209/TCP 5h
onap-sdc sdc-be 10.254.109.11 <nodes> 8443:30204/TCP,8080:30205/TCP 1m
onap-sdc sdc-cs None <none> 9042/TCP,9160/TCP 1m
onap-sdc sdc-es None <none> 9200/TCP,9300/TCP 1m
onap-sdc sdc-fe 10.254.219.219 <nodes> 9443:30207/TCP,8181:30206/TCP 1m
onap-sdc sdc-kb None <none> 5601/TCP 1m
onap-sdnc dbhost None <none> 3306/TCP 1m
onap-sdnc sdnc-dgbuilder 10.254.79.74 <nodes> 3000:30203/TCP 1m
onap-sdnc sdnc-portal 10.254.162.50 <nodes> 8843:30201/TCP 1m
onap-sdnc sdnctldb01 None <none> 3306/TCP 1m
onap-sdnc sdnctldb02 None <none> 3306/TCP 1m
onap-sdnc sdnhost 10.254.233.66 <nodes> 8282:30202/TCP 1m
onap-vid vid-mariadb None <none> 3306/TCP 1m
onap-vid vid-server 10.254.83.194 <nodes> 8080:30200/TCP 1m
OSX
Minicube (not in use)
Redhat 7.3
Running onap kubernetes services in a single VM using Redhat Kubernetes for 7.3
Redhat provides 2 docker containers for the scheduler and nbi components and spins up 2 (# is scalable) pod containers for use by onap.
[root@obrien-mbp oneclick]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ee02bbab2037 rhel7/kubernetes-scheduler "/usr/bin/kube-schedu" 40 hours ago Up 40 hours k8s_kube-scheduler.4e069b78_kube-scheduler-127.0.0.1_default_78147ee23cd674839c926daaa58595e5_f4ada53e f5031b2923ca rhel7/kubernetes-apiserver "/usr/bin/kube-apiser" 40 hours ago Up 40 hours k8s_kube-apiserver.41e368d_kube-apiserver-127.0.0.1_default_ab6617fd8366917b3d6b8c7bb6cbcfcf_8d671f6c 4c5e96ea1074 registry.access.redhat.com/rhel7/pod-infrastructure:latest "/pod" 40 hours ago Up 40 hours k8s_POD.ae8ee9ac_kube-scheduler-127.0.0.1_default_78147ee23cd674839c926daaa58595e5_0ce93fa0 3316c73036fc registry.access.redhat.com/rhel7/pod-infrastructure:latest "/pod" 40 hours ago Up 40 hours k8s_POD.ae8ee9ac_kube-apiserver-127.0.0.1_default_ab6617fd8366917b3d6b8c7bb6cbcfcf_8c0dda0f |
---|
Kubernetes setup
Uninstall docker-se (we installed earlier) Follow https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux_atomic_host/7/html-single/getting_started_with_kubernetes/ subscription-manager repos --enable=rhel-7-server-optional-rpms sudo yum remove docker-ce sudo yum remove docker-ce-selinux yum install docker kubernetes-client kubernetes-node etcd docker ps systemctl disable firewalld systemctl stop firewalld yum install docker-distribution systemctl start docker-distribution systemctl enable docker-distribution systemctl is-active docker-distribution docker images systemctl start docker docker ps docker images docker pull registry.access.redhat.com/rhel7/kubernetes-apiserver docker pull registry.access.redhat.com/rhel7/kubernetes-controller-mgr docker pull registry.access.redhat.com/rhel7/kubernetes-scheduler mkdir /etc/kubernetes/manifests vi /etc/kubernetes/manifests/apiserver-pod.json vi /etc/kubernetes/manifests/controller-mgr-pod.json vi /etc/kubernetes/manifests/scheduler-pod.json vi /etc/kubernetes/kubelet vi kubestart.sh chmod 777 kubestart.sh [root@obrien-mbp opt]# ./kubestart.sh Created symlink from /etc/systemd/system/multi-user.target.wants/docker.service to /usr/lib/systemd/system/docker.service. active Created symlink from /etc/systemd/system/multi-user.target.wants/etcd.service to /usr/lib/systemd/system/etcd.service. active Created symlink from /etc/systemd/system/multi-user.target.wants/kube-proxy.service to /usr/lib/systemd/system/kube-proxy.service. active Created symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /usr/lib/systemd/system/kubelet.service. active ? docker.service - Docker Application Container Engine Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled) Drop-In: /usr/lib/systemd/system/docker.service.d ??flannel.conf Active: active (running) since Tue 2017-06-27 14:26:09 PDT; 196ms ago Docs: http://docs.docker.com Main PID: 90732 (dockerd-current) Memory: 14.6M CGroup: /system.slice/docker.service ??90732 /usr/bin/dockerd-current --add-runtime docker-runc=/usr/libexec/docker/docker-runc-current --default-runtime=docker-runc --authorization-plugin=rhel-p... ??90740 /usr/bin/docker-containerd-current -l unix:///var/run/docker/libcontainerd/docker-containerd.sock --shim docker-containerd-shim --metrics-interval=0 -... Jun 27 14:26:08 obrien-mbp.onap.org dockerd-current[90732]: time="2017-06-27T14:26:08.923309259-07:00" level=info msg="[graphdriver] using prior storage driver \"overlay\"" Jun 27 14:26:08 obrien-mbp.onap.org dockerd-current[90732]: time="2017-06-27T14:26:08.932489041-07:00" level=info msg="Graph migration to content-addressability... seconds" Jun 27 14:26:08 obrien-mbp.onap.org dockerd-current[90732]: time="2017-06-27T14:26:08.933459958-07:00" level=info msg="Loading containers: start." Jun 27 14:26:08 obrien-mbp.onap.org dockerd-current[90732]: .time="2017-06-27T14:26:08.944192986-07:00" level=info msg="Firewalld running: false" Jun 27 14:26:09 obrien-mbp.onap.org dockerd-current[90732]: time="2017-06-27T14:26:09.063447350-07:00" level=info msg="Default bridge (docker0) is assigned with... address" Jun 27 14:26:09 obrien-mbp.onap.org dockerd-current[90732]: time="2017-06-27T14:26:09.106280840-07:00" level=info msg="Loading containers: done." Jun 27 14:26:09 obrien-mbp.onap.org dockerd-current[90732]: time="2017-06-27T14:26:09.106331336-07:00" level=info msg="Daemon has completed initialization" Jun 27 14:26:09 obrien-mbp.onap.org dockerd-current[90732]: time="2017-06-27T14:26:09.106343838-07:00" level=info msg="Docker daemon" commit="1398f24/1.12.6" gr...on=1.12.6 Jun 27 14:26:09 obrien-mbp.onap.org dockerd-current[90732]: time="2017-06-27T14:26:09.116502253-07:00" level=info msg="API listen on /var/run/docker.sock" Jun 27 14:26:09 obrien-mbp.onap.org systemd[1]: Started Docker Application Container Engine. Hint: Some lines were ellipsized, use -l to show in full. ? kube-proxy.service - Kubernetes Kube-Proxy Server Loaded: loaded (/usr/lib/systemd/system/kube-proxy.service; enabled; vendor preset: disabled) Active: active (running) since Tue 2017-06-27 14:26:09 PDT; 165ms ago Docs: https://github.com/GoogleCloudPlatform/kubernetes Main PID: 90864 (kube-proxy) Memory: 14.5M CGroup: /system.slice/kube-proxy.service ??90864 /usr/bin/kube-proxy --logtostderr=true --v=0 --master=http://127.0.0.1:8080 Jun 27 14:26:09 obrien-mbp.onap.org systemd[1]: Started Kubernetes Kube-Proxy Server. Jun 27 14:26:09 obrien-mbp.onap.org systemd[1]: Starting Kubernetes Kube-Proxy Server... ? kubelet.service - Kubernetes Kubelet Server Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled) Active: active (running) since Tue 2017-06-27 14:26:09 PDT; 151ms ago Docs: https://github.com/GoogleCloudPlatform/kubernetes Main PID: 90917 (kubelet) Memory: 13.9M CGroup: /system.slice/kubelet.service ??90917 /usr/bin/kubelet --logtostderr=true --v=0 --api-servers=http://127.0.0.1:8080 --address=127.0.0.1 --hostname-override=127.0.0.1 --allow-privileged=fal... Jun 27 14:26:09 obrien-mbp.onap.org systemd[1]: Started Kubernetes Kubelet Server. Jun 27 14:26:09 obrien-mbp.onap.org systemd[1]: Starting Kubernetes Kubelet Server... Jun 27 14:26:09 obrien-mbp.onap.org kubelet[90917]: Flag --api-servers has been deprecated, Use --kubeconfig instead. Will be removed in a future version. Jun 27 14:26:09 obrien-mbp.onap.org kubelet[90917]: Flag --config has been deprecated, Use --pod-manifest-path instead. Will be removed in a future version. Jun 27 14:26:09 obrien-mbp.onap.org kubelet[90917]: I0627 14:26:09.697783 90917 feature_gate.go:181] feature gates: map[] Jun 27 14:26:09 obrien-mbp.onap.org kubelet[90917]: W0627 14:26:09.698143 90917 server.go:605] Could not load kubeconfig file /var/lib/kubelet/kubeconfig: sta... instead. Jun 27 14:26:09 obrien-mbp.onap.org kubelet[90917]: I0627 14:26:09.698654 90917 docker.go:356] Connecting to docker on unix:///var/run/docker.sock Jun 27 14:26:09 obrien-mbp.onap.org kubelet[90917]: I0627 14:26:09.698667 90917 docker.go:376] Start docker client with request timeout=2m0s Jun 27 14:26:09 obrien-mbp.onap.org kubelet[90917]: E0627 14:26:09.702077 90917 cni.go:163] error updating cni config: No networks found in /etc/cni/net.d Jun 27 14:26:09 obrien-mbp.onap.org kubelet[90917]: I0627 14:26:09.711775 90917 manager.go:143] cAdvisor running in container: "/system.slice/kubelet.service" Hint: Some lines were ellipsized, use -l to show in full. [root@obrien-mbp opt]# ss -tulnp | grep -E "(kube)|(etcd)" tcp LISTEN 0 128 127.0.0.1:2379 *:* users:(("etcd",pid=90567,fd=6)) tcp LISTEN 0 128 127.0.0.1:2380 *:* users:(("etcd",pid=90567,fd=5)) tcp LISTEN 0 128 127.0.0.1:10255 *:* users:(("kubelet",pid=90917,fd=9)) tcp LISTEN 0 128 127.0.0.1:10248 *:* users:(("kubelet",pid=90917,fd=8)) tcp LISTEN 0 128 127.0.0.1:10249 *:* users:(("kube-proxy",pid=90864,fd=9)) tcp LISTEN 0 128 127.0.0.1:10250 *:* users:(("kubelet",pid=90917,fd=14)) tcp LISTEN 0 128 :::10251 :::* users:(("kube-scheduler",pid=91175,fd=8)) tcp LISTEN 0 128 :::6443 :::* users:(("kube-apiserver",pid=91126,fd=42)) tcp LISTEN 0 128 :::8080 :::* users:(("kube-apiserver",pid=91126,fd=43)) tcp LISTEN 0 128 :::4194 :::* users:(("kubelet",pid=90917,fd=13)) [root@obrien-mbp opt]# curl -s -L http://localhost:2379/version {"etcdserver":"3.1.7","etcdcluster":"3.1.0"}[root@obrien-mbp opt]# |
---|