Important: As Windriver is ONAP community lab, the access/privilege for all users are same. Hence do NOT delete any folders/files/vm unless you created it and are sole users (otherwise need to be communicated to the community/discuss-list and PTL before purging it). Other than jumpserver -10.12.5.50 (provided by Stephen) – all other VM’s should be considered volatile and hence any development changes being worked on windriver lab should be backed-up locally.
Redeployment steps
1. After login to Windriver, identify the K8S cluster and corresponding nfs server ip from dashboard - http://10.12.25.2/project/instances/. Typically there are two separate clusters setup, for Frankfurt – following can be used.
Note: The k8s-01 and k8s-02 are k8s node server. The cluster/nfs ip’s would change for every release and deployments will be periodically refreshed (every ~2 weeks)
2. Login using onap_dev key to required NFS server and su as “root”
ssh -i ~/onap_dev ubuntu@10.12.7.110
sudo su -
3. Check if required platform container/pod is running via kubectl
root@onap2-nfs:~# kubectl get pods -n onap | egrep "inventory|dashboard" dev-dcaegen2-dcae-dashboard-f6b57bd89-4fjv9 2/2 Running 0 3d19h dev-dcaegen2-dcae-dashboard-pg-primary-56c878df5d-c8z75 1/1 Running 0 4d19h dev-dcaegen2-dcae-dashboard-pg-replica-8659dd7f8-7m4h8 1/1 Running 0 4d19h dev-dcaegen2-dcae-dashboard-pgpool-6cb5c4dc76-ffzgs 1/1 Running 0 4d19h dev-dcaegen2-dcae-dashboard-pgpool-6cb5c4dc76-w6tl9 1/1 Running 0 4d19h dev-dcaegen2-dcae-inventory-api-6d98b567d4-zx6zq 1/1 Running 0 3d19h
4. All DCAE platform components are packages/deployed via helm as dcaegen2 chart - this can be verified using helm list command to identify how current platform components are deployed
root@onap2-nfs:~# helm list NAME REVISION UPDATED STATUS CHART APP VERSION NAMESPACE dcae-mongo 1 Fri Jan 24 21:32:23 2020 DEPLOYED dcae-mongo-6.0.0 onap dcae-redis 1 Fri Jan 24 20:48:23 2020 DEPLOYED dcae-redis-6.0.0 onap dev 1 Thu Jan 23 20:36:21 2020 DEPLOYED onap-5.0.0 El Alto onap dev-aaf 1 Thu Jan 23 20:36:22 2020 DEPLOYED aaf-5.0.0 onap dev-consul 1 Thu Jan 23 20:36:23 2020 DEPLOYED consul-5.0.0 onap dev-dcaegen2 2 Fri Jan 24 20:43:07 2020 DEPLOYED dcaegen2-6.0.0 onap dev-dmaap 1 Thu Jan 23 20:36:27 2020 FAILED dmaap-5.0.0 onap dev-msb 1 Thu Jan 23 20:41:29 2020 DEPLOYED msb-5.0.0 onap dev-policy 1 Thu Jan 23 20:41:30 2020 DEPLOYED policy-5.0.0 onap
helm charts for each platform component can be found under ~/oom/kubernetes/dcaegen2/components (maintained in gerrit under OOM repository https://git.onap.org/oom/tree/kubernetes/dcaegen2/components)
root@onap2-nfs:~/oom/kubernetes/dcaegen2/components# ls -lrt total 40 drwxr-xr-x 5 root root 4096 Jan 23 19:03 dcae-bootstrap drwxr-xr-x 5 root root 4096 Jan 23 19:03 dcae-cloudify-manager drwxr-xr-x 5 root root 4096 Jan 23 19:04 dcae-config-binding-service drwxr-xr-x 4 root root 4096 Jan 23 19:04 dcae-healthcheck drwxr-xr-x 5 root root 4096 Jan 23 19:04 dcae-servicechange-handler drwxr-xr-x 5 root root 4096 Jan 23 19:04 dcae-inventory-api drwxr-xr-x 5 root root 4096 Jan 23 19:04 dcae-deployment-handler drwxr-xr-x 5 root root 4096 Jan 23 19:04 dcae-policy-handler drwxr-xr-x 5 root root 4096 Jan 23 19:04 dcae-dashboard drwxr-xr-x 5 root root 4096 Jan 24 21:06 dcae-redis
5. If component is already running, to undeploy a specific component installed under dev-dcaegen2 helm release, edit (or make a copy) the following ~/dcae_component_override.yaml and undeploy it via helm upgrade command.
In the example below, updated both dcae-redis and dcae-dashboard enabled flag to “false”
root@onap2-nfs:~# cat dcae_component_override.yaml dcae-bootstrap: enabled: true dcae-cloudify-manager: enabled: true dcae-config-binding-service: enabled: true dcae-healthcheck: enabled: true dcae-redis: enabled: false dcae-servicechange-handler: enabled: true dcae-inventory-api: enabled: true dcae-deployment-handler: enabled: true dcae-policy-handler: enabled: true dcae-dashboard: enabled: false
Followed by doing helm upgrade with updated yaml to trigger the termination.
helm upgrade --install dev-dcaegen2 local/dcaegen2 --namespace onap -f ~/dcae_component_override.yaml
This will undeploy/terminate dashboard component (redis was undeployed previously)
root@onap2-nfs:~# kubectl get pods -n onap | egrep "inventory|dashboard" dev-dcaegen2-dcae-dashboard-f6b57bd89-4fjv9 0/2 Terminating 0 3d19h dev-dcaegen2-dcae-inventory-api-6d98b567d4-zx6zq 1/1 Running 0 3d19h root@onap2-nfs:~# kubectl get pods -n onap | egrep "inventory|dashboard" dev-dcaegen2-dcae-inventory-api-6d98b567d4-zx6zq 1/1 Running 0 3d19h
More details on helm upgrade and override file update per subcomponent can be found under parent page - https://wiki.onap.org/pages/viewpage.action?pageId=71837415
6. To reinstall new version, goto chart directly, ensure the values.yaml under step #4 is updated to reflect the new version and any required config changes in charts
helm install -n <release name> --namespace onap <chart locations>
Example :
helm install -n dcae-dashboard --namespace onap ./dcae-dashboard
This brings up dashboard pods and Helm list also indicates the new deployment
root@onap2-nfs:~/oom/kubernetes/dcaegen2/components# kubectl get pods -n onap | egrep "inventory|dashboard" dcae-dashboard-dcae-dashboard-6b5c9ccd4c-lx7cw 2/2 Running 0 2m40s dcae-dashboard-dcae-dashboard-pg-primary-d8b649c7-gqrsx 1/1 Running 0 2m40s dcae-dashboard-dcae-dashboard-pg-replica-c4cfbd8fc-tnbpf 1/1 Running 0 2m40s dcae-dashboard-dcae-dashboard-pgpool-7bcf656f68-4555q 1/1 Running 0 2m40s dcae-dashboard-dcae-dashboard-pgpool-7bcf656f68-gqbd5 1/1 Running 0 2m40s dev-dcaegen2-dcae-inventory-api-6d98b567d4-zx6zq 1/1 Running 0 3d19h root@onap2-nfs:~/oom/kubernetes/dcaegen2/components# helm list NAME REVISION UPDATED STATUS CHART APP VERSION NAMESPACE dcae-dashboard 1 Tue Jan 28 16:01:41 2020 DEPLOYED dcae-dashboard-6.0.0 1.0 onap dcae-mongo 1 Fri Jan 24 21:32:23 2020 DEPLOYED dcae-mongo-6.0.0 onap dcae-redis 1 Fri Jan 24 20:48:23 2020 DEPLOYED dcae-redis-6.0.0 onap dev 1 Thu Jan 23 20:36:21 2020 DEPLOYED onap-5.0.0 El Alto onap dev-aaf 1 Thu Jan 23 20:36:22 2020 DEPLOYED aaf-5.0.0 onap dev-consul 1 Thu Jan 23 20:36:23 2020 DEPLOYED consul-5.0.0 onap dev-dcaegen2 3 Tue Jan 28 15:58:47 2020 DEPLOYED dcaegen2-6.0.0 onap dev-dmaap 1 Thu Jan 23 20:36:27 2020 FAILED dmaap-5.0.0 onap dev-msb 1 Thu Jan 23 20:41:29 2020 DEPLOYED msb-5.0.0 onap dev-policy 1 Thu Jan 23 20:41:30 2020 DEPLOYED policy-5.0.0 onap
7. To delete the installed chart, use helm purge
helm del --purge <release_name>
Example : helm del --purge dcae-dashboard
root@onap2-nfs:~/oom/kubernetes/dcaegen2/components# kubectl get pods -n onap | egrep "inventory|dashboard" dcae-dashboard-dcae-dashboard-6b5c9ccd4c-lx7cw 1/2 Terminating 0 6m55s dcae-dashboard-dcae-dashboard-pg-primary-d8b649c7-gqrsx 0/1 Terminating 0 6m55s dcae-dashboard-dcae-dashboard-pgpool-7bcf656f68-4555q 0/1 Terminating 0 6m55s dev-dcaegen2-dcae-inventory-api-6d98b567d4-zx6zq 1/1 Running 0 3d19h root@onap2-nfs:~/oom/kubernetes/dcaegen2/components# kubectl get pods -n onap | egrep "inventory|dashboard" dev-dcaegen2-dcae-inventory-api-6d98b567d4-zx6zq 1/1 Running 0 3d19h
Note: For redeploy, follow step #6 again.
Development/build of components in Windriver
- Find the k8s node server where POD is running using kubectl
kubectl get pods -n onap -o=wide | grep <podname>
- Login to k8s node server (same key/user) and create a sub-folder on your name
- clone the repo/patchset from gerrit (there is download option in gerrit, when clicked clicked it will give the exact pull request (or checkout) command)
- Build image (using mvn <goals>) to generate docker image
- Once build is successful and docker image generated, retag it for uniqueness ( nexus3.onap.org:10001/onap/<imagename>:<uniquetag>)
- For redeploy, you can either update the values.yaml charts (under ~/oom/kubernetes/dcaegen2/components) and follow helm commands above to redeploy, or find the corresponding deployment descriptor (via kubectl) and update the version and save (k8s would automatically terminate current pod and redeploy new based on version provided)Note: Any changes done on helm charts/values.yaml should be submitted into OOM repository (https://git.onap.org/oom/tree/kubernetes/dcaegen2/components) after new images has been released in DCAE.