Blog

For the Beijing release of ONAP on OOM, accessing the ONAP portal from the user's own environment (laptop etc.) was a frequently requested feature. 

So to achieve that, what we've decided on doing is to expose the portal application's port 8989 through a K8s LoadBalancer object.

This has some non-obvious implications in a clustered Kubernetes environment specifically where the K8s cluster nodes communicate with each other via a private network that isn't publicly accessible (i.e. Openstack VMs with private internal network).

Typically, to be able to access the K8s nodes publicly a public address is assigned.  In Openstack this is a floating IP address.


What happens when the portal-app chart is deployed is that a K8s service is created that instantiates a load balancer.  The LB chooses the private interface of one of the nodes as in the example below (10.0.0.4 is private to the K8s cluster only).  Then to be able to access the portal on port 8989 from outside theK K8s & Openstack environment, the user needs to assign/get the floating IP address that corresponds to the private IP.

kubectl -n onap get services|grep "portal-app"

portal-app                LoadBalancer   10.43.142.201   10.0.0.4                               8989:30215/TCP,8006:30213/TCP,8010:30214/TCP                                 1d        app=portal-app,release=dev

In this example, that public floating IP is 10.12.5.155 which can be obtained through the horizon GUI or the Openstack CLI for your tenant (openstack server list) .  That IP is then used in your /etc/hosts to map the fixed DNS aliases required by the ONAP Portal as shown below.

10.12.6.155 portal.api.simpledemo.onap.org
10.12.6.155 vid.api.simpledemo.onap.org
10.12.6.155 sdc.api.fe.simpledemo.onap.org
10.12.6.155 portal-sdk.simpledemo.onap.org
10.12.6.155 policy.api.simpledemo.onap.org
10.12.6.155 aai.api.sparky.simpledemo.onap.org
10.12.6.155 cli.api.simpledemo.onap.org
10.12.6.155 msb.api.discovery.simpledemo.onap.org

Ensure you've disabled any proxy settings the browser you are using to access the portal and then simply access the familiar URL:

http://portal.api.simpledemo.onap.org:8989/ONAPPORTAL/login.htm


Other things we tried:

We went through using Kubernetes port forwarding but thought it a little clunky for the end user to have to use a script to open up port forwarding tunnels to each K8s pod that provides a portal application widget

We considered bringing back the VNC chart with a different image but there were many issues with resolution, lack of volume mount, /etc/hosts dynamic update, file upload that were a tall order to solve in time for the Beijing release.






I was banging my head against the wall trying to figure out why my K8s configmap was ignoring my file permission settings and injecting a read-only version into the container.  What was stranger was that it worked in my personal environment and not in the new one I set up for myself in the Windriver integration lab.

Well it turns out that there is a version difference between the 2 environments and this is exactly what the issue is.  In the 1.8.9 version of K8s, a fix went in that ensures all configmaps, secrets,

My personal environment is running K8s 1.8.5 and the Windriver environment K8s 1.8.10

Fix impact:

Secret, configMap, downwardAPI and projected volumes will be mounted as read-only volumes. Applications that attempt to write to these volumes will receive read-only filesystem errors. Previously, applications were allowed to make changes to these volumes, but those changes were reverted at an arbitrary interval by the system. Applications should be re-configured to write derived files to another location.


Here is the bug that led me to the discovery: https://github.com/coreos/bugs/issues/2384

The commit: https://github.com/kubernetes/kubernetes/issues/60814

And the changelog: https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.8.md#changelog-since-v188


This was the ticket I was working at the time and I know that SO has a similar problem where the container is trying to modify/move a file/directory that is injected via configmap.

OOM-900 - Getting issue details... STATUS

Here is the code.  Note that setting the defaultMode: or mode: of a volume to something that is writeable isn't being honored anymore.

https://gerrit.onap.org/r/#/c/42325/



In OOM, many of us use Rancher to deploy Kubernetes to run ONAP.

Recently, I've had to update my lab environment from Rancher and Kubernetes 1.5 to rancher to v1.6 in order to upgrade the Kubernetes stack to v1.7.   I followed the steps on the Rancher website with 1 minor tweak: I used the rancher/server:stable image tag instead of latest

http://rancher.com/docs/rancher/latest/en/upgrading/#single-container

My rancher server container name was pensive_saha

docker stop pensive_saha
docker create --volumes-from pensive_saha --name rancher-data rancher/server:stable
docker pull rancher/server:stable
docker run -d --volumes-from rancher-data --restart=unless-stopped -p 8080:8080 rancher/server:stable

Once the server is back up, navigate to the rancher web page and navigate to Kubernetes > Infrastructure Stacks.

The "Up to date" button shown below, will say "Upgrade Available" for each stack. 

For each stack component, hit the button and select the most recent template from drop down, leave any input fields as their default and choose "Upgrade". 

Example: upgrading the healthcheck service stack

Rancher will do its thing to each K8s stack component and after it is done there will be a button that says "Complete upgrade".  Click that and the upgrade of the stack component will be complete.

You can now move on to the next component.

Once all stack status is "Green", the upgrade is complete.

To be deleted