ONAP OOM Beijing - Hosting docker images locally

This page is intended for discussions on the various ways of hosting the docker images locally and addressing the problem of re-pulling the images from the public ONAP nexus proxy every time ONAP OOM is installed. This would provide a local caching of images, minimizing the pod synchronization issues faced in a multi-node k8s setup which often prevents them from running successfully.

Local Docker registry setup

Follow the steps in following link to setup Rancher and k8s cluster:   

https://onap.readthedocs.io/en/beijing/submodules/oom.git/docs/oom_setup_kubernetes_rancher.html

Create a new VM from a ubuntu 16.04 cloud image with 4GB RAM, 4 vCPUs, 80GB storage to host the docker registry

Add this VM to same network to which the Rancher and k8s nodes are attached.

Install docker on it

$ apt update

$ apt install docker.io

Create the self signed certificate:

$ mkdir -p certs

$ openssl req \
  -newkey rsa:4096 -nodes -sha256 -keyout certs/domain.key \
  -x509 -days 365 -out certs/domain.crt

Make sure to enter the correct hostname of the registry host as canonical name, rest all questions can be skipped. In this example I have kept the hostname of the registry VM as "registry"

Pull and run the registry pointing to public ONAP nexus proxy:

$ docker run -d \
 --restart=unless-stopped \
 --name registry \
 -v `pwd`/certs:/certs \
 -e REGISTRY_HTTP_ADDR=0.0.0.0:5000 \
 -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt \
 -e REGISTRY_HTTP_TLS_KEY=/certs/domain.key \
 -e REGISTRY_PROXY_REMOTEURL=https://nexus3.onap.org:10001 \
 -p 5000:5000 \
 registry:2

Registry is listening on 5000 port

On each VM (Rancher and all k8s nodes) add the "registry" hostname and IP address in /etc/hosts file

Also make following directory on each node

/etc/docker/certs.d/registry:5000 

copy domain.crt generated on the "registry" host to this directory and re-name it as ca.crt

While launching the pods using "helm install"  override the default repository settings as follows

$ helm install local/onap -n onap --namespace onap --set global.repository=registry:5000

This will point to the local registry and start pulling the images from there. The local registry will in turn pull the images from external nexus proxy on the first attempt which will be then cached locally for any subsequent image pulls. 

To see if the registry is getting populated with the images, invoke following on the registry VM: 

curl --cacert certs/domain.crt -X GET https://registry:5000/v2/_catalog