Create VMs

Create two VMs - master and worker with below config -

Configure Kubernetes Cluster

Follow steps from below link to install a Kubernetes cluster using kubeadm, comprising just two VMs created in above step -

https://wiki.onap.org/display/DW/Deploying+Kubernetes+Cluster+with+kubeadm

Verify below pods up and running after completing all steps -

Note that there is no kube-dns pod. Instead a coredns pod is created.

Modify coredns deployment to configure DNS zone -

1. Put attached files - Corefile and zone.db in /dockerdata-nfs using root user.

   ubuntu@kubefed-2:/dockerdata-nfs$ ls -lrt
   total 44
   -rw-r--r-- 1 root root  1366 Mar  1 15:00 Corefile
   -rw-r--r-- 1 root root   979 Mar  5 14:06 zone.db
   ubuntu@kubefed-2:/dockerdata-nfs$
   
   

2. Edit coredns configmap to reflect the new zone file and adding the key.

   #below command opens the file for editing. Edit and save the file to get the "configmap coredns edited" response.
   ubuntu@kubefed-1:~# kubectl edit configmap coredns -n kube-system -oyaml
   # Please edit the object below. Lines beginning with a '#' will be ignored,
   # and an empty file will abort the edit. If an error occurs while saving this file will be
   # reopened with the relevant failures.
   #
   apiVersion: v1
   data:
     Corefile: |
       .:53 {
           errors
           log
           health
           kubernetes cluster.local 10.96.0.0/12 {
              pods insecure
           }
           file /dockerdata-nfs/zone.db example.com
           prometheus
           proxy . /etc/resolv.conf
           cache 30
       }
     zone.db: "$ORIGIN example.com.     ; designates the start of this zone file in the
       namespace\n$TTL 1h         ; default expiration time of all resource records without
       their own TTL value\nexample.com.  IN  SOA   ns.example.com. username.example.com.
       ( 2007120710 1d 2h 4w 1h )\nexample.com.  IN  NS    ns                    ; ns.example.com
       is a nameserver for example.com\nexample.com.  IN  NS    ns.somewhere.example.
       ; ns.somewhere.example is a backup nameserver for example.com\nexample.com.  IN
       \ A     10.147.101.135             ; IPv4 address for example.com\nns            IN
       \ A     10.247.5.11             ; IPv4 address for ns.example.com\nwww           IN
       \ CNAME example.com.          ; www.example.com is an alias for example.com\nwwwtest
       \      IN  CNAME www              ; wwwtest.example.com is another alias for www.example.com\nsdnc.example.com.
       \   IN      SRV    30202 10 10 example.com.\n;;site1\n;;sdnc         IN  A  10.147.99.140\n;;site2\nsdnc\t\t
       IN A   10.147.101.23"
   kind: ConfigMap
   metadata:
     creationTimestamp: 2018-02-28T20:13:03Z
     name: coredns
     namespace: kube-system
     resourceVersion: "102077"
     selfLink: /api/v1/namespaces/kube-system/configmaps/coredns
     uid: c8489771-1cc3-11e8-a0cb-fa163eabcb60
    
   configmap "coredns" edited
   ubuntu@kubefed-1:~#

   Make following changes:

   Where	Added Line
   Corefile: | .:53{ ... }	file /dockerdata-nfs/zone.db example.com
   Before line: kind: ConfigMap Note that IP addresses should point to your environment. Only one of the site A record should be uncommented at a time. example.com. IN A <fed-vm master ip address> ns IN A <nameserver on fed VM as configured originally in /etc/resolv.conf> ;;site1 ;;sdnc IN A <site1 master IP> //note that site1 is commented here ;;site2 sdnc IN A <site2 master IP>	zone.db: "$ORIGIN example.com. ; designates the start of this zone file in the namespace\n$TTL 1h ; default expiration time of all resource records without their own TTL value\nexample.com. IN SOA ns.example.com. username.example.com. ( 2007120710 1d 2h 4w 1h )\nexample.com. IN NS ns ; ns.example.com is a nameserver for example.com\nexample.com. IN NS ns.somewhere.example. ; ns.somewhere.example is a backup nameserver for example.com\nexample.com. IN \ A 10.147.101.135 ; IPv4 address for example.com\nns IN \ A 10.247.5.11 ; IPv4 address for ns.example.com\nwww IN \ CNAME example.com. ; www.example.com is an alias for example.com\nwwwtest \ IN CNAME www ; wwwtest.example.com is another alias for www.example.com\nsdnc.example.com. \ IN SRV 30202 10 10 example.com.\n;;site1\n;;sdnc IN A 10.147.99.140\n;;site2\nsdnc\t\t IN A 10.147.101.23"

   The files copied in Step 1 should have matching entries as described in Step 2. That is, the IP addresses should point to your environment as explained in step 2. Make sure to have the same site as commented, as in Step 2.

3. Edit coredns deployment to specify mount volumes and zone key for coredns.

   #use below command to start editing and make changes as mentioned in the table.
   ubuntu@kubefed-1:~# kubectl -n kube-system edit deployment coredns
   deployment "coredns" edited
   ubuntu@kubefed-1:~#

   do the following changes:
   Purpose	Changes
   mount point for new zone file and coredns config file ; defining new zone key in volumes.	Field Added/Replaced Value .spec.template.spec.containers.volumeMounts .spec.template.spec.containers.args Replaced Values: - mountPath: /dockerdata-nfs name: config-volume - conf - /dockerdata-nfs/Corefile  .spec.template.spec.volumes.items Added Values: - key: zone.db path: zone.db

   Deployment config will look as in file coredns-edit-deploymnt.yaml after making below changes.
   Note that editing the deployment will create a new coredns pod and terminate the previous one. Verify the new pod is up and running after editing deployment.

   ubuntu@kubefed-2:/root$ kubectl get pods --all-namespaces --context=kubernetes-admin-host | grep core
   kube-system         coredns-789ff8fd8d-2gxnc                        1/1       Running   0          5d
   ubuntu@kubefed-2:/root$

4. Edit /etc/resolv.conf to point to configured coreDNS server.

   ubuntu@kubefed-1:~# kubectl get svc --all-namespaces --context=kubernetes-admin-host | grep kube-dns
   kube-system         kube-dns               ClusterIP   10.96.0.10       <none>        53/UDP,53/TCP   5d
    
   #Use the above cluster IP to configure in /etc/resolv.conf as shown below
   ubuntu@kubefed-2:/root$ sudo vi /etc/resolv.conf
   # Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
   #     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
   nameserver 10.96.0.10
   #nameserver 10.247.5.11
   search openstacklocal
   ubuntu@kubefed-2:
    

5.  Lookup the configured domain to verify it is pointing to site as configured above.

   ubuntu@kubefed-2:/dockerdata-nfs$ nslookup sdnc.example.com
   Server:         10.96.0.10
   Address:        10.96.0.10#53
   Name:   sdnc.example.com
   Address: 10.147.101.23
    
   #verify it is resolving sites like google.ca backwardly
   ubuntu@kubefed-2:/dockerdata-nfs$ nslookup google.ca
   Server:         10.96.0.10
   Address:        10.96.0.10#53
   Non-authoritative answer:
   Name:   google.ca
   Address: 172.217.23.131