Versions Compared

Old Version 1

changes.mady.by.user Beili Zhou

Saved on

compared with

New Version Current

changes.mady.by.user Mohammadreza Pasandideh

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Info
titleWorking in progress

Although the exciting instructions on Steps described in this page are working for kubernetes, the cluster configuration part is still working in progress. Will be updated as soon as it becomes available.run by "ubuntu", a non-root user.


This page describes the steps of using rancher (rancher environment) to deploy Kubernetes node and config Kubernetes cluster nodes.


Table of Contents


Deploy The First Kubernetes Node

This section, similar as Quickstart Installation in ONAP on Kubernetes, provides the instruction of setting kubernestes deploying Kubernetes node through Rancher. The steps described on this page include using rancher to configure clustered kubernetes nodes.

Table of Contents

The first Kubernetes node in the Kubernetes cluster is to be deployed on the rancher server VM which also is configured with ranch client running.

Setup Rancher Server

...

Follow the steps below to install and configure Rancher:

#PurposeCommand and Example
1

Add host name to /etc/hosts file

Adding all cluster nodes to /etc/hosts file

1

on each node


sudo vi /etc/hosts

Expand
titleExample result of /etc/hosts file
ubuntu@sdnc-k8s:~$

$ more /etc/hosts

127.0.0.1 sdnc-k8s localhost

...

10.147.132.10 sdnc-k8s

10.147.90.6 sdnc-k8s-2

2 

Clone the OOM project from ONAP gerrit

git clone http://gerrit.onap.org/r/oom

3 

Install Docker 1.12 2

curl https://releases.rancher.com/install-docker/1.12.sh | sh

Expand
titleExample of docker installation

+ sudo -E sh -c docker version

Client:

 Version:      1.12.6

 API version:  1.24

 Go version:   go1.6.4

 Git commit:   78d1802

 Built:        Tue Jan 10 20:38:45 2017

 OS/Arch:      linux/amd64

Server:

 Version:      1.12.6

 API version:  1.24

 Go version:   go1.6.4

 Git commit:   78d1802

 Built:        Tue Jan 10 20:38:45 2017

 OS/Arch:      linux/amd64

If you would like to use Docker as a non-root user, you should now consider

adding your user to the "docker" group with something like:

  sudo usermod -aG docker ubuntu

Remember that you will have to log out and back in for this to take effect!

4

Install/Start Rancher server 3

Only on master node (if you are installing more than one Kubernetes nodes),
2

Install/Start Rancher server


There may be issues with the DNS pod in Rancher after a reboot or when running clustered hosts as described in ONAP bug:

A clean system is OK.

Use port 8880 instead of 8080.

docker run -d --restart=unless-stopped -p 8880:8080 rancher/server:v1.6.10

Expand
titleExample of rancher start in docker

$ docker run -d --restart=unless-stopped -p 8880:8080 rancher/server:v1.6.10

Unable to find image 'rancher/server:latest' locally

latest: Pulling from rancher/server

6599cadaf950: Pull complete

23eda618d451: Pull complete

f0be3084efe9: Pull complete

52de432f084b: Pull complete

a3ed95caeb02: Pull complete

e75cd91a1dc5: Pull complete

997f1b48f59f: Pull complete

313c28fb4e37: Pull complete

2a0730d1275c: Pull complete

8848fbebd2c8: Pull complete

906504ea9ea6: Pull complete

9329940f8e65: Pull complete

e849debd7945: Pull complete

4883bd135dd2: Pull complete

605c6a0fe940: Pull complete

274bc004c933: Pull complete

a6cb25e8d1a2: Pull complete

3ded9d4c8c2b: Pull complete

a571d2f40012: Pull complete

Digest: sha256:096026aa3eed411077690c904cd5dd3cce66eb39312c78a84209eab23387c36d

Status: Downloaded newer image for rancher/server:latest

4b3020cfccc2db47905e89e66cd8533fea82d60b886b2f4509e5f1c56764563a

Expand
titleExample of validate rancher server is up in docker
ubuntu@sdnc-k8s:~$

$ docker ps
CONTAINER

ID    IMAGE               COMMAND                  CREATED             STATUS              PORTS                              

ID IMAGE COMMAND CREATED STATUS PORTS NAMES

4b3020cfccc2       

e3f1c2ce3234 rancher/

server     

server:v1.6.10 "/usr/bin/entry /usr/"

   12 minutes ago      Up 12 minutes      

7 seconds ago Up 6 seconds 3306/tcp, 0.0.0.0:8880->8080/

tcp   backstabbing

tcp pensive_

wright

Notes:

  1. Adding all cluster nodes to /etc/hosts file on each node is optional
  2. Check out compartible versions among Rancher, Kubernetes and Docker
  3. There may be issues with the DNS pod in Rancher after a reboot or when running clustered hosts as described in ONAP bug OOM-236 (Rancher DNS pod non-functional after system reboot - only in cluster mode). A clean system is OK.

goldberg
$


Configure Rancher Client

You can access the Rancher UI through http:<master <master node host IP>:8880 ; it takes you to the apps/stacks page.

...

From here, follow the steps below to configure the Rancher:

#PurposeScript shots illustrated steps
1

Create Rancher environment:

  • Launch "Manage
enviroments
  • Enviroments" page
 Image Removed Image Removed
  • Image Added
  • Click on "Add Enviroment"
  • Image Added
  • Fill in the template with the following, and then click on the Create button
    • Name: "sdnc-ha"
    • (Optional) Add Description of the new environment
    • Environment Template: Kubernetes
  • Click on the Create button
    • Image Added 
    • Check that the new enviroment has been created and displays as "Active" 
     Image Removed
    • Image Added
    • Expand
      title
    (Optional) delete
    • Delete pre-existing Default environment
    Image Removed


    • Image Added

    Image Removed

    • Image Added


    2

    Add host to INFRASTRUCTURE

    • Go to the Hosts page via INFRASTRUCTURE -> Hosts
    • Click on the AddHost button
    Image Removed

    Image Added

    • For the first time only, the following confirmation page will appear (click the Save button to continue):
    Image Removed

    Image Added

    • The following Add Host page will appear:
    Image Removed

    Image Added

      • Skip steps 1, 2, and 3 listed in the page
      • In step 4, fill in the blank with the server IP
      • In step 5:

        • Click the Copy to Clipboard button to copy the command and run it on the server

          Expand
          titleExample of registering rancher agent on the server
    ubuntu@sdnc-k8s:~$

        • $ docker run -e CATTLE_AGENT_IP="10.147.132.10"  --rm --privileged -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher rancher/agent:v1.2.6http://10.147.132.10:8880/v1/scripts/774E9A523D82DBCBDE13:1483142400000:FPPmrMz8IurKQwVLV2UpfGR10AY

          Unable to find image 'rancher/agent:v1.2.6' locally

          v1.2.6: Pulling from rancher/agent

          b3e1c725a85f: Pull complete

          6a710864a9fc: Pull complete

          d0ac3b234321: Pull complete

          87f567b5cf58: Pull complete

          063e24b217c4: Pull complete

          d0a3f58caef0: Pull complete

          16914729cfd3: Pull complete

          2ce3828c0b9d: Pull complete

          7df47a98fc4b: Pull complete

          Digest: sha256:a68afd351c7417e6d66a77e97144113ceb7a9c3cdd46fb6e1fd5f5a5a33111cd

          Status: Downloaded newer image for rancher/agent:v1.2.6

           

          INFO: Running Agent Registration Process, CATTLE_URL=http://10.147.132.10:8880/v1

          INFO: Attempting to connect to: http://10.147.132.10:8880/v1

          INFO: http://10.147.132.10:8880/v1 is accessible

          INFO: Inspecting host capabilities

          INFO: Boot2Docker: false

          INFO: Host writable: true

          INFO: Token: xxxxxxxx

          INFO: Running registration

          INFO: Printing Environment

          INFO: ENV: CATTLE_ACCESS_KEY=0CB90B4247A136F6E43E

          INFO: ENV: CATTLE_AGENT_IP=10.147.132.10

          INFO: ENV: CATTLE_HOME=/var/lib/cattle

          INFO: ENV: CATTLE_REGISTRATION_ACCESS_KEY=registrationToken

          INFO: ENV: CATTLE_REGISTRATION_SECRET_KEY=xxxxxxx

          INFO: ENV: CATTLE_SECRET_KEY=xxxxxxx

          INFO: ENV: CATTLE_URL=http://10.147.132.10:8880/v1

          INFO: ENV: DETECTED_CATTLE_AGENT_IP=172.17.0.1

          INFO: ENV: RANCHER_AGENT_IMAGE=rancher/agent:v1.2.6

          INFO: Launched Rancher Agent: 4e3b1e4c2b4c33095ef550cbece626277b63783b012efe74ed2713df63cc925c

          Expand
          titleExample of validating rancher agent started
    ubuntu@sdnc-k8s:~$

        • $ docker ps
          CONTAINER

    ID        IMAGE                   COMMAND                  CREATED             STATUS              PORTS                               NAMES43eff054da6a       

        • ID IMAGE COMMAND CREATED STATUS PORTS NAMES
          64562118a30c rancher/etcd:v2.3.7-13 "/.r/r /opt/rancher/r" 1 seconds ago Up Less than a second r-kubernetes-etcd-1-b95cd471
          308b3fbf1882 rancher/net:holder "/.r/r /rancher-entry" 12 seconds ago Up 11 seconds r-ipsec-ipsec-1-9198fa8b
          523278d910f4 rancher/dns:v0.15.3 "/rancher-entrypoint." 12 seconds ago Up 11 seconds r-network-services-metadata-dns-1-2730cb6f
          8520cc50c5ff rancher/metadata:v0.9.4 "/rancher-entrypoint." 13 seconds ago Up 12 seconds r-network-services-metadata-1-53bf4aaf
          1f6dcb021f5b rancher/healthcheck:v0.3.3 "/.r/r /rancher-entry" 14 seconds ago Up 13 seconds r-healthcheck-healthcheck-1-3d14df58
          1d7ad141ceec rancher/network-manager:v0.7.8 "/rancher-entrypoint." 14 seconds ago Up 13 seconds r-network-services-network-manager-1-fdd38b68
          c7dfdde262d7 rancher/net:v0.11.

    9   

        • 9 "/rancher-entrypoint."

       5

        • 14 seconds

    ago       Up 4 seconds                                           

        • ago Up 13 seconds r-ipsec-cni-driver-1-

    4a5c6e16

        • 415b67fe

    4e3b1e4c2b4c       

        • 64e2f05b0394 rancher/agent:v1.2.

    6  

        • 6 "/run.sh run"

                27

        • 20 seconds

    ago      Up 27 seconds                                          

        • ago Up 19 seconds rancher-agent

    4b3020cfccc2        

        • e3f1c2ce3234 rancher/

    server         

        • server:v1.6.10 "/usr/bin/entry /usr/"

       46

        • 9 minutes

    ago     

        • ago Up

    46 minutes      

        • 9 minutes 3306/tcp, 0.0.0.0:8880->8080/

    tcp   backstabbing_wright

        • tcp pensive_goldberg
          $

      • Click on the Close button to close the Add Host page
    • Go to the Hosts page from INFRASTRUCTURE -> Hosts

      • You should see the newly added host:

        Expand
        titleExample of Hosts page with one node

    Image Removed

    Expand
    titleExample of Hosts page has 2 nodes
    Image Removed

      • Image Added



    3

    Confirm that the host process has completed

    The end of adding the host operation is indicated by the CLI option shown under the KUBERNETES dropdown menu:

    Expand
    titleExample of CLI option in KUBERNETES drop down

    Image Removed

    Install kubectl

    kubectl is a command line interface for running commands against Kubernetes clusters.

    Image Added


    Configure Kubernetes Cluster Nodes

    Repeat the following steps as many times as the number of nodes you are going to have in the Kubernetes cluster.Follow the steps below to install kubectrl:

    #PurposeCommand and Example
    1

    Download kubectl on the server

    Add host name to /etc/hosts file

    Adding all cluster nodes to /etc/hosts file on each node


    sudo vi /etc/hosts

    curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl

    Expand
    titleExample
    of download kubectl process

      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current

                                     Dload  Upload   Total   Spent    Left  Speed

    100 49.8M  100 49.8M    0     0  15.3M      0  0:00:03  0:00:03 --:--:-- 15.3M

    2

    Enable the kubectl command

    chmod +x ./kubectl

    sudo mv ./kubectl /usr/local/bin/kubectl

    3

    Generate Config from the Rancher UI

    result of /etc/hosts file

    $ more /etc/hosts

    127.0.0.1 sdnc-k8s-2 localhost

    ...

    10.147.132.10 sdnc-k8s

    10.147.90.6 sdnc-k8s-2

    2

    Add host to INFRASTRUCTURE

    See examples from step 2 of section "Configure Rancher Client"

    • Go to the Hosts page via INFRASTRUCTURE -> Hosts
    • Click on the AddHost button
    • In Add Host page:
      • Skip steps 1, 2, and 3 listed in the page
      • In step 4, fill in the blank with the server IP
      • In step 5:

        • Click

    • From the Rancher UI, go to a Kubernetes CLI page through KUBERNETES -> CLI
    • Click on the Generate Config button to generate the config

    Image Removed

    The following content will be brought up; click on

        • the Copy to Clipboard button to copy the

    generated config

        • command and

    use it in next step:

    Image Removed

    4Create the kube config

        • run it on the server

    mkdir ~/.kube

    vi ~/.kube/config

    Paste the generated config from rancher UI (from last step) to this file, then save the file.

    5

    Validate the kube config

    kubectl cluster-info

    Expand
    titleExample of validation result on kube config through "kubectl cluster-info"

    Kubernetes master is running at https://10.147.132.10:8880/r/projects/1a8/kubernetes:6443

    Heapster is running at https://10.147.132.10:8880/r/projects/1a8/kubernetes:6443/api/v1/namespaces/kube-system/services/heapster/proxy

    KubeDNS is running at https://10.147.132.10:8880/r/projects/1a8/kubernetes:6443/api/v1/namespaces/kube-system/services/kube-dns/proxy

    kubernetes-dashboard is running at https://10.147.132.10:8880/r/projects/1a8/kubernetes:6443/api/v1/namespaces/kube-system/services/kubernetes-dashboard/proxy

    monitoring-grafana is running at https://10.147.132.10:8880/r/projects/1a8/kubernetes:6443/api/v1/namespaces/kube-system/services/monitoring-grafana/proxy

    monitoring-influxdb is running at https://10.147.132.10:8880/r/projects/1a8/kubernetes:6443/api/v1/namespaces/kube-system/services/monitoring-influxdb/proxy

    tiller-deploy is running at https://10.147.132.10:8880/r/projects/1a8/kubernetes:6443/api/v1/namespaces/kube-system/services/tiller-deploy/proxy

    To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

    Configure Kubernetes Cluster

    ...

      • Click on the Close button to close the Add Host page
    • Go to the Hosts page from INFRASTRUCTURE -> Hosts

      • You should see the newly added host:

        Image Added


    3

    Confirm that the Setup has Finished

    		Wait until all hosts in Rancher show green, indicating that setup is complete!