Tracking OOM-1598 - Getting issue details... STATUS
WIP - as of 20190225
https://gerrit.onap.org/r/#/c/79067/
Move to https://onap.readthedocs.io/en/beijing/submodules/oom.git/docs/oom_cloud_setup_guide.html or similar when this documentation is released
Automated Installation Video of RKE install
20190227
20190228_rke_install_preliminary.mp4
20190227_rke_016_preliminary_1click_install_oom-1598-6min.mp4
Versions
Currently Docker 18.06, RKE 0.16, Kubernetes 1.11.6, Kubectl 1.11.6, Helm 2.9.1
TODO: verify later versions of helm and a way to get RKE to install Kubernetes 1.13
Prerequisites
Ubuntu 16.04 VM
Determine RKE and Docker versions
Don't just use the latest docker version - check the RKE release page to get the version pair - 0.15/17.03 and 0.16/18.06 - see https://github.com/docker/docker-ce/releases - currently https://github.com/docker/docker-ce/releases/tag/v18.06.3-ce
ubuntu@a-rke:~$ sudo curl https://releases.rancher.com/install-docker/18.06.sh | sh ubuntu@a-rke:~$ sudo usermod -aG docker ubuntu ubuntu@a-rke:~$ sudo docker version Client: Version: 18.06.3-ce API version: 1.38 Go version: go1.10.3 Git commit: d7080c1 Built: Wed Feb 20 02:27:18 2019 # install RKE sudo wget https://github.com/rancher/rke/releases/download/v0.1.16/rke_linux-amd64 mv rke_linux-amd64 rke sudo mv ./rke /usr/local/bin/rke ubuntu@a-rke:~$ rke --version rke version v0.1.16
Private SSH key
scp your key to the box - ideally to ~/.ssh and chmod 400 it
Elastic Reserved IP
get a VIP or EIP
generate cluster.yaml
azure config - no need to hand build the yaml
Watch the path of your 2 keys
Also don't add an "addon" until you have one of the config job will fail
{noformat}
ubuntu@a-rke:~$ rke config --name cluster.yml
[+] Cluster Level SSH Private Key Path [~/.ssh/id_rsa]: ~/.ssh/onap_rsa
[+] Number of Hosts [1]:
[+] SSH Address of host (1) [none]: rke.onap.cloud
[+] SSH Port of host (1) [22]:
[+] SSH Private Key Path of host (rke.onap.cloud) [none]: ~/.ssh/onap_rsa
[+] SSH User of host (rke.onap.cloud) [ubuntu]:
[+] Is host (rke.onap.cloud) a Control Plane host (y/n)? [y]: y
[+] Is host (rke.onap.cloud) a Worker host (y/n)? [n]: y
[+] Is host (rke.onap.cloud) an etcd host (y/n)? [n]: y
[+] Override Hostname of host (rke.onap.cloud) [none]:
[+] Internal IP of host (rke.onap.cloud) [none]:
[+] Docker socket path on host (rke.onap.cloud) [/var/run/docker.sock]:
[+] Network Plugin Type (flannel, calico, weave, canal) [canal]:
[+] Authentication Strategy [x509]:
[+] Authorization Mode (rbac, none) [rbac]:
[+] Kubernetes Docker image [rancher/hyperkube:v1.11.6-rancher1]:
[+] Cluster domain [cluster.local]:
[+] Service Cluster IP Range [10.43.0.0/16]:
[+] Enable PodSecurityPolicy [n]:
[+] Cluster Network CIDR [10.42.0.0/16]:
[+] Cluster DNS Service IP [10.43.0.10]:
[+] Add addon manifest URLs or YAML files [no]: no
ubuntu@a-rke:~$ sudo cat cluster.yml
# If you intened to deploy Kubernetes in an air-gapped environment,
# please consult the documentation on how to configure custom RKE images.
nodes:
- address: rke.onap.cloud
port: "22"
internal_address: ""
role:
- controlplane
- worker
- etcd
hostname_override: ""
user: ubuntu
docker_socket: /var/run/docker.sock
ssh_key: ""
ssh_key_path: ~/.ssh/onap_rsa
labels: {}
services:
etcd:
image: ""
extra_args: {}
extra_binds: []
extra_env: []
external_urls: []
ca_cert: ""
cert: ""
key: ""
path: ""
snapshot: null
retention: ""
creation: ""
kube-api:
image: ""
extra_args: {}
extra_binds: []
extra_env: []
service_cluster_ip_range: 10.43.0.0/16
service_node_port_range: ""
pod_security_policy: false
kube-controller:
image: ""
extra_args: {}
extra_binds: []
extra_env: []
cluster_cidr: 10.42.0.0/16
service_cluster_ip_range: 10.43.0.0/16
scheduler:
image: ""
extra_args: {}
extra_binds: []
extra_env: []
kubelet:
image: ""
extra_args: {}
extra_binds: []
extra_env: []
cluster_domain: cluster.local
infra_container_image: ""
cluster_dns_server: 10.43.0.10
fail_swap_on: false
kubeproxy:
image: ""
extra_args: {}
extra_binds: []
extra_env: []
network:
plugin: canal
options: {}
authentication:
strategy: x509
options: {}
sans: []
system_images:
etcd: rancher/coreos-etcd:v3.2.18
alpine: rancher/rke-tools:v0.1.15
nginx_proxy: rancher/rke-tools:v0.1.15
cert_downloader: rancher/rke-tools:v0.1.15
kubernetes_services_sidecar: rancher/rke-tools:v0.1.15
kubedns: rancher/k8s-dns-kube-dns-amd64:1.14.10
dnsmasq: rancher/k8s-dns-dnsmasq-nanny-amd64:1.14.10
kubedns_sidecar: rancher/k8s-dns-sidecar-amd64:1.14.10
kubedns_autoscaler: rancher/cluster-proportional-autoscaler-amd64:1.0.0
kubernetes: rancher/hyperkube:v1.11.6-rancher1
flannel: rancher/coreos-flannel:v0.10.0
flannel_cni: rancher/coreos-flannel-cni:v0.3.0
calico_node: rancher/calico-node:v3.1.3
calico_cni: rancher/calico-cni:v3.1.3
calico_controllers: ""
calico_ctl: rancher/calico-ctl:v2.0.0
canal_node: rancher/calico-node:v3.1.3
canal_cni: rancher/calico-cni:v3.1.3
canal_flannel: rancher/coreos-flannel:v0.10.0
wave_node: weaveworks/weave-kube:2.1.2
weave_cni: weaveworks/weave-npc:2.1.2
pod_infra_container: rancher/pause-amd64:3.1
ingress: rancher/nginx-ingress-controller:0.16.2-rancher1
ingress_backend: rancher/nginx-ingress-controller-defaultbackend:1.4
metrics_server: rancher/metrics-server-amd64:v0.2.1
ssh_key_path: ~/.ssh/onap_rsa
ssh_agent_auth: false
authorization:
mode: rbac
options: {}
ignore_docker_version: false
kubernetes_version: ""
private_registries: []
ingress:
provider: ""
options: {}
node_selector: {}
extra_args: {}
cluster_name: ""
cloud_provider:
name: ""
prefix_path: ""
addon_job_timeout: 0
bastion_host:
address: ""
port: ""
user: ""
ssh_key: ""
ssh_key_path: ""
monitoring:
provider: ""
options: {}
{noformat}
Kubernetes Single Node Developer Installation
sudo chmod 777 cluster.yml rke up # install kubectl sudo curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.11.6/bin/linux/amd64/kubectl sudo chmod +x ./kubectl sudo mv ./kubectl /usr/local/bin/kubectl sudo mkdir ~/.kube sudo cp kube_config_cluster.yml ~/.kube/config sudo chmod 777 ~/.kube/config kubectl get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE ingress-nginx default-http-backend-797c5bc547-45msr 1/1 Running 0 17m ingress-nginx nginx-ingress-controller-dfhp8 1/1 Running 0 17m kube-system canal-lc6g6 3/3 Running 0 17m kube-system kube-dns-7588d5b5f5-6k286 3/3 Running 0 17m kube-system kube-dns-autoscaler-5db9bbb766-6slz7 1/1 Running 0 17m kube-system metrics-server-97bc649d5-q84tz 1/1 Running 0 17m kube-system rke-ingress-controller-deploy-job-5q2w7 0/1 Completed 0 17m kube-system rke-kubedns-addon-deploy-job-7vq49 0/1 Completed 0 17m kube-system rke-metrics-addon-deploy-job-2hnbl 0/1 Completed 0 17m kube-system rke-network-plugin-deploy-job-6fzt2 0/1 Completed 0 17m
Kubernetes HA Cluster Production Installation
Design Issues
DI 20190225-1: RKE/Docker version pair
As of 20190215 RKE 0.16 supports Docker 18.06-ce (and 18.09 non-ce) (up from 0.15 supporting 17.03)
https://github.com/docker/docker-ce/releases/tag/v18.06.3-ce
https://github.com/rancher/rke/releases/tag/v0.1.16
ubuntu@a-rke:~$ sudo rke up INFO[0000] Building Kubernetes cluster INFO[0000] [dialer] Setup tunnel for host [rke.onap.cloud] FATA[0000] Unsupported Docker version found [18.06.3-ce], supported versions are [1.11.x 1.12.x 1.13.x 17.03.x]
DI 20190225-2: RKE upgrade from 0.15 to 0.16 - not working
Does rke remove, regenerate the yaml (or hand upgrade the versions) then rke up
ubuntu@a-rke:~$ sudo rke remove Are you sure you want to remove Kubernetes cluster [y/n]: y INFO[0002] Tearing down Kubernetes cluster INFO[0002] [dialer] Setup tunnel for host [rke.onap.cloud] INFO[0002] [worker] Tearing down Worker Plane.. INFO[0002] [remove/kubelet] Successfully removed container on host [rke.onap.cloud] INFO[0003] [remove/kube-proxy] Successfully removed container on host [rke.onap.cloud] INFO[0003] [remove/service-sidekick] Successfully removed container on host [rke.onap.cloud] INFO[0003] [worker] Successfully tore down Worker Plane.. INFO[0003] [controlplane] Tearing down the Controller Plane.. INFO[0003] [remove/kube-apiserver] Successfully removed container on host [rke.onap.cloud] INFO[0003] [remove/kube-controller-manager] Successfully removed container on host [rke.onap.cloud] INFO[0004] [remove/kube-scheduler] Successfully removed container on host [rke.onap.cloud] INFO[0004] [controlplane] Host [rke.onap.cloud] is already a worker host, skipping delete kubelet and kubeproxy. INFO[0004] [controlplane] Successfully tore down Controller Plane.. INFO[0004] [etcd] Tearing down etcd plane.. INFO[0004] [remove/etcd] Successfully removed container on host [rke.onap.cloud] INFO[0004] [etcd] Successfully tore down etcd plane.. INFO[0004] [hosts] Cleaning up host [rke.onap.cloud] INFO[0004] [hosts] Cleaning up host [rke.onap.cloud] INFO[0004] [hosts] Running cleaner container on host [rke.onap.cloud] INFO[0005] [kube-cleaner] Successfully started [kube-cleaner] container on host [rke.onap.cloud] INFO[0005] [hosts] Removing cleaner container on host [rke.onap.cloud] INFO[0005] [hosts] Removing dead container logs on host [rke.onap.cloud] INFO[0006] [cleanup] Successfully started [rke-log-cleaner] container on host [rke.onap.cloud] INFO[0006] [remove/rke-log-cleaner] Successfully removed container on host [rke.onap.cloud] INFO[0006] [hosts] Successfully cleaned up host [rke.onap.cloud] INFO[0006] [hosts] Cleaning up host [rke.onap.cloud] INFO[0006] [hosts] Cleaning up host [rke.onap.cloud] INFO[0006] [hosts] Running cleaner container on host [rke.onap.cloud] INFO[0007] [kube-cleaner] Successfully started [kube-cleaner] container on host [rke.onap.cloud] INFO[0008] [hosts] Removing cleaner container on host [rke.onap.cloud] INFO[0008] [hosts] Removing dead container logs on host [rke.onap.cloud] INFO[0008] [cleanup] Successfully started [rke-log-cleaner] container on host [rke.onap.cloud] INFO[0009] [remove/rke-log-cleaner] Successfully removed container on host [rke.onap.cloud] INFO[0009] [hosts] Successfully cleaned up host [rke.onap.cloud] INFO[0009] [hosts] Cleaning up host [rke.onap.cloud] INFO[0009] [hosts] Cleaning up host [rke.onap.cloud] INFO[0009] [hosts] Running cleaner container on host [rke.onap.cloud] INFO[0010] [kube-cleaner] Successfully started [kube-cleaner] container on host [rke.onap.cloud] INFO[0010] [hosts] Removing cleaner container on host [rke.onap.cloud] INFO[0010] [hosts] Removing dead container logs on host [rke.onap.cloud] INFO[0011] [cleanup] Successfully started [rke-log-cleaner] container on host [rke.onap.cloud] INFO[0011] [remove/rke-log-cleaner] Successfully removed container on host [rke.onap.cloud] INFO[0011] [hosts] Successfully cleaned up host [rke.onap.cloud] INFO[0011] Removing local admin Kubeconfig: ./kube_config_cluster.yml INFO[0011] Local admin Kubeconfig removed successfully INFO[0011] Cluster removed successfully ubuntu@a-rke:~$ rke config --name cluster.ym ubuntu@a-rke:~$ sudo rke up INFO[0000] Building Kubernetes cluster INFO[0000] [dialer] Setup tunnel for host [rke.onap.cloud] INFO[0000] [network] Deploying port listener containers INFO[0001] [network] Successfully started [rke-etcd-port-listener] container on host [rke.onap.cloud] INFO[0001] [network] Successfully started [rke-cp-port-listener] container on host [rke.onap.cloud] INFO[0002] [network] Successfully started [rke-worker-port-listener] container on host [rke.onap.cloud] INFO[0002] [network] Port listener containers deployed successfully INFO[0002] [network] Running control plane -> etcd port checks INFO[0003] [network] Successfully started [rke-port-checker] container on host [rke.onap.cloud] INFO[0003] [network] Running control plane -> worker port checks INFO[0004] [network] Successfully started [rke-port-checker] container on host [rke.onap.cloud] INFO[0004] [network] Running workers -> control plane port checks INFO[0005] [network] Successfully started [rke-port-checker] container on host [rke.onap.cloud] INFO[0005] [network] Checking KubeAPI port Control Plane hosts INFO[0005] [network] Removing port listener containers INFO[0005] [remove/rke-etcd-port-listener] Successfully removed container on host [rke.onap.cloud] INFO[0006] [remove/rke-cp-port-listener] Successfully removed container on host [rke.onap.cloud] INFO[0006] [remove/rke-worker-port-listener] Successfully removed container on host [rke.onap.cloud] INFO[0006] [network] Port listener containers removed successfully INFO[0006] [certificates] Attempting to recover certificates from backup on [etcd,controlPlane] hosts INFO[0007] [certificates] No Certificate backup found on [etcd,controlPlane] hosts INFO[0007] [certificates] Generating CA kubernetes certificates INFO[0007] [certificates] Generating Kubernetes API server certficates INFO[0008] [certificates] Generating Kube Controller certificates INFO[0008] [certificates] Generating Kube Scheduler certificates INFO[0008] [certificates] Generating Kube Proxy certificates INFO[0009] [certificates] Generating Node certificate INFO[0009] [certificates] Generating admin certificates and kubeconfig INFO[0009] [certificates] Generating etcd-rke.onap.cloud certificate and key INFO[0009] [certificates] Generating Kubernetes API server aggregation layer requestheader client CA certificates INFO[0009] [certificates] Generating Kubernetes API server proxy client certificates INFO[0010] [certificates] Temporarily saving certs to [etcd,controlPlane] hosts INFO[0016] [certificates] Saved certs to [etcd,controlPlane] hosts INFO[0016] [reconcile] Reconciling cluster state INFO[0016] [reconcile] This is newly generated cluster INFO[0016] [certificates] Deploying kubernetes certificates to Cluster nodes INFO[0022] Successfully Deployed local admin kubeconfig at [./kube_config_cluster.yml] INFO[0022] [certificates] Successfully deployed kubernetes certificates to Cluster nodes INFO[0022] Pre-pulling kubernetes images INFO[0022] Kubernetes images pulled successfully INFO[0022] [etcd] Building up etcd plane.. INFO[0023] [etcd] Successfully started [etcd] container on host [rke.onap.cloud] INFO[0023] [etcd] Saving snapshot [etcd-rolling-snapshots] on host [rke.onap.cloud] INFO[0028] [certificates] Successfully started [rke-bundle-cert] container on host [rke.onap.cloud] INFO[0029] [certificates] successfully saved certificate bundle [/opt/rke/etcd-snapshots//pki.bundle.tar.gz] on host [rke.onap.cloud] INFO[0029] [etcd] Successfully started [rke-log-linker] container on host [rke.onap.cloud] INFO[0030] [remove/rke-log-linker] Successfully removed container on host [rke.onap.cloud] INFO[0030] [etcd] Successfully started etcd plane.. INFO[0030] [controlplane] Building up Controller Plane.. INFO[0031] [controlplane] Successfully started [kube-apiserver] container on host [rke.onap.cloud] INFO[0031] [healthcheck] Start Healthcheck on service [kube-apiserver] on host [rke.onap.cloud] INFO[0045] [healthcheck] service [kube-apiserver] on host [rke.onap.cloud] is healthy INFO[0046] [controlplane] Successfully started [rke-log-linker] container on host [rke.onap.cloud] INFO[0046] [remove/rke-log-linker] Successfully removed container on host [rke.onap.cloud] INFO[0047] [controlplane] Successfully started [kube-controller-manager] container on host [rke.onap.cloud] INFO[0047] [healthcheck] Start Healthcheck on service [kube-controller-manager] on host [rke.onap.cloud] INFO[0052] [healthcheck] service [kube-controller-manager] on host [rke.onap.cloud] is healthy INFO[0053] [controlplane] Successfully started [rke-log-linker] container on host [rke.onap.cloud] INFO[0053] [remove/rke-log-linker] Successfully removed container on host [rke.onap.cloud] INFO[0054] [controlplane] Successfully started [kube-scheduler] container on host [rke.onap.cloud] INFO[0054] [healthcheck] Start Healthcheck on service [kube-scheduler] on host [rke.onap.cloud] INFO[0059] [healthcheck] service [kube-scheduler] on host [rke.onap.cloud] is healthy INFO[0060] [controlplane] Successfully started [rke-log-linker] container on host [rke.onap.cloud] INFO[0060] [remove/rke-log-linker] Successfully removed container on host [rke.onap.cloud] INFO[0060] [controlplane] Successfully started Controller Plane.. INFO[0060] [authz] Creating rke-job-deployer ServiceAccount INFO[0060] [authz] rke-job-deployer ServiceAccount created successfully INFO[0060] [authz] Creating system:node ClusterRoleBinding INFO[0060] [authz] system:node ClusterRoleBinding created successfully INFO[0060] [certificates] Save kubernetes certificates as secrets INFO[0060] [certificates] Successfully saved certificates as kubernetes secret [k8s-certs] INFO[0060] [state] Saving cluster state to Kubernetes INFO[0061] [state] Successfully Saved cluster state to Kubernetes ConfigMap: cluster-state INFO[0061] [state] Saving cluster state to cluster nodes INFO[0061] [state] Successfully started [cluster-state-deployer] container on host [rke.onap.cloud] INFO[0062] [remove/cluster-state-deployer] Successfully removed container on host [rke.onap.cloud] INFO[0062] [worker] Building up Worker Plane.. INFO[0062] [remove/service-sidekick] Successfully removed container on host [rke.onap.cloud] INFO[0063] [worker] Successfully started [kubelet] container on host [rke.onap.cloud] INFO[0063] [healthcheck] Start Healthcheck on service [kubelet] on host [rke.onap.cloud] INFO[0068] [healthcheck] service [kubelet] on host [rke.onap.cloud] is healthy INFO[0069] [worker] Successfully started [rke-log-linker] container on host [rke.onap.cloud] INFO[0070] [remove/rke-log-linker] Successfully removed container on host [rke.onap.cloud] INFO[0070] [worker] Successfully started [kube-proxy] container on host [rke.onap.cloud] INFO[0070] [healthcheck] Start Healthcheck on service [kube-proxy] on host [rke.onap.cloud] INFO[0076] [healthcheck] service [kube-proxy] on host [rke.onap.cloud] is healthy INFO[0076] [worker] Successfully started [rke-log-linker] container on host [rke.onap.cloud] INFO[0077] [remove/rke-log-linker] Successfully removed container on host [rke.onap.cloud] INFO[0077] [worker] Successfully started Worker Plane.. INFO[0077] [sync] Syncing nodes Labels and Taints INFO[0077] [sync] Successfully synced nodes Labels and Taints INFO[0077] [network] Setting up network plugin: canal INFO[0077] [addons] Saving addon ConfigMap to Kubernetes INFO[0077] [addons] Successfully Saved addon to Kubernetes ConfigMap: rke-network-plugin INFO[0077] [addons] Executing deploy job.. INFO[0082] [addons] Setting up KubeDNS INFO[0082] [addons] Saving addon ConfigMap to Kubernetes INFO[0082] [addons] Successfully Saved addon to Kubernetes ConfigMap: rke-kubedns-addon INFO[0082] [addons] Executing deploy job.. INFO[0087] [addons] KubeDNS deployed successfully.. INFO[0087] [addons] Setting up Metrics Server INFO[0087] [addons] Saving addon ConfigMap to Kubernetes INFO[0087] [addons] Successfully Saved addon to Kubernetes ConfigMap: rke-metrics-addon INFO[0087] [addons] Executing deploy job.. INFO[0092] [addons] KubeDNS deployed successfully.. INFO[0092] [ingress] Setting up nginx ingress controller INFO[0092] [addons] Saving addon ConfigMap to Kubernetes INFO[0092] [addons] Successfully Saved addon to Kubernetes ConfigMap: rke-ingress-controller INFO[0092] [addons] Executing deploy job.. INFO[0097] [ingress] ingress controller nginx is successfully deployed INFO[0097] [addons] Setting up user addons INFO[0097] [addons] Checking for included user addons WARN[0097] [addons] Unable to determine if is a file path or url, skipping INFO[0097] [addons] Deploying rke-user-includes-addons INFO[0097] [addons] Saving addon ConfigMap to Kubernetes INFO[0097] [addons] Successfully Saved addon to Kubernetes ConfigMap: rke-user-includes-addons INFO[0097] [addons] Executing deploy job.. WARN[0128] Failed to deploy addon execute job [rke-user-includes-addons]: Failed to get job complete status: <nil> INFO[0128] Finished building Kubernetes cluster successfully ubuntu@a-rke:~$ sudo docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ec26c4bd24b5 846921f0fe0e "/server" 10 minutes ago Up 10 minutes k8s_default-http-backend_default-http-backend-797c5bc547-45msr_ingress-nginx_0eddfe19-394e-11e9-b708-000d3a0e23f3_0 f8d5db205e14 8a7739f672b4 "/sidecar --v=2 --lo…" 10 minutes ago Up 10 minutes k8s_sidecar_kube-dns-7588d5b5f5-6k286_kube-system_08c13783-394e-11e9-b708-000d3a0e23f3_0 490461545ae4 rancher/metrics-server-amd64 "/metrics-server --s…" 10 minutes ago Up 10 minutes k8s_metrics-server_metrics-server-97bc649d5-q84tz_kube-system_0c566ec8-394e-11e9-b708-000d3a0e23f3_0 aaf03b62bd41 6816817d9dce "/dnsmasq-nanny -v=2…" 10 minutes ago Up 10 minutes k8s_dnsmasq_kube-dns-7588d5b5f5-6k286_kube-system_08c13783-394e-11e9-b708-000d3a0e23f3_0 58ec007db72f 55ffe31ac578 "/kube-dns --domain=…" 10 minutes ago Up 10 minutes k8s_kubedns_kube-dns-7588d5b5f5-6k286_kube-system_08c13783-394e-11e9-b708-000d3a0e23f3_0 0a95c06f6aa6 e183460c484d "/cluster-proportion…" 10 minutes ago Up 10 minutes k8s_autoscaler_kube-dns-autoscaler-5db9bbb766-6slz7_kube-system_08b5495c-394e-11e9-b708-000d3a0e23f3_0 968a7c99b210 rancher/pause-amd64:3.1 "/pause" 10 minutes ago Up 10 minutes k8s_POD_default-http-backend-797c5bc547-45msr_ingress-nginx_0eddfe19-394e-11e9-b708-000d3a0e23f3_0 69969b331e49 rancher/pause-amd64:3.1 "/pause" 10 minutes ago Up 10 minutes k8s_POD_metrics-server-97bc649d5-q84tz_kube-system_0c566ec8-394e-11e9-b708-000d3a0e23f3_0 baa5f03c16ff rancher/pause-amd64:3.1 "/pause" 10 minutes ago Up 10 minutes k8s_POD_kube-dns-7588d5b5f5-6k286_kube-system_08c13783-394e-11e9-b708-000d3a0e23f3_0 82b2a9f640cb rancher/pause-amd64:3.1 "/pause" 10 minutes ago Up 10 minutes k8s_POD_kube-dns-autoscaler-5db9bbb766-6slz7_kube-system_08b5495c-394e-11e9-b708-000d3a0e23f3_0 953a4d4be0c1 df4469c42185 "/usr/bin/dumb-init …" 10 minutes ago Up 10 minutes k8s_nginx-ingress-controller_nginx-ingress-controller-dfhp8_ingress-nginx_0ed3bdbf-394e-11e9-b708-000d3a0e23f3_0 cce552840749 rancher/pause-amd64:3.1 "/pause" 10 minutes ago Up 10 minutes k8s_POD_nginx-ingress-controller-dfhp8_ingress-nginx_0ed3bdbf-394e-11e9-b708-000d3a0e23f3_0 baa65f9c6f97 f0fad859c909 "/opt/bin/flanneld -…" 10 minutes ago Up 10 minutes k8s_kube-flannel_canal-lc6g6_kube-system_05904de9-394e-11e9-b708-000d3a0e23f3_0 1736ce68f41a 9f355e076ea7 "/install-cni.sh" 10 minutes ago Up 10 minutes k8s_install-cni_canal-lc6g6_kube-system_05904de9-394e-11e9-b708-000d3a0e23f3_0 615d3f702ee7 7eca10056c8e "start_runit" 10 minutes ago Up 10 minutes k8s_calico-node_canal-lc6g6_kube-system_05904de9-394e-11e9-b708-000d3a0e23f3_0 1c4a702f0f18 rancher/pause-amd64:3.1 "/pause" 10 minutes ago Up 10 minutes k8s_POD_canal-lc6g6_kube-system_05904de9-394e-11e9-b708-000d3a0e23f3_0 0da1cada08e1 rancher/hyperkube:v1.11.6-rancher1 "/opt/rke-tools/entr…" 10 minutes ago Up 10 minutes kube-proxy 57f44998f34a rancher/hyperkube:v1.11.6-rancher1 "/opt/rke-tools/entr…" 11 minutes ago Up 11 minutes kubelet 50f424c4daec rancher/hyperkube:v1.11.6-rancher1 "/opt/rke-tools/entr…" 11 minutes ago Up 11 minutes kube-scheduler 502d327912d9 rancher/hyperkube:v1.11.6-rancher1 "/opt/rke-tools/entr…" 11 minutes ago Up 11 minutes kube-controller-manager 9fc706bbf3a5 rancher/hyperkube:v1.11.6-rancher1 "/opt/rke-tools/entr…" 11 minutes ago Up 11 minutes kube-apiserver 2e7630c2047c rancher/coreos-etcd:v3.2.18 "/usr/local/bin/etcd…" 11 minutes ago Up 11 minutes etcd fef566337eb6 rancher/rke-tools:v0.1.15 "/opt/rke-tools/rke-…" 26 minutes ago Up 26 minutes etcd-rolling-snapshots
DI 20190226-1: RKE up segmentation fault on 0.1.16 - use correct user
amdocs@obriensystemsu0:~$ sudo rke up Segmentation fault (core dumped) # issue was I was using ubuntu as the yml user not amdocs in this case for a particular VM
DI 20190227-1: Verify no 110 pod limit per VM
Notes
Pre-RKE installation details in Cloud Native Deployment