see http://wiki.obrienlabs.cloud/display/DEV/Raspberry+PI

As part of OPNFV integration we need to validate running ONAP on ARM processor cores.  My only experience with ARM is during iOS development or on Broadcom chips.  We will run on the Broadcom chips to start.  Note that this is pure experimentation - in that the granularity of the v3 boards is only 1Gb - which will not run most of our pods properly.

Holiday setup of a kubernetes cluster on arm based Broadcom 2837 SOC boards

Bare Metal Setup


Flash Raspian to all the SD cards

Enable ssh, wireless, rename host

 sudo nano /etc/wpa_supplicant//wpa_supplicant.conf


Docker Setup

curl -sSL https://get.docker.com | sh

sudo usermod -aG docker pi

Note: Docker 17.11.0-ce requires memory control groups - either downgrade to 17.09.0-ce or adjust /boot/cmdline.txt

pi@pi0:~ $ docker run hello-world
Status: Downloaded newer image for hello-world:latest
docker: Error response from daemon: cgroups: memory cgroup not supported on this system: unknown.
ERRO[0007] error waiting for container: context canceled 
pi@pi0:~ $ docker version
Client:
 Version:      17.11.0-ce
 OS/Arch:      linux/arm
pi@pi0:~ $ sudo apt install docker-ce=17.09.0~ce-0~raspbian
pi@pi0:~ $ docker run hello-world
Hello from Docker!
pi@pi0:~ $ sudo nano /boot/cmdline.txt
pi@pi0:~ $ cat /boot/cmdline.txt 
# add to the end of first line: cgroup_enable=memory cgroup_memory=1
# reboot


Kubernetes Setup

http://blog.kubernetes.io/2015/12/creating-raspberry-pi-cluster-running.html

https://blog.alexellis.io/serverless-kubernetes-on-raspberry-pi/

https://gist.github.com/alexellis/fdbc90de7691a1b9edb545c17da2d975

https://blog.zhaw.ch/icclab/supporting-container-based-application-deployment-to-heterogeneous-hardware-using-rancher/


Debian Bootstrap

enable wireless, add WPA region


  • No labels

2 Comments

  1. Michael O'Brien

    pi@pi0:~ $ sudo k8s-on-rpi-master/install-k8s-master.sh

    Reading package lists... Done

    Building dependency tree       

    Reading state information... Done

    The following NEW packages will be installed:

      bridge-utils

    0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.

    Need to get 32.2 kB of archives.

    After this operation, 96.3 kB of additional disk space will be used.

    Get:1 http://raspbian.mirror.colo-serv.net/raspbian stretch/main armhf bridge-utils armhf 1.5-13+deb9u1 [32.2 kB]

    Fetched 32.2 kB in 0s (47.6 kB/s)       

    Selecting previously unselected package bridge-utils.

    (Reading database ... 123348 files and directories currently installed.)

    Preparing to unpack .../bridge-utils_1.5-13+deb9u1_armhf.deb ...

    Unpacking bridge-utils (1.5-13+deb9u1) ...

    Setting up bridge-utils (1.5-13+deb9u1) ...

    Processing triggers for man-db (2.7.6.1-2) ...

    Copying kubernetes service configuration files

    Reloading the system service configuration

    Stopping the docker service

    Enabling the new services

    Created symlink /etc/systemd/system/multi-user.target.wants/docker-bootstrap.service → /lib/systemd/system/docker-bootstrap.service.

    Created symlink /etc/systemd/system/multi-user.target.wants/k8s-etcd.service → /lib/systemd/system/k8s-etcd.service.

    Created symlink /etc/systemd/system/multi-user.target.wants/k8s-flannel.service → /lib/systemd/system/k8s-flannel.service.

    Created symlink /etc/systemd/system/multi-user.target.wants/k8s-master.service → /lib/systemd/system/k8s-master.service.

    Starting the docker bootstrap service

    Job for docker-bootstrap.service failed because the control process exited with error code.

    See "systemctl status docker-bootstrap.service" and "journalctl -xe" for details.

    Pulling necessary etcd Docker image


    ^C

    pi@pi0:~ $ systemctl status docker-bootstrap.service

    ? docker-bootstrap.service - Docker Bootstrap Application Container Engine

       Loaded: loaded (/lib/systemd/system/docker-bootstrap.service; enabled; vendor preset: enabled)

       Active: activating (start-post) (Result: exit-code) since Mon 2017-12-25 03:31:05 UTC; 8s ago

         Docs: https://docs.docker.com

      Process: 3761 ExecStart=/usr/bin/docker daemon -H unix:///var/run/docker-bootstrap.sock -p /var/run/docker-bootstrap.pid --storage-driver=o

     Main PID: 3761 (code=exited, status=1/FAILURE); Control PID: 3762 (sleep)

       CGroup: /system.slice/docker-bootstrap.service

               ??control

                 ??3762 sleep 10


    Dec 25 03:31:05 pi0 systemd[1]: Starting Docker Bootstrap Application Container Engine...

    Dec 25 03:31:05 pi0 docker[3761]: `docker daemon` is not supported on Linux. Please run `dockerd` directly

    Dec 25 03:31:05 pi0 systemd[1]: docker-bootstrap.service: Main process exited, code=exited, status=1/FAILURE


    pi@pi0:~ $ journalctl -xe

    -- 

    -- The result is failed.

    Dec 25 03:31:55 pi0 systemd[1]: docker-bootstrap.service: Unit entered failed state.

    Dec 25 03:31:55 pi0 systemd[1]: docker-bootstrap.service: Failed with result 'exit-code'.

    Dec 25 03:31:55 pi0 systemd[1]: Starting Docker Bootstrap Application Container Engine...

    -- Subject: Unit docker-bootstrap.service has begun start-up

    -- Defined-By: systemd

    -- Support: https://www.debian.org/support

    -- 

    -- Unit docker-bootstrap.service has begun starting up.

    Dec 25 03:31:55 pi0 docker[3806]: `docker daemon` is not supported on Linux. Please run `dockerd` directly

    Dec 25 03:31:55 pi0 systemd[1]: docker-bootstrap.service: Main process exited, code=exited, status=1/FAILURE

    Dec 25 03:32:05 pi0 systemd[1]: Failed to start Docker Bootstrap Application Container Engine.

    -- Subject: Unit docker-bootstrap.service has failed

    -- Defined-By: systemd

    -- Support: https://www.debian.org/support

    -- 

    -- Unit docker-bootstrap.service has failed.

    -- 

    -- The result is failed.

    Dec 25 03:32:05 pi0 systemd[1]: docker-bootstrap.service: Unit entered failed state.

    Dec 25 03:32:05 pi0 systemd[1]: docker-bootstrap.service: Failed with result 'exit-code'.

    Dec 25 03:32:05 pi0 systemd[1]: Starting Docker Bootstrap Application Container Engine...

    -- Subject: Unit docker-bootstrap.service has begun start-up

    -- Defined-By: systemd

    -- Support: https://www.debian.org/support

    -- 

    -- Unit docker-bootstrap.service has begun starting up.

    Dec 25 03:32:05 pi0 docker[3822]: `docker daemon` is not supported on Linux. Please run `dockerd` directly

    Dec 25 03:32:05 pi0 systemd[1]: docker-bootstrap.service: Main process exited, code=exited, status=1/FAILURE

  2. Michael O'Brien

    Attempting to join a cluster - even though we are running 17.09 and there is no manifest for raspian

    pi@pi3:~ $ docker ps

    CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

    pi@pi3:~ $ sudo curl https://releases.rancher.com/install-docker/1.12.sh | sh

      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current

                                     Dload  Upload   Total   Spent    Left  Speed

    100 16609  100 16609    0     0  34205      0 --:--:-- --:--:-- --:--:-- 34245

    Warning: the "docker" command appears to already exist on this system.


    If you already have Docker installed, this script can cause trouble, which is

    why we're displaying this warning and provide the opportunity to cancel the

    installation.


    If you installed the current Docker package using this script and are using it

    again to update Docker, you can safely ignore this message.


    You may press Ctrl+C now to abort this script.

    + sleep 20

    + sudo -E sh -c mkdir -p /etc/systemd/system/docker.service.d

    + sudo -E sh -c echo '[Service]\nExecStart=\nExecStart=/usr/bin/dockerd --storage-driver overlay -H fd://' > /etc/systemd/system/docker.service.d/overlay.conf

    + sudo -E sh -c sleep 3; apt-get update

    Hit:1 http://archive.raspberrypi.org/debian stretch InRelease

    Hit:2 http://mirrordirector.raspbian.org/raspbian stretch InRelease

    Hit:3 https://download.docker.com/linux/raspbian stretch InRelease

    Reading package lists... Done

    + sudo -E sh -c sleep 3; apt-get install -y -q dirmngr

    Reading package lists...

    Building dependency tree...

    Reading state information...

    Suggested packages:

      pinentry-gnome3 tor

    The following NEW packages will be installed:

      dirmngr

    0 upgraded, 1 newly installed, 0 to remove and 33 not upgraded.

    Need to get 546 kB of archives.

    After this operation, 963 kB of additional disk space will be used.

    Get:1 http://raspbian.mirror.colo-serv.net/raspbian stretch/main armhf dirmngr armhf 2.1.18-8~deb9u1 [546 kB]

    Fetched 546 kB in 0s (723 kB/s)

    Selecting previously unselected package dirmngr.

    (Reading database ... 123352 files and directories currently installed.)

    Preparing to unpack .../dirmngr_2.1.18-8~deb9u1_armhf.deb ...

    Unpacking dirmngr (2.1.18-8~deb9u1) ...

    Processing triggers for man-db (2.7.6.1-2) ...

    Setting up dirmngr (2.1.18-8~deb9u1) ...

    + sudo -E sh -c apt-key adv --keyserver hkp://ha.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D

    Executing: /tmp/apt-key-gpghome.6qnyfSGPgI/gpg.1.sh --keyserver hkp://ha.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D

    gpg: key F76221572C52609D: public key "Docker Release Tool (releasedocker) <docker@docker.com>" imported

    gpg: Total number processed: 1

    gpg:               imported: 1

    + break

    + sudo -E sh -c apt-key adv -k 58118E89F3A912897C070ADBF76221572C52609D >/dev/null

    Warning: apt-key output should not be parsed (stdout is not a terminal)

    + sudo -E sh -c mkdir -p /etc/apt/sources.list.d

    + dpkg --print-architecture

    + sudo -E sh -c echo deb \[arch=armhf\] https://apt.dockerproject.org/repo raspbian-9 main > /etc/apt/sources.list.d/docker.list

    + sudo -E sh -c sleep 3; apt-get update

    Hit:1 http://mirrordirector.raspbian.org/raspbian stretch InRelease

    Ign:2 https://apt.dockerproject.org/repo raspbian-9 InRelease

    Ign:3 https://apt.dockerproject.org/repo raspbian-9 Release

    Ign:4 https://apt.dockerproject.org/repo raspbian-9/main all Packages

    Ign:5 https://apt.dockerproject.org/repo raspbian-9/main armhf Packages

    Ign:6 https://apt.dockerproject.org/repo raspbian-9/main Translation-en

    Ign:7 https://apt.dockerproject.org/repo raspbian-9/main Translation-en_GB

    Ign:4 https://apt.dockerproject.org/repo raspbian-9/main all Packages

    Ign:5 https://apt.dockerproject.org/repo raspbian-9/main armhf Packages

    Ign:6 https://apt.dockerproject.org/repo raspbian-9/main Translation-en

    Ign:7 https://apt.dockerproject.org/repo raspbian-9/main Translation-en_GB

    Ign:4 https://apt.dockerproject.org/repo raspbian-9/main all Packages

    Ign:5 https://apt.dockerproject.org/repo raspbian-9/main armhf Packages

    Ign:6 https://apt.dockerproject.org/repo raspbian-9/main Translation-en

    Ign:7 https://apt.dockerproject.org/repo raspbian-9/main Translation-en_GB

    Ign:4 https://apt.dockerproject.org/repo raspbian-9/main all Packages

    Ign:5 https://apt.dockerproject.org/repo raspbian-9/main armhf Packages

    Ign:6 https://apt.dockerproject.org/repo raspbian-9/main Translation-en

    Ign:7 https://apt.dockerproject.org/repo raspbian-9/main Translation-en_GB

    Ign:4 https://apt.dockerproject.org/repo raspbian-9/main all Packages

    Ign:5 https://apt.dockerproject.org/repo raspbian-9/main armhf Packages

    Ign:6 https://apt.dockerproject.org/repo raspbian-9/main Translation-en

    Ign:7 https://apt.dockerproject.org/repo raspbian-9/main Translation-en_GB

    Ign:4 https://apt.dockerproject.org/repo raspbian-9/main all Packages

    Err:5 https://apt.dockerproject.org/repo raspbian-9/main armhf Packages

      403  Forbidden

    Ign:6 https://apt.dockerproject.org/repo raspbian-9/main Translation-en

    Ign:7 https://apt.dockerproject.org/repo raspbian-9/main Translation-en_GB

    Hit:8 http://archive.raspberrypi.org/debian stretch InRelease

    Reading package lists... Done

    W: The repository 'https://apt.dockerproject.org/repo raspbian-9 Release' does not have a Release file.

    N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use.

    N: See apt-secure(8) manpage for repository creation and user configuration details.

    E: Failed to fetch https://apt.dockerproject.org/repo/dists/raspbian-9/main/binary-armhf/Packages  403  Forbidden

    E: Some index files failed to download. They have been ignored, or old ones used instead.

    pi@pi3:~ $ docker ps

    CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

    pi@pi3:~ $ docker version

    Client:

     Version:      17.11.0-ce

     API version:  1.34

     Go version:   go1.8.3

     Git commit:   1caf76c

     Built:        Mon Nov 20 19:01:07 2017

     OS/Arch:      linux/arm


    Server:

     Version:      17.11.0-ce

     API version:  1.34 (minimum version 1.12)

     Go version:   go1.8.3

     Git commit:   1caf76c

     Built:        Mon Nov 20 18:54:12 2017

     OS/Arch:      linux/arm

     Experimental: false

    pi@pi3:~ $ sudo docker run --rm --privileged -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher rancher/agent:v1.2.6 http://192.168.15.101:8880/v1/scripts/5A1D67EEE3D03C510296:1483142400000:LuCv9cX9vjRdNCG6suJHn3s

    Unable to find image 'rancher/agent:v1.2.6' locally

    v1.2.6: Pulling from rancher/agent

    b3e1c725a85f: Pull complete 

    6a710864a9fc: Pull complete 

    d0ac3b234321: Pull complete 

    87f567b5cf58: Pull complete 

    063e24b217c4: Pull complete 

    d0a3f58caef0: Pull complete 

    16914729cfd3: Pull complete 

    2ce3828c0b9d: Pull complete 

    7df47a98fc4b: Pull complete 

    Digest: sha256:a68afd351c7417e6d66a77e97144113ceb7a9c3cdd46fb6e1fd5f5a5a33111cd

    Status: Downloaded newer image for rancher/agent:v1.2.6

    standard_init_linux.go:195: exec user process caused "exec format error"

    pi@pi3:~ $