see http://wiki.obrienlabs.cloud/display/DEV/Raspberry+PI
As part of OPNFV integration we need to validate running ONAP on ARM processor cores. My only experience with ARM is during iOS development or on Broadcom chips. We will run on the Broadcom chips to start. Note that this is pure experimentation - in that the granularity of the v3 boards is only 1Gb - which will not run most of our pods properly.
Holiday setup of a kubernetes cluster on arm based Broadcom 2837 SOC boards
Bare Metal Setup
Flash Raspian to all the SD cards
Enable ssh, wireless, rename host
sudo nano /etc/wpa_supplicant//wpa_supplicant.conf
Docker Setup
curl -sSL https://get.docker.com | sh
sudo usermod -aG docker pi
Note: Docker 17.11.0-ce requires memory control groups - either downgrade to 17.09.0-ce or adjust /boot/cmdline.txt
pi@pi0:~ $ docker run hello-world Status: Downloaded newer image for hello-world:latest docker: Error response from daemon: cgroups: memory cgroup not supported on this system: unknown. ERRO[0007] error waiting for container: context canceled pi@pi0:~ $ docker version Client: Version: 17.11.0-ce OS/Arch: linux/arm pi@pi0:~ $ sudo apt install docker-ce=17.09.0~ce-0~raspbian pi@pi0:~ $ docker run hello-world Hello from Docker! pi@pi0:~ $ sudo nano /boot/cmdline.txt pi@pi0:~ $ cat /boot/cmdline.txt # add to the end of first line: cgroup_enable=memory cgroup_memory=1 # reboot
Kubernetes Setup
http://blog.kubernetes.io/2015/12/creating-raspberry-pi-cluster-running.html
https://blog.alexellis.io/serverless-kubernetes-on-raspberry-pi/
https://gist.github.com/alexellis/fdbc90de7691a1b9edb545c17da2d975
Debian Bootstrap
enable wireless, add WPA region
2 Comments
Michael O'Brien
pi@pi0:~ $ sudo k8s-on-rpi-master/install-k8s-master.sh
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
bridge-utils
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 32.2 kB of archives.
After this operation, 96.3 kB of additional disk space will be used.
Get:1 http://raspbian.mirror.colo-serv.net/raspbian stretch/main armhf bridge-utils armhf 1.5-13+deb9u1 [32.2 kB]
Fetched 32.2 kB in 0s (47.6 kB/s)
Selecting previously unselected package bridge-utils.
(Reading database ... 123348 files and directories currently installed.)
Preparing to unpack .../bridge-utils_1.5-13+deb9u1_armhf.deb ...
Unpacking bridge-utils (1.5-13+deb9u1) ...
Setting up bridge-utils (1.5-13+deb9u1) ...
Processing triggers for man-db (2.7.6.1-2) ...
Copying kubernetes service configuration files
Reloading the system service configuration
Stopping the docker service
Enabling the new services
Created symlink /etc/systemd/system/multi-user.target.wants/docker-bootstrap.service → /lib/systemd/system/docker-bootstrap.service.
Created symlink /etc/systemd/system/multi-user.target.wants/k8s-etcd.service → /lib/systemd/system/k8s-etcd.service.
Created symlink /etc/systemd/system/multi-user.target.wants/k8s-flannel.service → /lib/systemd/system/k8s-flannel.service.
Created symlink /etc/systemd/system/multi-user.target.wants/k8s-master.service → /lib/systemd/system/k8s-master.service.
Starting the docker bootstrap service
Job for docker-bootstrap.service failed because the control process exited with error code.
See "systemctl status docker-bootstrap.service" and "journalctl -xe" for details.
Pulling necessary etcd Docker image
^C
pi@pi0:~ $ systemctl status docker-bootstrap.service
? docker-bootstrap.service - Docker Bootstrap Application Container Engine
Loaded: loaded (/lib/systemd/system/docker-bootstrap.service; enabled; vendor preset: enabled)
Active: activating (start-post) (Result: exit-code) since Mon 2017-12-25 03:31:05 UTC; 8s ago
Docs: https://docs.docker.com
Process: 3761 ExecStart=/usr/bin/docker daemon -H unix:///var/run/docker-bootstrap.sock -p /var/run/docker-bootstrap.pid --storage-driver=o
Main PID: 3761 (code=exited, status=1/FAILURE); Control PID: 3762 (sleep)
CGroup: /system.slice/docker-bootstrap.service
??control
??3762 sleep 10
Dec 25 03:31:05 pi0 systemd[1]: Starting Docker Bootstrap Application Container Engine...
Dec 25 03:31:05 pi0 docker[3761]: `docker daemon` is not supported on Linux. Please run `dockerd` directly
Dec 25 03:31:05 pi0 systemd[1]: docker-bootstrap.service: Main process exited, code=exited, status=1/FAILURE
pi@pi0:~ $ journalctl -xe
--
-- The result is failed.
Dec 25 03:31:55 pi0 systemd[1]: docker-bootstrap.service: Unit entered failed state.
Dec 25 03:31:55 pi0 systemd[1]: docker-bootstrap.service: Failed with result 'exit-code'.
Dec 25 03:31:55 pi0 systemd[1]: Starting Docker Bootstrap Application Container Engine...
-- Subject: Unit docker-bootstrap.service has begun start-up
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- Unit docker-bootstrap.service has begun starting up.
Dec 25 03:31:55 pi0 docker[3806]: `docker daemon` is not supported on Linux. Please run `dockerd` directly
Dec 25 03:31:55 pi0 systemd[1]: docker-bootstrap.service: Main process exited, code=exited, status=1/FAILURE
Dec 25 03:32:05 pi0 systemd[1]: Failed to start Docker Bootstrap Application Container Engine.
-- Subject: Unit docker-bootstrap.service has failed
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- Unit docker-bootstrap.service has failed.
--
-- The result is failed.
Dec 25 03:32:05 pi0 systemd[1]: docker-bootstrap.service: Unit entered failed state.
Dec 25 03:32:05 pi0 systemd[1]: docker-bootstrap.service: Failed with result 'exit-code'.
Dec 25 03:32:05 pi0 systemd[1]: Starting Docker Bootstrap Application Container Engine...
-- Subject: Unit docker-bootstrap.service has begun start-up
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- Unit docker-bootstrap.service has begun starting up.
Dec 25 03:32:05 pi0 docker[3822]: `docker daemon` is not supported on Linux. Please run `dockerd` directly
Dec 25 03:32:05 pi0 systemd[1]: docker-bootstrap.service: Main process exited, code=exited, status=1/FAILURE
Michael O'Brien
Attempting to join a cluster - even though we are running 17.09 and there is no manifest for raspian
pi@pi3:~ $ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
pi@pi3:~ $ sudo curl https://releases.rancher.com/install-docker/1.12.sh | sh
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 16609 100 16609 0 0 34205 0 --:--:-- --:--:-- --:--:-- 34245
Warning: the "docker" command appears to already exist on this system.
If you already have Docker installed, this script can cause trouble, which is
why we're displaying this warning and provide the opportunity to cancel the
installation.
If you installed the current Docker package using this script and are using it
again to update Docker, you can safely ignore this message.
You may press Ctrl+C now to abort this script.
+ sleep 20
+ sudo -E sh -c mkdir -p /etc/systemd/system/docker.service.d
+ sudo -E sh -c echo '[Service]\nExecStart=\nExecStart=/usr/bin/dockerd --storage-driver overlay -H fd://' > /etc/systemd/system/docker.service.d/overlay.conf
+ sudo -E sh -c sleep 3; apt-get update
Hit:1 http://archive.raspberrypi.org/debian stretch InRelease
Hit:2 http://mirrordirector.raspbian.org/raspbian stretch InRelease
Hit:3 https://download.docker.com/linux/raspbian stretch InRelease
Reading package lists... Done
+ sudo -E sh -c sleep 3; apt-get install -y -q dirmngr
Reading package lists...
Building dependency tree...
Reading state information...
Suggested packages:
pinentry-gnome3 tor
The following NEW packages will be installed:
dirmngr
0 upgraded, 1 newly installed, 0 to remove and 33 not upgraded.
Need to get 546 kB of archives.
After this operation, 963 kB of additional disk space will be used.
Get:1 http://raspbian.mirror.colo-serv.net/raspbian stretch/main armhf dirmngr armhf 2.1.18-8~deb9u1 [546 kB]
Fetched 546 kB in 0s (723 kB/s)
Selecting previously unselected package dirmngr.
(Reading database ... 123352 files and directories currently installed.)
Preparing to unpack .../dirmngr_2.1.18-8~deb9u1_armhf.deb ...
Unpacking dirmngr (2.1.18-8~deb9u1) ...
Processing triggers for man-db (2.7.6.1-2) ...
Setting up dirmngr (2.1.18-8~deb9u1) ...
+ sudo -E sh -c apt-key adv --keyserver hkp://ha.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D
Executing: /tmp/apt-key-gpghome.6qnyfSGPgI/gpg.1.sh --keyserver hkp://ha.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D
gpg: key F76221572C52609D: public key "Docker Release Tool (releasedocker) <docker@docker.com>" imported
gpg: Total number processed: 1
gpg: imported: 1
+ break
+ sudo -E sh -c apt-key adv -k 58118E89F3A912897C070ADBF76221572C52609D >/dev/null
Warning: apt-key output should not be parsed (stdout is not a terminal)
+ sudo -E sh -c mkdir -p /etc/apt/sources.list.d
+ dpkg --print-architecture
+ sudo -E sh -c echo deb \[arch=armhf\] https://apt.dockerproject.org/repo raspbian-9 main > /etc/apt/sources.list.d/docker.list
+ sudo -E sh -c sleep 3; apt-get update
Hit:1 http://mirrordirector.raspbian.org/raspbian stretch InRelease
Ign:2 https://apt.dockerproject.org/repo raspbian-9 InRelease
Ign:3 https://apt.dockerproject.org/repo raspbian-9 Release
Ign:4 https://apt.dockerproject.org/repo raspbian-9/main all Packages
Ign:5 https://apt.dockerproject.org/repo raspbian-9/main armhf Packages
Ign:6 https://apt.dockerproject.org/repo raspbian-9/main Translation-en
Ign:7 https://apt.dockerproject.org/repo raspbian-9/main Translation-en_GB
Ign:4 https://apt.dockerproject.org/repo raspbian-9/main all Packages
Ign:5 https://apt.dockerproject.org/repo raspbian-9/main armhf Packages
Ign:6 https://apt.dockerproject.org/repo raspbian-9/main Translation-en
Ign:7 https://apt.dockerproject.org/repo raspbian-9/main Translation-en_GB
Ign:4 https://apt.dockerproject.org/repo raspbian-9/main all Packages
Ign:5 https://apt.dockerproject.org/repo raspbian-9/main armhf Packages
Ign:6 https://apt.dockerproject.org/repo raspbian-9/main Translation-en
Ign:7 https://apt.dockerproject.org/repo raspbian-9/main Translation-en_GB
Ign:4 https://apt.dockerproject.org/repo raspbian-9/main all Packages
Ign:5 https://apt.dockerproject.org/repo raspbian-9/main armhf Packages
Ign:6 https://apt.dockerproject.org/repo raspbian-9/main Translation-en
Ign:7 https://apt.dockerproject.org/repo raspbian-9/main Translation-en_GB
Ign:4 https://apt.dockerproject.org/repo raspbian-9/main all Packages
Ign:5 https://apt.dockerproject.org/repo raspbian-9/main armhf Packages
Ign:6 https://apt.dockerproject.org/repo raspbian-9/main Translation-en
Ign:7 https://apt.dockerproject.org/repo raspbian-9/main Translation-en_GB
Ign:4 https://apt.dockerproject.org/repo raspbian-9/main all Packages
Err:5 https://apt.dockerproject.org/repo raspbian-9/main armhf Packages
403 Forbidden
Ign:6 https://apt.dockerproject.org/repo raspbian-9/main Translation-en
Ign:7 https://apt.dockerproject.org/repo raspbian-9/main Translation-en_GB
Hit:8 http://archive.raspberrypi.org/debian stretch InRelease
Reading package lists... Done
W: The repository 'https://apt.dockerproject.org/repo raspbian-9 Release' does not have a Release file.
N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: Failed to fetch https://apt.dockerproject.org/repo/dists/raspbian-9/main/binary-armhf/Packages 403 Forbidden
E: Some index files failed to download. They have been ignored, or old ones used instead.
pi@pi3:~ $ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
pi@pi3:~ $ docker version
Client:
Version: 17.11.0-ce
API version: 1.34
Go version: go1.8.3
Git commit: 1caf76c
Built: Mon Nov 20 19:01:07 2017
OS/Arch: linux/arm
Server:
Version: 17.11.0-ce
API version: 1.34 (minimum version 1.12)
Go version: go1.8.3
Git commit: 1caf76c
Built: Mon Nov 20 18:54:12 2017
OS/Arch: linux/arm
Experimental: false
pi@pi3:~ $ sudo docker run --rm --privileged -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher rancher/agent:v1.2.6 http://192.168.15.101:8880/v1/scripts/5A1D67EEE3D03C510296:1483142400000:LuCv9cX9vjRdNCG6suJHn3s
Unable to find image 'rancher/agent:v1.2.6' locally
v1.2.6: Pulling from rancher/agent
b3e1c725a85f: Pull complete
6a710864a9fc: Pull complete
d0ac3b234321: Pull complete
87f567b5cf58: Pull complete
063e24b217c4: Pull complete
d0a3f58caef0: Pull complete
16914729cfd3: Pull complete
2ce3828c0b9d: Pull complete
7df47a98fc4b: Pull complete
Digest: sha256:a68afd351c7417e6d66a77e97144113ceb7a9c3cdd46fb6e1fd5f5a5a33111cd
Status: Downloaded newer image for rancher/agent:v1.2.6
standard_init_linux.go:195: exec user process caused "exec format error"
pi@pi3:~ $