Can a Kubernetes cluster be built using Raspberry Pis? Yes, it can.
The Raspberry Pi is a fantastic computer and I’ve used it in a variety of projects to assist with me learning more. Over the summer (2020) I used an Ansible playbook created by Jeff Geerling to briefly create a small Pi Kubernetes cluster, then moved onto another project. I wanted to return back to Kubernetes on Pi and my recent Pi network building has given me an opportunity to do just that.
First a note or two. There are resources out on the web that can automate a lot of the installation, from bash scripts to (as mentioned above) Ansible playbooks. There are also operating systems like Hypriot that come with Docker preinstalled. If you are after a quicker / easier method of building a Raspberry Pi Kubernetes cluster then please consider them. I’ve gone the longer, winding route as I want to understand more of the components / settings involved (e.g. see any errors, make my own mistakes and learn from them).
I’m using:
1x Raspberry Pi 4 4GB
2x Raspberry Pi 4 2GB
1x Raspberry Pi 3 1GB (not yet added)
All running Ubuntu server 20.4 64-bit as their operating system, all connected via ethernet cables via a small 5 port desktop switch.
I’ve split the project into several pieces:
Pi Networking
Docker – needs doing on all devices
Kubernetes – needs doing on all devices
Pi Networking
I covered the majority of this in my previous blog post. Each Raspberry Pi has been given a different hostname and I have edited the /etc/hosts on each device so that the device knows about the other hosts/ip addresses.
I have also set up SSH via a private key so that the primary node (also my DHCP server) can connect to each worker node via SSH without the need for passwords.
The DHCP / networking of my previous post is not needed if you are using an existing solution (e.g. home router) but I would recommend placing some IP reservations, the keyless SSH and edited /etc/hosts changes in place.
Docker
Ubuntu may come with some components of Docker installed, which need uninstalling. This can be done with the command:
sudo apt-get remove docker docker-engine docker.io containerd runc
Don’t worry if apt fails to find the components, it just means they were not already installed. Then some components need installing:
sudo apt-get install apt-transport-https ca-certificates curl gnupg-agent software-properties-common
With those components in place the Docker repository and signature key can be added:
curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=arm64] https://download.docker.com/linux/debian buster stable"
Apt should now know about the Docker repository, time to install the Docker components required:
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io
To check that Docker has installed run the command:
sudo docker --version
To allow Docker to run without the need for sudo, use the command:
sudo usermod -aG docker ubuntu
NOTE: The Docker documentation contains a warning about security and root access. I’m running this in a test environment and highly recommend that if you are following along you read: https://docs.docker.com/engine/security/#docker-daemon-attack-surface
Kubernetes
The Kubernetes installation is quite similar to the Docker install, but I found that I had to adjust some options for cgroups in Ubuntu.
The commands to add the Kubernetes apt-key, repository and download the Kubernetes components required are:
curl -fsSL https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
sudo echo deb http://apt.kubernetes.io/ kubernetes-xenial main>> /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
sudo apt install -y kubelet kubeadm kubectl kubernetes-cni
To edit the cgroup needed so that Kubernetes could run I checked /boot/firmware/config.txt for a line that reads cmdline=, to see which file cmdline= was referencing. For me /boot/firmware/config.txt ‘s cmdline=/boot/firmware/cmdline.txt.
I recommended backing up cmdline.txt (copy it to your home folder) before editing it, at least to provide a reference to what the original contained.
sudo nano /boot/firmware/cmdline.txt
And add:
cgroup_enable=cpu cgroup_enable=memory
to the end of the line in cmdline.txt.
Reboot.
Kubernetes is now installed and should be ready for use.
Kubernetes uses a primary device (currently referred to as a master node) to instruct and give work to other nodes. For me that primary device is also running the DHCP for my Pi network, and on that device I used the command:
sudo kubeadm init --pod-network-cidr CIDR_IP_ADDRESS_FOR_KUBERNETES --apiserver-advertise-address IP_ADDRESS_OF_PRIMARY_DEVICE --apiserver-cert-extra-sans kubernetes.test.local
CIDR_IP_ADDRESS_FOR_KUBERNETES is a CIDR address that Kubernetes can use, it is not the CIDR address of the network. Most examples I found online use 10.244.0.0/16
IP_ADDRESS_OF_PRIMARY_DEVICE is the IP address of my Pi 4 4GB as it will be organising the other nodes.
test.local is the name of my Pi network.
After running the command Kubernetes takes a few minutes to start up and will generate a command to use on the worker nodes so that they can join the cluster. Make sure to note this command down as it contains token values that are unique to the cluster.
On the primary node run:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Then on the worker nodes run the command that was generated above (i.e. the one with the token values). The nodes should then join the cluster.
On the primary node run the command:
kubectl get nodes
to see the status of the nodes.
The above took me sometime and I still have a few things to do in my Pi Kubernetes cluster including cluster networking (which is different to the networking I have already covered), the Kubernetes dashboard and actually deploying some stuff into it. Hopefully I will cover these in my next blog post.
GeekTechStuff 