Multipass and Kubernetes setup
If you have access to public cloud providers like AWS, GCP most of them provides enough credits when you sign up with them to prepare for these exams. Another alternate option is to use Multipass from the comfort of your local data-center a.k.a desktop/laptop.
For this set up we are going to use Canconical's multipass. To install multipass you can use the following instructions from the official documentation.
Multipass is a mini-cloud on your workstation using native hypervisors of all the supported platforms (Windows, macOS and Linux), it will give you an Ubuntu command line in just a click ("Open shell") or a simple multipass shell command, or even a keyboard shortcut. Find what images are available with multipass find and create new instances with multipass launch
I have intentionally avoided using kind or minikube for certification preparation as they mask the cluster setup instructions from the user. The advantage of multipass is that you can play around with commands like kubeadm and learn along with a more hands-on approach.
After installation of multipass is complete, you can see the ubuntu image versions available to create Kubernetes hosts.
ubuntu@jumpbox:~$ multipass find
Image Aliases Version Description
snapcraft:core18 20201111 Snapcraft builder for Core 18
snapcraft:core20 20201111 Snapcraft builder for Core 20
snapcraft:core 20201211 Snapcraft builder for Core 16
core core16 20200818 Ubuntu Core 16
core18 20200812 Ubuntu Core 18
16.04 xenial 20201210 Ubuntu 16.04 LTS
18.04 bionic 20201211.1 Ubuntu 18.04 LTS
20.04 focal,lts 20201210 Ubuntu 20.04 LTS
20.10 groovy 20201210 Ubuntu 20.10
daily:21.04 devel,hirsute 20201215 Ubuntu 21.04
Launching instances¶
Launching instances in multipass is just a matter of a single command. The commands below have created 3 instances of Ubuntu 18.04 which you can use to set up clusters.
multipass launch bionic --name control-plane --cpus 2 -m 2G
multipass launch bionic --name workera --cpus 2 -m 2G
multipass launch bionic --name workerb --cpus 2 -m 2G
View launched instances
multipass list
Name State IPv4 Image
control-plane Running 10.130.101.41 Ubuntu 18.04 LTS
workera Running 10.130.101.105 Ubuntu 18.04 LTS
workerb Running 10.130.101.232 Ubuntu 18.04 LTS
Preparing instances for K8s installation¶
These commands should be run on all the instances created earlier to prepare them for Kubernetes installation. I have selected K8S_VERSION=1.19.3 but this can be changed to your desired Kubernetes version.
export K8S_VERSION=1.19.3
# Setup required sysctl params, these persist across reboots.
cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF
# Apply sysctl params without reboot
sudo sysctl --system
# (Install containerd)
sudo apt-get update && sudo apt-get install -y containerd
# Configure containerd
sudo mkdir -p /etc/containerd
sudo containerd config default | sudo tee /etc/containerd/config.toml
# Restart containerd
sudo systemctl restart containerd
# Install kubelet, kubectl and kubeadm
sudo apt-get update && sudo apt-get install -y apt-transport-https curl
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
cat <<EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF
sudo apt-get update
sudo apt-get install -y kubelet=${K8S_VERSION}-00 kubeadm=${K8S_VERSION}-00 kubectl=${K8S_VERSION}-00
sudo apt-mark hold kubelet kubeadm kubectl
Initializing Kubernetes control plane¶
After the instances are prepared for K8s installation the multipass shell command can be used to get access to the instance and initialize the control plane.
# Login to the control-plane instance
multipass shell control-plane
# Initialize the control plane
sudo kubeadm init --pod-network-cidr=10.200.0.0/16
When kubeadm init command finishes executing it prints out the instructions to get access to the cluster and configuring ~/.kube/config. It also prints the join command that can be used to add other nodes to the cluster. Here is an example of a similar message
# To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
# Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 10.130.101.41:6443 --token 0s61b7.suucm0jj3yt5j9b4 \
--discovery-token-ca-cert-hash sha256:c12795cb17b7f4434575eddef26f215e4ae94973ff619c290a00ba8bca61ba5a
Status after kubeadm init¶
kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
control-plane NotReady master 52s v1.19.3 10.130.101.41 <none> Ubuntu 18.04.5 LTS 4.15.0-124-generic containerd://1.3.3
Joining Worker Nodes¶
To add worker nodes to the cluster the kubeadm join command from the previous section can be used on each worker.
# Joining Worker A
multipass exec workera -- sudo kubeadm join 10.130.101.41:6443 --token 0s61b7.suucm0jj3yt5j9b4 \
--discovery-token-ca-cert-hash sha256:c12795cb17b7f4434575eddef26f215e4ae94973ff619c290a00ba8bca61ba5a
#Joining Worker B
multipass exec workera -- sudo kubeadm join 10.130.101.41:6443 --token 0s61b7.suucm0jj3yt5j9b4 \
--discovery-token-ca-cert-hash sha256:c12795cb17b7f4434575eddef26f215e4ae94973ff619c290a00ba8bca61ba5a
To access the Kubernetes cluster easily you can transfer the kubeconfig from control-plane node to your local desktop/laptop.
# Transfer kubeconfig to your local machine
ubuntu@jumpbox:~/multipass$ multipass transfer control-plane:/home/ubuntu/.kube/config - > kubeconfig
ubuntu@jumpbox:~/multipass$ export KUBECONFIG=kubeconfig
# Node status after adding workers
kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
control-plane NotReady master 29m v1.19.3 10.130.101.41 <none> Ubuntu 18.04.5 LTS 4.15.0-124-generic containerd://1.3.3
workera NotReady <none> 2m25s v1.19.3 10.130.101.105 <none> Ubuntu 18.04.5 LTS 4.15.0-124-generic containerd://1.3.3
workerb NotReady <none> 79s v1.19.3 10.130.101.232 <none> Ubuntu 18.04.5 LTS 4.15.0-124-generic containerd://1.3.3
This marks the completion of the Kubernetes cluster creation but you will notice 👆 that some of the nodes are still in Not Ready state. This is because we have not deployed a CNI yet. For the CNI I have picked up Antrea as I'm trying to learn and explore it. You can choose a CNI of your choice.
Antrea is a Kubernetes-native open source CNI and comes with CLI and UI tools that provide features like packet tracing, policy analysis, flow inspection. Antrea is a Kubernetes networking solution intended to be Kubernetes native. It operates at Layer¾ to provide networking and security services for a Kubernetes cluster, leveraging Open vSwitch as the networking data plane.
The CNI installation is just running a single command. Once Antrea is successfully installed we can see that the status of all the nodes is changed to Ready and the status of all the pods is switched to Running state.
# Antrea Installation
kubectl apply -f https://github.com/vmware-tanzu/antrea/releases/download/v0.12.0/antrea.yml
# Node Status
kubectl get nodes
NAME STATUS ROLES AGE VERSION
control-plane Ready master 48m v1.19.3
workera Ready <none> 21m v1.19.3
workerb Ready <none> 20m v1.19.3
# Pod Status
kubectl get pods -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system antrea-agent-495m7 2/2 Running 0 86s
kube-system antrea-agent-c2h4r 2/2 Running 0 86s
kube-system antrea-agent-nkl5h 2/2 Running 0 86s
kube-system antrea-controller-84c7944b8d-g4jvw 1/1 Running 0 86s
kube-system coredns-f9fd979d6-4wrmn 1/1 Running 0 45m
kube-system coredns-f9fd979d6-dgqmj 1/1 Running 0 45m
kube-system etcd-control-plane 1/1 Running 0 45m
kube-system kube-apiserver-control-plane 1/1 Running 0 45m
kube-system kube-controller-manager-control-plane 1/1 Running 0 45m
kube-system kube-proxy-5279z 1/1 Running 0 18m
kube-system kube-proxy-dxxk8 1/1 Running 0 17m
kube-system kube-proxy-tv7gr 1/1 Running 0 45m
kube-system kube-scheduler-control-plane 1/1 Running 0 45m
Hope this helps in easily setting up Kubernetes clusters when you are preparing for your certification exams or in general if you want to play around with Kubernetes.