Kubernetes: Installation and Test Deployment

Introduction

Kubernetes (also abbreviated as k8s) has become the de-facto standard for container orchestration and cluster deployment.

This series will develop a good sample app inside a Kubernetes cluster and show a number of features that are likely to be necessary when developing and deploying a distributed application: one or more web servers that host an application, a database, a work queue, and processes that drain the work queue. It will also, eventually, include observability components such as searchable logging.

This will be a journey, certainly more a marathon than a sprint.

What is Kubernetes?

There are several good introductions to Kubernetes which aren’t retread here. However, a good analogy might be to compare it to an operating system.

Kubernetes is like an operating system for distributed apps. As an operating system abstracts and manages resources necessary for applications to run on a single machine, Kubernetes abstracts and manages resources for the distributed application(s) running in a cluster. Whereas the process is the base unit of work for the operating system, the container is the base unit of work in the cluster.

Time Lapse Caveat

Kubernetes is currently a moving target. As of this writing, multiple tutorials are outdated and do not match current conditions, some just a year old.

Command line parameters change, version numbers change, etc. This article will not stand the test of time. Hopefully it will still be of use to you.

Environment Setup

There are many tutorials and articles on setting up Kubernetes based on your environment. It will likely take you some experimentation to properly install and configure a K8s environment on your local system. Production systems usually require additional configuration during deployment, though tools like Helm can make this process easier.

Below is the setup used for these articles. It is based on using multipass VMs to allow for (eventual) multi-node installations.

Local Environment

For this experiment, the cluster will live on and use:

Mac M1 - arm64

Multipass - Ubuntu VMs for nodes

k3s - Kubernetes small implementation for development and edge devices

There are many possible Kubernetes implementations, some of which use VMs and some using docker containers. This article uses VMs installed with multipass (linked above).

Given the ephemeral nature and rapid change of k8s right now, use the links for more information if you run into any problems installing multipass and k3s using the directions below.

Installation

Installing multipass is straightforward on macOS using Homebrew:

$ brew install multipass

Once installed, use multipass to create a VM of Ubuntu Linux LTS (as of now, version 22.04) named k3s with 2 CPUs, 4G of memory and a 20Gb disk, followed by entering a shell in the VM. (The cluster could probably get away with less memory and disk space, but not a lower CPU count.)

$ multipass launch -m 4G -d 20G -c 2 -n k3s lts
...
$ multipass shell k3s
Welcome to Ubuntu...
...
ubuntu@k3s:~$

Install k3s directly on the Linux VM. This will be the control plane (master) node as well as the sole worker node. (Multi-node installation and deployments to be described later.)

ubuntu@k3s:~$ sudo curl -sfL https://get.k3s.io | sh -
# Check for Ready node, takes ~30 seconds 
ubuntu@k3s:~$ sudo k3s kubectl cluster-info
Kubernetes control plane is running at https://127.0.0.1:6443
CoreDNS is running at https://127.0.0.1:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
Metrics-server is running at https://127.0.0.1:6443/api/v1/namespaces/kube-system/services/https:metrics-server:https/proxy

Initial Deployment

To test the deployment, the whomai app reports certain information about the pod it’s in.

To create a deployment, k3s needs a yaml file that describes the components. These will be described in more detail in the next article.

kind: Deployment
apiVersion: apps/v1
metadata:
  name: whoami
  labels:
    app: whoami

spec:
  replicas: 1
  selector:
    matchLabels:
      app: whoami
  template:
    metadata:
      labels:
        app: whoami
    spec:
      containers:
        - name: whoami
          image: traefik/whoami
          ports:
            - name: web
              containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: whoami
spec:
  type: NodePort
  ports:
    - port: 80
      nodePort: 31000
  selector:
    app: whoami

Put the above deployment description into a text file named whoami.yaml and deploy it from the multipass prompt with:

ubuntu@k3s:~$ sudo k3s kubectl apply -f whoami.yaml

View that it is deployed with:

ubuntu@k3s:~$ sudo k3s kubectl get pods

which should list a pod named whoami....

Test it working with curl to see output similar to the following:

ubuntu@k3s:~$ curl http://localhost:31000
Hostname: whoami-5dfdf459f4-d7k6j
IP: 127.0.0.1
IP: ::1
IP: 10.42.0.17
IP: fe80::703a:b9ff:fedc:af3a
RemoteAddr: 10.42.0.1:9597
GET / HTTP/1.1
Host: localhost:31000
User-Agent: curl/7.81.0
Accept: */*

🍌

If something doesn’t appear as mentioned, use sudo k3s kubectl describe pods to get information about the state of the pod, including the last few lines of events.

To clean up the cluster, use:

ubuntu@k3s:~$ sudo /usr/local/bin/k3s-killall.sh

which removes the cluster, though the k3s installation is still present.

🐞

multipass has had some instability with running after the laptop sleeps. Over time multipass becomes unresponsive, timing out on a variety of operations. The solution is to set up a sleep event to stop multipass. Hammerspoon and the script at the bottom of this article will sleep multipass when the laptop lid closes.

Next Time

The next article will describe the components used in the test deployment.

Sources

k3s on Raspberry Pi: Introduction

I'm taking a few days to play with Kubernetes. Since I have a stack of Raspberry Pi 2s otherwise doing nothing, I decided to use them. I started with Scott Hanselman's How to Build a Kubernetes Cluster with ARM Raspberry Pi then run .NET Core on OpenFaas blog post.

blog.differentpla.net

K3s

Kubeconfig is written to /etc/rancher/k3s/k3s.yaml sudo k3s kubectl get node # On a different node run the below command. # NODE_TOKEN comes from /var/lib/rancher/k3s/server/node-token on your server sudo k3s agent --server https://myserver:6443 --token ${NODE_TOKEN}

k3s.io

Local Kubernetes Cluster with K3s on Mac M1

Introduction Kubernetes(k8s) is an open-source system for managing large scale containerized applications. K3s is lightweight K8s in a single binary file. However, K3s won't work directly on Macbook as it needs systemd/OpenRC. To setup k8s/k3s on Mac, we need to setup a Linux layer on top of Mac.

avilpage.com

Local Kubernetes Cluster with K3s on Mac M1

Hammerspoon Script to Sleep Multipass

This Hammerspoon script seems to help the mentioned instability by stopping multipass when the laptop sleeps. Restarting multipass on wake up is left as an exercise for the reader. 😀

-- modified from https://gist.github.com/RafhaanShah/47b35ec0b291c8a5ca5824e804025696
function caffeinateWatcher(eventType)
    if (eventType == hs.caffeinate.watcher.systemWillSleep or
            eventType == hs.caffeinate.watcher.systemWillPowerOff) then
            print ("Going to Sleep... Stopping multipass")
            -- Execute sleep script
            hs.task.new("/usr/local/bin/multipass", nil, {"stop", "--all"}):start()
    elseif (eventType == hs.caffeinate.watcher.systemDidWake) then
            print ("Waking from Sleep...")
            -- Execute wake script
            -- hs.task.new("/Users/username/scripts/on_wake.sh", nil):start()
    end
end

sleepWatcher = hs.caffeinate.watcher.new(caffeinateWatcher)
sleepWatcher:start()