My K3S

Updated: September 28, 2024

k3s is best for creating a lightweight rpi cluster.

These are the steps used for creating my k3s pi cluster which contains
3 control nodes and 4 workers (stacked HA cluster)

Table of Contents

• Gather Facts
• Hardware
• Test
• Install
• Arguments
• CRI

Gather Facts

Commands to find necessary information.
EEPROM:

# get UUID information
blkid

# current EEPROM version
vcgencmd bootloader_version

# man page
rpi-eeprom-update -h

# check if update is available
sudo rpi-eeprom-update

# install the update
sudo rpi-eeprom-update -a

# read current EEPROM configuration
rpi-eeprom-config  # or
vcgencmd bootloader_config

# read from an EEPROM image
pri-eeprom-config pieeprom.bin

Hardware

1. Use Raspberry Pi Imager to install ubuntu or raspbianOS then SSH in:

• sudo apt update
• sudo apt full-upgrade -y
• sudo rpi-update
• sudo reboot now

2. SSH back into the sd card:

• sudo rpi-eeprom-update -d -a
• sudo raspi-config
  • 5 Localisation Options » L1 Locale » en_US.UTF-8 UTF-8 (select with spacebar) » OK
  • 5 Localisation Options » L2 Timezone » US » Eastern
  • 6 Advanced Options » A7 Bootloader Version » E1 Latest » OK » no
  • 6 Advanced Options » A6 Boot Order » B2 USB Boot » Finish » yes, reboot

3. Install git and rpi-clone

• sudo apt install git -y
• git clone https://github.com/billw2/rpi-clone.git
• cd rpi-clone
• sudo cp rpi-clone rpi-clone-setup /usr/local/sbin
• sudo rpi-clone-setup -t $(hostname -s)
• sudo rpi-clone -s

4. Connect USB || NVME SSD to RPI

• sudo fdisk -l || lsblk || diskutil list # linux || Linux || mac
• sudo sfdisk –delete /dev/sda
• sudo rpi-clone sda
  • leave blank for optional file system label
• hit enter to unmount
• it will tell you PARTUUID, I note it for future reference for other scripting

5. Shutdown pi4, disconnect all power, unplug SD, plug power back in, verify

• sudo shutdown -h now

6. Check where you are booting from:

• mount | egrep “/([[:space:]]|boot)”

Test Write && Read

dd if=/dev/zero of=./speedTestFile bs=20M count=5 oflag=direct
dd if=./speedTestFile of=/dev/zero bs=20M count=5 oflag=dsync

If ethernet keeps droping it may be issue with EEE (energy efficient ethernet)

# check eee status
ethtool --show-eee eth0

# turn eee off
sudo ethtool --set-eee eth0 eee off

Rest of setup can be done with ansible

3. Set networking
4. Establish SSH
5. Install dependencies for kubernetes
6. Configure cgroup driver

sudo rfkill block wifi
sudo rfkill block bluetooth

The Chosen

Calico # pod network
Traefik # load balancer
Glusterfs # storage volume
Helm # package manager
Prometheus # collect metric data
Grafana # visualize Prometheus data