In our previous step we cleaned up the FAH installation on our Stack of NUCs.

Now we are going to install Kubernetes with:

• one NUC master node this we can call NUC 3
• remaining NUCs “workers”

Purpose:

• Demonstrate a running a complex workload of web applications on Kubernetes

References

• https://github.com/brettplarson/nuctestlab
• https://www.linuxtechi.com/install-kubernetes-on-ubuntu-22-04/
• https://github.com/torgeirl/kubernetes-playbooks

1. From NUC 1, log in to the Ansible control node, NUC 2.
2. Create the k8s folder on the Ansible control node, NUC 2
   • mkdir /home/ansible/my-project/k8s
3. Create inventory file
   • Choose one IP address to be the Kubernetes master, add it to [master]
   • The rest of the IP address should be added to [workers]

   • inventory

     [master]
     [workers]
      
     [all:vars]
     ansible_python_interpreter=/usr/bin/python3
     ansible_user='ansible'
     ansible_become=true
     ansible_become_method=sudo

4. Set up ansible.cfg file to tell Ansible to use the inventory file

   • ansible.cfg

     [defaults]
     inventory         = inventory

5. Set up the host files for name resolution
   • Playbook will update the '/etc/hosts' files on all the nodes to allow them to resolve each other by name, but without DNS
   • Also adds this information to the Ansible control node (see 'localhost' in the script)

   • updatehostsfile.yml

     ---
     - name: Update etc/hosts file
       hosts: all, localhost
       gather_facts: true
       tasks:
         - name: Populate all /etc/hosts files
           tags: etchostsupdate
           become: true
           become_user: root
           lineinfile:
             path: "/etc/hosts"
             regexp: '.*{{ item }}$'
             line: "{{ hostvars[item]['ansible_default_ipv4'].address }}\t{{ hostvars[item]['ansible_hostname'] }}\t{{ hostvars[item]['ansible_hostname'] }}"
             state: present
           with_items: '{{ groups.all }}'

   • ansible-playbook updatehostsfile.yml

Check https://github.com/torgeirl/kubernetes-playbooks for updates to the playbooks below

1. Install some prerequisites on ALL the Kubernetes nodes

   • kube-dependencies.yml

     ---
     - hosts: all
       become: true
       tasks:
         - fail:
             msg: "OS should be Ubuntu 22.04, not {{ ansible_distribution }} {{ ansible_distribution_version }}"
           when: ansible_distribution != 'Ubuntu' or ansible_distribution_version != '22.04'
     
         - name: Update APT packages
           apt:
             update_cache: true
     
         - name: Reboot and wait for reboot to complete
           reboot:
     
         - name: Disable SWAP (Kubeadm requirement)
           shell: |
             swapoff -a
     
         - name: Disable SWAP in fstab (Kubeadm requirement)
           replace:
             path: /etc/fstab
             regexp: '^([^#].*?\sswap\s+sw\s+.*)$'
             replace: '# \1'
     
         - name: Create an empty file for the Containerd module
           copy:
             content: ""
             dest: /etc/modules-load.d/containerd.conf
             force: false
     
         - name: Configure modules for Containerd
           blockinfile:
             path: /etc/modules-load.d/containerd.conf
             block: |
                  overlay
                  br_netfilter
     
         - name: Create an empty file for Kubernetes sysctl params
           copy:
             content: ""
             dest: /etc/sysctl.d/99-kubernetes-cri.conf
             force: false
     
         - name: Configure sysctl params for Kubernetes
           lineinfile:
             path: /etc/sysctl.d/99-kubernetes-cri.conf
             line: "{{ item }}"
           with_items:
             - 'net.bridge.bridge-nf-call-iptables  = 1'
             - 'net.ipv4.ip_forward                 = 1'
             - 'net.bridge.bridge-nf-call-ip6tables = 1'
     
         - name: Apply sysctl params without reboot
           command: sysctl --system
     
         - name: Install APT Transport HTTPS
           apt:
             name: apt-transport-https
             state: present
     
         - name: Add Docker apt-key
           apt_key:
             url: https://download.docker.com/linux/ubuntu/gpg
             state: present
     
         - name: Add Docker's APT repository
           apt_repository:
             repo: "deb [arch=amd64] https://download.docker.com/linux/ubuntu {{ ansible_distribution_release }} stable"
             filename: "docker-{{ ansible_distribution_release }}"
     
         - name: Add Kubernetes apt-key
           apt_key:
             url: https://packages.cloud.google.com/apt/doc/apt-key.gpg
             state: present
     
         - name: Add Kubernetes' APT repository
           apt_repository:
             repo: deb https://apt.kubernetes.io/ kubernetes-xenial main
             state: present
             filename: 'kubernetes'
     
         - name: Install Containerd
           apt:
             name: containerd.io
             state: present
     
         - name: Create Containerd directory
           file:
             path: /etc/containerd
             state: directory
     
         - name: Add Containerd configuration
           shell: /usr/bin/containerd config default > /etc/containerd/config.toml
     
         - name: Configuring the systemd cgroup driver for Containerd
           lineinfile:
             path: /etc/containerd/config.toml
             regexp: '            SystemdCgroup = false'
             line: '            SystemdCgroup = true'
     
         - name: Enable the Containerd service and start it
           systemd:
             name: containerd
             state: restarted
             enabled: true
             daemon-reload: true
     
         - name: Install Kubelet
           apt:
             name: kubelet=1.26.*
             state: present
             update_cache: true
     
         - name: Install Kubeadm
           apt:
             name: kubeadm=1.26.*
             state: present
     
         - name: Enable the Kubelet service, and enable it persistently
           service:
             name: kubelet
             enabled: true
     
         - name: Load br_netfilter kernel module
           modprobe:
             name: br_netfilter
             state: present
     
         - name: Set bridge-nf-call-iptables
           sysctl:
             name: net.bridge.bridge-nf-call-iptables
             value: 1
     
         - name: Set ip_forward
           sysctl:
             name: net.ipv4.ip_forward
             value: 1
     
         - name: Check Kubelet args in Kubelet config
           shell: grep "^Environment=\"KUBELET_EXTRA_ARGS=" /etc/systemd/system/kubelet.service.d/10-kubeadm.conf || true
           register: check_args
      
         - name: Add runtime args in Kubelet config
           lineinfile:
             dest: "/etc/systemd/system/kubelet.service.d/10-kubeadm.conf"
             line: "Environment=\"KUBELET_EXTRA_ARGS= --runtime-cgroups=/system.slice/containerd.service --container-runtime-endpoint=unix:///run/containerd/containerd.sock\""
             insertafter: '\[Service\]'
           when: check_args.stdout == ""
      
         - name: Reboot and wait for reboot to complete
           reboot:
      
     - hosts: master
       become: true
       tasks:
         - name: Install Kubectl
           apt:
             name: kubectl=1.26.*
             state: present
             force: true  # allow downgrades

   • ansible-playbook kube-dependencies.yml
2. Configure kubernetes cluster on master node

   • master.yml

     ---
     - hosts: master
       become: true
       tasks:
         - name: Create an empty file for Kubeadm configuring
           copy:
             content: ""
             dest: /etc/kubernetes/kubeadm-config.yaml
             force: false
     
         - name: Configuring the container runtime including its cgroup driver
           blockinfile:
             path: /etc/kubernetes/kubeadm-config.yaml
             block: |
                  kind: ClusterConfiguration
                  apiVersion: kubeadm.k8s.io/v1beta3
                  networking:
                    podSubnet: "10.244.0.0/16"
                  ---
                  kind: KubeletConfiguration
                  apiVersion: kubelet.config.k8s.io/v1beta1
                  runtimeRequestTimeout: "15m"
                  cgroupDriver: "systemd"
                  systemReserved:
                    cpu: 100m
                    memory: 350M
                  kubeReserved:
                    cpu: 100m
                    memory: 50M
                  enforceNodeAllocatable:
                  - pods
     
         - name: Initialize the cluster (this could take some time)
           shell: kubeadm init --config /etc/kubernetes/kubeadm-config.yaml >> cluster_initialized.log
           args:
             chdir: /home/ansible
             creates: cluster_initialized.log
     
         - name: Create .kube directory
           become: true
           become_user: ansible
           file:
             path: $HOME/.kube
             state: directory
             mode: 0755
     
         - name: Copy admin.conf to user's kube config
           copy:
             src: /etc/kubernetes/admin.conf
             dest: /home/ansible/.kube/config
             remote_src: true
             owner: ansible
     
         - name: Install Pod network
           become: true
           become_user: ansible
           shell: kubectl apply -f https://raw.githubusercontent.com/flannel-io/flannel/master/Documentation/kube-flannel.yml >> pod_network_setup.log
           args:
             chdir: $HOME
             creates: pod_network_setup.log

   • ansible-playbook master.yml
3. SSH to the master and verify the master nodes gets status Ready
   • ssh MASTER_IP kubectl get nodes
4. Initialize workers
   • Modify the file to replace MASTERIP with the IP address of your master node in two (2) places

   • workers.yml

     ---
     - hosts: master
       become: true
       # gather_facts: false
       tasks:
         - name: Get join command
           shell: kubeadm token create --print-join-command
           register: join_command_raw
     
         - name: Set join command
           set_fact:
             join_command: "{{ join_command_raw.stdout_lines[0] }}"
     
     - hosts: workers
       become: true
       tasks:
         - name: TCP port 6443 on master is reachable from worker
           wait_for: "host={{ 'MASTERIP' }} port=6443 timeout=1"
     
         - name: Join cluster
           shell: "{{ hostvars['MASTERIP'].join_command }} >> node_joined.log"
           args:
             chdir: /home/ansible
             creates: node_joined.log

   • ansible-playbook workers.yml
5. SSH to the master and verify all the nodes return status Ready
   • ssh MASTER_IP kubectl get nodes
6. Install kubectl on NUC 2 for automation with Kubernetes
   • create file /home/ansible/my-project/k8s/kubectlcontrolnode.yml

   • kubectlcontrolnode.yml

     ---
     - hosts: localhost
       become: true
       gather_facts: false
       tasks:
         - name: Update APT packages
           apt:
             update_cache: true
     
         - name: Add Kubernetes apt-key
           apt_key:
             url: https://packages.cloud.google.com/apt/doc/apt-key.gpg
             state: present
     
         - name: Add Kubernetes' APT repository
           apt_repository:
             repo: deb https://apt.kubernetes.io/ kubernetes-xenial main
             state: present
             filename: 'kubernetes'
     
         - name: Install Kubectl
           apt:
             name: kubectl=1.26.*
             state: present
             force: true  # allow downgrades

   • ansible-playbook kubectlcontrolnode.yml
   • Running kubctl version will fail at this point because you don't have credentials
   • Copy credentials
     • scp -r IPMASTER:/home/ansible/.kube ~/
   • Confirm it's working by running kubectl version
7. Install kubectl on NUC 1 for remote testing

   • sudo apt update
     sudo apt install -y ca-certificates curl
     sudo curl -fsSLo /etc/apt/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg
     echo "deb [signed-by=/etc/apt/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list
     sudo apt update
     sudo apt install -y kubectl

   • Running kubctl version will fail at this point because you don't have credentials
   • Copy credentials
     • scp -r tux@IPMASTER:/home/ansible/.kube ~/
   • Confirm it's working by running
     • kubectl version
     • kubectl get nodes

You have successful installed Kubernetes on your Stack of NUCs! Next we will demonstrate running a web app on Kubernetes.

Do some tests with Kubernetes.

1. kubectl run speedtester --image=docker.io/doritoes/speedtester:latest

2. kubectl get pods
   • kubectl get pods -w will “watch” the process of the container being created and changing status to Running; press control-c to exit
3. kubectl describe pod speedtester
4. kubectl delete pods/speedtester
   • this demonstrates another way to specify the pod, same as kubectl delete pod speedtester

1. Create yaml file

   • speedtest-pod.yml

     apiVersion: v1
     kind: Pod
     metadata:
       name: speedtester
     spec:
       containers:
         - image: docker.io/doritoes/speedtester:latest
           name: speedtester
           ports:
             - containerPort: 8080
               name: http
               protocol: TCP

2. Build the pod
   • kubectl apply -f speedtest-pod.yml
3. Examine the pod
   • kubectl get pods
   • kubectl describe pods speedtester
4. Test connecting to the speedtest application on the pod
   • from NUC 1
     • kubectl port-forward speedtester 8080:80
     • browse to http://localhost:8080
     • NOTE in testing, the port forwarding tended to break when it the upload test started (“error creating error stream for port 8080 → 80: Timeout occurred”)
       • the worker node NUC doesn't seem to be loaded at all
         • kubectl exec –stdin –tty speedtester – /bin/bash
         • htop
       • one workaround is to keep re-launching the port forward command, but that doesn't solve the upload issue
       • later on, we will demonstrate it is stable when using a better forwarding mechanism
       • while true; do kubectl port-forward speedtester 8080:80; done
5. Manage the pod

   • kubectl logs speedtester
     kubectl exec speedtester -- date
     kubectl exec speedtester -- uname -a

6. Delete the pod

   • kubectl delete -f speedtest-pod.yaml
     kubectl get pods

You can experiment running k9s on your Kubernetes master node

• https://github.com/derailed/k9s/releases

• mkdir k9s
  cd k9s
  wget https://github.com/derailed/k9s/releases/download/v0.27.3/k9s_Linux_amd64.tar.gz
  tar xzvf k9s_Linux_amd64.tar.gz
  ./k9s

https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/

This is a more advanced exercise to try. I won't be covering this in detail here.