Nodes in NotReady state

#1 · 15/02/2023, 7:01 am

Problem

User runs the command kubectl get nodes and gets all nodes in not ready state

NAME STATUS ROLES AGE VERSION
master01 NotReady control-plane 3d9h v1.25.0
worker01 NotReady <none> 3d9h v1.25.0
worker02 NotReady <none> 3d9h v1.25.0
worker03 NotReady <none> 14m v1.26.1

Solution

First installation

check the pods in kube-system and see the errors if the core-dns is not ready then your network configuration is not set properly

default-scheduler 0/4 nodes are available: 2 node(s) had untolerated taint {node.kubernetes.io/not-ready: }, 2 node(s) had untolerated taint {node.kubernetes.io/unreachable: }. preemption: 0/4 nodes are available: 4 Preemption is not helpful for scheduling.

This error can occur due to bad / no network configuration. Install weave networks

kubectl apply -f https://github.com/weaveworks/weave/releases/download/v2.8.1/weave-daemonset-k8s.yaml

#2 · 21/02/2023, 5:58 pm

Make Sure docker service is install and running

sudo apt install apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add –
sudo add-apt-repository “deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable”
apt-cache policy docker-ce
sudo apt install docker-ce
sudo systemctl status docker
sudo usermod -aG docker ${USER}
su – ${USER}

Uploaded files:

commands.txt

#3 · 21/02/2023, 6:05 pm

Check Node startup services status

service kubelet status

kubelet.service – kubelet: The Kubernetes Node Agent
Loaded: loaded (/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled)
Drop-In: /etc/systemd/system/kubelet.service.d
└─10-kubeadm.conf
Active: activating (auto-restart) (Result: exit-code) since Tue 2023-02-21 18:03:16 UTC; 4s ago
Docs: https://kubernetes.io/docs/home/
Process: 2185 ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS (code=exited, status=1/FAILUR>
Main PID: 2185 (code=exited, status=1/FAILURE)

check the logs

E0221 18:04:28.216775 2372 run.go:74] “command failed” err=”failed to validate kubelet flags: the container runtime endpoint address was not specified or empty, use –container-runtime-endpoint to set”
Feb 21 18:04:28 worker03 systemd[1]: kubelet.service: Main process exited, code=exited, status=1/FAILURE
Feb 21 18:04:28 worker03 systemd[1]: kubelet.service: Failed with result ‘exit-code’.

The node is not configured properly. It is advice to reset the node and re-configure and join it to the cluster

kubeadm reset

rm -rf /etc/cni/net.d/*

#On the Master Node

kubeadm token create –print-join-command

#on the node run the output command

kubeadm join 10.0.0.60:6443 –token ogiise.vxobyivdxi9y03fb –discovery-token-ca-cert-hash sha256:8d57bd15c57ee07aa790…………………

check the status of the nodes

kubectl get nodes

kubectl describe node worker01