kubectl logs or kubectl exec not responding

Problem

Kubernetes cluster seems to work ok. but when you run the command

kubectl logs or any other commands produce various errors such as

rror from server (InternalError): Internal error occurred: failed calling webhook "mutating.eip.network.kubesphere.io": failed to call webhook: Post "https://openelb-admission.openelb-system.svc:443/validate-network-kubesphere-io-v1alpha2-svc?timeout=10s": context deadline exceeded

Error from server: Get "https://10.17.100.4:10250/containerLogs/default/dte-dev-server-6f7dffbcd5-45mjd/dte-dev-container": dial tcp 10.17.100.4:10250: i/o timeout

Solution

Need to open ports between the control plane and the worker nodes

For IPTABLES

iptables -I INPUT -s 10.0.0.0/8 -p tcp --dport 6443 -j ACCEPT
iptables -I INPUT -s 10.0.0.0/8 -p tcp --dport 2379 -j ACCEPT
iptables -I INPUT -s 10.0.0.0/8 -p tcp --dport 2380 -j ACCEPT
iptables -I INPUT -s 10.0.0.0/8 -p tcp --dport 10250 -j ACCEPT
iptables -I INPUT -s 10.0.0.0/8 -p tcp --dport 10251 -j ACCEPT
iptables -I INPUT -s 10.0.0.0/8 -p tcp --dport 10252 -j ACCEPT
iptables -I INPUT -s 10.0.0.0/8 -p tcp --dport 10255 -j ACCEPT

Firewall-CMD

sudo firewall-cmd --permanent --add-port=6443/tcp sudo firewall-cmd --permanent --add-port=2379-2380/tcp sudo firewall-cmd --permanent --add-port=10250/tcp sudo firewall-cmd --permanent --add-port=10251/tcp sudo firewall-cmd --permanent --add-port=10252/tcp sudo firewall-cmd --permanent --add-port=10255/tcp sudo firewall-cmd –reload