PLEG is not healthy: Kubernetes Worker Node is in "NotReady" state

PLEG is not healthy: Kubernetes Worker Node is in "NotReady" state

The Kubernetes worker node shows "NotReady" state. The Error Shows as follows:

kubelet: I1014    16235 setters.go:518] Node became not ready: {Type:Ready Status:False LastHeartbeatTime:2019-10-14 01:40:08.225067533 -0500 CDT m=+3261716.350230569 LastTransitionTime:2019-10-14 01:40:08.225067533 -0500 CDT m=+3261716.350230569 Reason:KubeletNotReady Message:PLEG is not healthy: pleg was last seen active 3m5.74712971s ago; threshold is 3m0s}


When node enters a NotReady state due to an unhealthy PLEG. Check if you can pull all container Information.

 # curl --unix-socket /var/run/docker.sock http://v1.26/containers/json?all=1

# docker ps

# docker info

The PLEG issue can be due to node Overload, so we need to check cpu, memory. it may be due to over scheduled of pods.
PLEG is a simple loop that does the following:

list pod sandbox containers
list pod containers
wait 1s
return to step 1.

PLEG is marked unhealthy if the interval for any of these loops exceeds 3 minutes. An unhealthy PLEG marks a node NotReady to defer the scheduler from scheduling additional pods to the node until the machine stabilizes. A NotReady node is still doing work. PLEG returned to normal after the docker daemon stabilized on the system.

Workaround:

The workaround for this issue is to restart the docker services on the problematic Node.

systemctl restart docker

OR

You can reboot the problematic Node.