Fixing Pods Stuck in Pending State in Kubernetes

A pod stuck in Pending means the Kubernetes scheduler hasn’t been able to find any node satisfying its requirements — the fix is reading exactly why from the pod’s own events rather than guessing.

Step 1: read the pod’s events directly

kubectl describe pod mypod

The Events section at the bottom almost always states the scheduling failure reason explicitly — insufficient CPU/memory, an unsatisfied node affinity rule, or an untolerated taint are the most common, and describe names which one directly rather than leaving you to infer it.

Step 2: check for insufficient cluster resources

Events: 0/5 nodes are available: 5 Insufficient cpu.

This means every node in the cluster currently lacks enough allocatable CPU (or memory) to satisfy the pod’s resource requests — check current cluster capacity:

kubectl top nodes
kubectl describe nodes | grep -A5 "Allocated resources"

The fix is freeing or adding compatible capacity, or correcting requests only when measurements prove they are overstated. kubectl top shows current use, while scheduling uses requested resources against node allocatable capacity; a quiet cluster can still be fully requested. Check namespace quota and LimitRanges too.

Step 3: check for an unsatisfied node affinity or selector

Events: 0/5 nodes are available: 5 node(s) didn't match
        Pod's node affinity/selector.

kubectl get pod mypod -o yaml | grep -A10 affinity
kubectl get nodes --show-labels

Compare the pod’s affinity rules or nodeSelector against the labels actually present on your nodes — a typo in a label key or value here produces exactly this symptom, with the pod otherwise perfectly schedulable.

Step 4: check for an untolerated taint

Events: 0/5 nodes are available: 5 node(s) had untolerated taint.

kubectl describe nodes | grep Taints

Nodes with taints (commonly used to reserve nodes for specific workloads) reject any pod that doesn’t explicitly declare a matching toleration — add the appropriate toleration to the pod spec, or target untainted nodes instead, depending on whether the taint is intentional infrastructure policy or unexpected.

Step 5: check for a PersistentVolumeClaim that can’t be satisfied

kubectl get pvc
kubectl describe pvc mypvc

A pod requiring a PersistentVolumeClaim that itself can’t be bound (no matching PersistentVolume, or a StorageClass provisioning failure) will show the pod stuck Pending even though the compute resources it needs are otherwise perfectly available — a storage-side rather than scheduling-side cause.

Step 6: check for pod topology spread constraints being too strict

kubectl get pod mypod -o yaml | grep -A10 topologySpreadConstraints

Overly strict topology spread rules (requiring pods to be evenly distributed across zones or nodes in ways the current cluster topology can’t satisfy) can produce a Pending state that looks like a resource problem but is actually a distribution-constraint problem.

Step 7: check the scheduler’s own logs for additional context, in rare unclear cases

kubectl logs -n kube-system -l component=kube-scheduler

Useful when describe pod’s events don’t clearly explain the failure — the scheduler’s own logs occasionally provide more detail than what’s surfaced to pod events directly.

Managed control planes may not expose scheduler logs. Use provider diagnostics and scheduler events rather than assuming access. If a custom scheduler name is configured, inspect that scheduler and verify it is running; the default scheduler will not process pods assigned elsewhere.

Check less obvious placement blockers

Pod anti-affinity, required topology spread, host ports, extended resources such as GPUs, RuntimeClasses, volume node affinity, and maximum pod counts can each eliminate every node. Preemption messages explain whether removing lower-priority pods would help; do not add a high PriorityClass merely to force placement because it can disrupt critical workloads elsewhere.

A pod can also remain Pending while containers have not started because an init-time admission or sandbox dependency is failing. Inspect the full status and events chronologically. For a bound pod with PodScheduled=True, move from scheduler diagnosis to kubelet, CNI, volume mount, or image-pull diagnosis.

Preserve the failed pod specification and event output before editing its controller. Change one constraint at a time in version control and verify the owning Deployment, StatefulSet, or Job creates a schedulable replacement. Never delete protective affinity, taints, or topology rules without understanding why they exist; the apparent fix can place a workload on an unsupported architecture, wrong security tier, or single failure domain.

After recovery, verify Ready replicas, zone and node distribution, storage attachment, and customer behavior. Alert on Pending duration and unschedulable reasons, not merely the instantaneous count, so short expected scheduling delays do not create noise.

Why reading events first beats guessing at the cause

Kubernetes’ scheduler is explicit and specific about why it rejected every node for a given pod — the events section of kubectl describe pod names the actual constraint that couldn’t be satisfied directly, in nearly every case. Troubleshooting resource limits, then affinity, then taints, in sequence without first reading what the scheduler itself already reported wastes time re-deriving information Kubernetes already handed you directly.

Related:

Sources: