How the Kubernetes Scheduler Actually Places Workloads

When you kubectl apply a pod, deciding which node it actually runs on is the job of a single, replaceable component: kube-scheduler. Its process is a well-defined two-phase pipeline — filtering out nodes that can’t work, then scoring the ones that remain — and understanding both phases is what turns “why did my pod land there” from a mystery into a traceable decision.

Phase 1: filtering — which nodes even qualify

The scheduler starts with every node in the cluster and eliminates any that can’t satisfy the pod’s hard requirements: insufficient CPU/memory (requests), a nodeSelector or required node affinity that doesn’t match, a toleration mismatch against a node’s taints, or a port conflict with something already running there.

resources:
  requests:
    cpu: "500m"
    memory: "256Mi"

kubectl describe node worker-3 | grep -A5 "Allocated resources"

A pod requesting more CPU than any single node has available (not just installed — already-allocated resources count against this) simply never passes the filtering phase on that node, regardless of how good a fit it would otherwise be.

Taints and tolerations: nodes opting out by default

A taint on a node repels pods unless they carry a matching toleration — the mechanism behind dedicating nodes to specific workloads (GPU nodes, for instance) without needing every other pod in the cluster to explicitly avoid them:

kubectl taint nodes gpu-node-1 dedicated=ml-workloads:NoSchedule

tolerations:
  - key: "dedicated"
    operator: "Equal"
    value: "ml-workloads"
    effect: "NoSchedule"

Phase 2: scoring — ranking the nodes that survive filtering

Every node that passes filtering gets scored by a set of scoring plugins, each contributing points based on a specific concern — spreading pods evenly across nodes (NodeResourcesBalancedAllocation), preferring nodes with more free resources left over (NodeResourcesFit), respecting pod affinity/anti-affinity preferences, and more. The node with the highest combined score wins.

kubectl get events --field-selector reason=Scheduled

Affinity and anti-affinity: expressing “near” and “away from”

Pod affinity and anti-affinity let you express placement relative to other running pods rather than absolute node properties — spreading replicas of the same deployment across failure domains (anti-affinity) or co-locating a cache alongside the service that uses it (affinity):

affinity:
  podAntiAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
            - key: app
              operator: In
              values: ["web"]
        topologyKey: "kubernetes.io/hostname"

topologyKey determines the granularity — spreading across hosts, availability zones, or any other labeled grouping the cluster’s nodes carry.

Priority and preemption: what happens under resource pressure

Pods carry a priority class, and when a high-priority pod can’t be scheduled due to resource pressure, the scheduler can preempt (evict) lower-priority pods elsewhere to make room — a deliberate, auditable trade-off, not a silent one:

apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: high-priority
value: 1000000

Custom schedulers: replacing the pipeline entirely

Because kube-scheduler is just a controller watching for unscheduled pods and writing a nodeName back, it’s fully replaceable — a cluster can run multiple schedulers simultaneously, with each pod choosing which one handles it via schedulerName, useful for specialized batch or ML workload schedulers with placement logic the default scheduler doesn’t support:

spec:
  schedulerName: my-custom-scheduler

Diagnosing a pod stuck in Pending

kubectl describe pod on a pod that never gets scheduled shows the scheduler’s own recorded reasoning in its Events section — almost always naming the specific filtering step every node failed (insufficient CPU, a taint with no matching toleration, an affinity rule with no satisfying node), which turns scheduling failures from a black box into a direct, per-node breakdown of exactly why:

kubectl describe pod my-pending-pod
# Events: 0/5 nodes are available: 3 Insufficient cpu, 2 node(s) had taint...

Why this two-phase design matters

Separating hard filtering from soft scoring is what lets Kubernetes express both absolute constraints (“this pod cannot run without a GPU”) and soft preferences (“prefer spreading these pods across zones, but don’t fail if you can’t”) in the same system without conflating the two — a pod either survives filtering everywhere and gets ranked, or it’s genuinely unschedulable, and that distinction is exactly what kubectl describe pod’s event log is designed to surface directly.