Skip to content
SRE & DevOpsDeep Dive Published Updated 7 min readViews unavailable

Kubernetes EndpointSlices: Backend Membership, Readiness, and Service Scale

Inspect EndpointSlice grouping and conditions to debug Service routing, dual-stack backends, selectorless services, and rollout traffic changes.

An ordinary Kubernetes Service provides a stable name and virtual address while the Pods behind it change. EndpointSlices are the API objects that represent those changing backends in manageable groups. They are not just a new spelling for the legacy Endpoints object: the API allows the control plane and service proxies to update membership more efficiently as a Service scales, and it records endpoint conditions and topology used by traffic consumers.

EndpointSlices became stable in Kubernetes 1.21. For a Service with a selector, the control plane’s EndpointSlice controller watches the matching Pods and creates or updates slices. Each slice groups endpoints by address family, protocol, port configuration, and Service. A Service with many backends therefore commonly has multiple EndpointSlice objects, and a consumer must join all matching slices rather than assume one object contains the whole backend set.

Trace the service-to-pod relationship

The controller associates managed EndpointSlices with a Service by the kubernetes.io/service-name label and an owner reference. The control plane’s own slices also identify the manager through endpointslice.kubernetes.io/managed-by. Labels make the slices discoverable; the owner reference indicates lifecycle ownership. These are separate fields and should not be substituted for one another.

kubectl get service catalog -n production -o yaml
kubectl get endpointslices -n production \
  -l kubernetes.io/service-name=catalog -o wide
kubectl get endpointslices -n production \
  -l kubernetes.io/service-name=catalog -o yaml

Compare the Service selector with Pod labels, then inspect each slice’s addressType, ports, endpoints[].addresses, and conditions. A selector mismatch can produce a Service with no matching endpoints even though the Pods themselves are Running. A Pod that is not Ready normally is not treated as an available Service backend, unless the Service opts into publishing not-ready addresses. Also check named port resolution: a named targetPort is resolved against container ports, and different resolved values may require separate slices.

The default controller target is at most 100 endpoints per slice, configurable with the kube-controller-manager’s --max-endpoints-per-slice flag up to 1000. A large slice size reduces the number of API objects but increases the amount of data carried in an update; smaller slices create more objects and events. Managed slices are filled efficiently, but the controller does not actively rebalance them after every endpoint change. A slice count that looks uneven is not, on its own, evidence of a routing defect.

Interpret conditions during a rollout

EndpointSlices expose ready, serving, and terminating conditions. For a Pod-backed endpoint, serving reflects whether it is serving, and terminating becomes true when deletion begins. ready is effectively the shortcut for serving and not terminating, except for Services with spec.publishNotReadyAddresses: true, which report readiness differently for discovery purposes.

Service proxies normally avoid terminating endpoints. If every available endpoint is terminating, proxies can route to endpoints that are both serving and terminating so that a rolling update does not necessarily lose all Service traffic during the transition. The conditions let consumers make a better decision than a single boolean, but the data plane implementation determines how those conditions are used. A service mesh or third-party proxy must watch the API and implement compatible semantics itself.

apiVersion: discovery.k8s.io/v1
kind: EndpointSlice
metadata:
  name: catalog-manual-a
  namespace: production
  labels:
    kubernetes.io/service-name: catalog-manual
    endpointslice.kubernetes.io/managed-by: ops.example.net/manual
addressType: IPv4
ports:
  - name: http
    protocol: TCP
    port: 8080
endpoints:
  - addresses: ["192.0.2.44"]
    conditions:
      ready: true
      serving: true
      terminating: false

This is a shape example for a manually managed endpoint, not a universal production manifest. The address must be routable from the cluster and must not be loopback, link-local, or another Service’s virtual IP. Give every independently managed slice a unique managed-by value rather than using the controller’s reserved identity. A manually created slice needs an explicit ownership and cleanup process because the built-in selector controller is not responsible for its endpoint data.

Dual-stack and port grouping

An EndpointSlice represents one address type. A dual-stack Service can have IPv4 and IPv6 backends represented in different slices; a tool that reads only one addressType can report a partial set. Each slice’s ports apply to all of its endpoints. If Pods with the same named port resolve to different numeric target ports, the controller may create distinct slices so that a port tuple remains consistent within each slice.

When building a controller or custom observer, list all EndpointSlices in the Service’s namespace with the matching Service-name label. Do not select a slice by a guessed suffix or assume the slice name remains stable. Join endpoints from every matching slice, de-duplicate by the appropriate identity, and react to watch updates. Watch clients must handle resource versions, relists, and transient API errors using the standard Kubernetes API machinery rather than periodically guessing from object names.

EndpointSlices also carry node and zone fields that can help consumers understand topology. These fields report endpoint location; they do not by themselves guarantee zone-local routing. Topology-aware routing depends on cluster and proxy configuration. Treat the endpoint metadata as input for a policy, not as proof that traffic is being delivered according to that policy.

Services without selectors and compatibility boundaries

A selectorless Service can have EndpointSlices created directly to point at backends outside the cluster’s Pod selection mechanism. This is useful for an external database or a service implemented by a separately managed system, but the operator then owns address health, endpoint updates, and cleanup. The Kubernetes API server does not allow its Service proxy mechanisms to proxy to arbitrary endpoints that are not mapped to Pods; commands such as port-forwarding a selectorless Service can therefore fail even when ordinary data-plane routing is configured.

The older Endpoints API is deprecated. Kubernetes can mirror many user-created Endpoints objects to EndpointSlices for compatibility, but the mirror behavior is limited and deprecated alongside the older API. For a new controller or manually managed selectorless Service, write EndpointSlice objects directly and follow the current API reference. Do not create both a hand-managed EndpointSlice and an Endpoints object for the same backend set unless you have explicitly modeled their ownership and the duplication behavior.

Diagnose a missing backend without guessing

Work through the data path in layers. Confirm the Service namespace, selector, and publishNotReadyAddresses setting. List all associated slices and examine their address families and port names. Compare endpoint addresses with the Pod IPs and Ready conditions. Then inspect the service proxy or CNI-specific service dataplane and verify the destination port is listening. A matching slice is evidence that the control plane has published an endpoint; it is not proof that every node’s dataplane has programmed it or that network policy permits packets.

For a rollout, compare the number of ready, serving, and terminating endpoints over time while also checking Deployment and Pod conditions. If the slice contains the expected addresses but requests fail, investigate routing, policy, port translation, and application readiness separately. If an address never appears, investigate selectors and controller events before changing firewall rules. Keep snapshots of the Service, Pods, slices, and relevant events together so that the state can be compared at the same point in time.

The strongest validation is an end-to-end test that confirms a request reaches the intended backend while a Pod is added, becomes unready, and is terminated. Include IPv4 and IPv6 if the Service advertises both. Test selectorless endpoint refresh and cleanup if you use custom slices. This verifies the full contract: endpoint publication, consumer interpretation, data-plane programming, and application response.

Custom controllers that consume EndpointSlices should use the normal Kubernetes list-and-watch lifecycle. A controller can begin with a list, record the collection resource version, then watch for changes and relist after an expired watch or compaction response. It must reconcile the union of every slice for a Service, not process only the latest slice event as if that event were the complete backend set. If an endpoint moves between slices, a consumer may temporarily observe both the old and new representations while updates propagate; idempotent reconciliation and de-duplication avoid routing duplicates.

Do not store a long-lived endpoint cache without handling deletion tombstones, relists, and API permission failures. A controller that silently stops watching can keep routing to stale IP addresses even while kubectl get endpointslices shows current data. Expose the last successful sync time, watched resource version, number of slices consumed, and number of eligible endpoints. Those signals distinguish stale control-plane observation from a service proxy that has current input but incorrect dataplane programming.

Related:

Sources:

Comments