Kubernetes Resource Metrics API v1: A Production Migration Guide

Kubernetes 1.37 promotes the Resource Metrics API to the stable metrics.k8s.io/v1 version. The API exposes CPU and memory usage for Nodes and Pods; it is consumed by commands such as kubectl top and by resource-based autoscaling components. The graduation provides a stable API contract, but it does not install a metrics implementation, guarantee that metrics are fresh, or make every client switch versions at once.

For a production migration, separate three concerns: whether the cluster’s metrics implementation serves the stable endpoint, whether each client version can consume it, and whether the sampled data is good enough for the operational decisions that depend on it. In Kubernetes 1.37 specifically, kubectl top can prefer v1, while the HorizontalPodAutoscaler controller still supports only v1beta1. A safe rollout therefore keeps both versions working during this transition.

What the stable API does and does not promise

The API has two resource types: NodeMetrics for Node CPU and memory usage, and PodMetrics for Pod usage with a per-container breakdown. In Kubernetes 1.37, metrics.k8s.io/v1 has the same resource types and fields as metrics.k8s.io/v1beta1; this graduation changes the API version’s stability status, not the collected values or their meaning.

Each metrics object includes a timestamp and window that describe the interval from which the usage was collected. Node and container records contain resource quantities such as CPU and memory. For Pods, the individual container entries are collected within the same time window. Consumers should consider the timestamp and window when judging freshness; a response can be structurally valid while reflecting an old measurement.

The Resource Metrics API is intentionally small. It provides the CPU and memory usage needed for basic inspection and resource-based scaling. It is not a general time-series store, a long-term history API, or a replacement for custom metrics (custom.metrics.k8s.io), external metrics (external.metrics.k8s.io), or a Prometheus-based observability pipeline. Do not infer request rate, queue depth, latency percentiles, or business throughput from this API.

Understand the collection and serving path

The metrics implementation typically collects data from kubelets and serves it through the Kubernetes API aggregation layer. Metrics Server is the reference implementation, but a cluster can use another implementation of metrics.k8s.io. The API aggregation layer exposes that implementation to Kubernetes API clients through an APIService registration.

kubelet resource statistics
          │
          ▼
metrics implementation (for example, Metrics Server)
          │
          ▼
API aggregation layer and metrics.k8s.io APIService
          │
          ├── kubectl top
          ├── HorizontalPodAutoscaler / VerticalPodAutoscaler
          └── other API clients

A running implementation Pod is only one component in this path. The APIService must be registered and available, the aggregator must be able to reach the backend, the backend must be able to collect from the kubelets, and clients must request a served API version. Diagnose each boundary separately rather than treating a healthy Deployment as evidence that autoscaling can read usable values.

The API aggregation layer and an implementation are required regardless of whether a user requests v1 or v1beta1. The stable version does not require a feature gate in Kubernetes 1.37, but a metrics implementation must actually serve v1.metrics.k8s.io and register its APIService before clients can retrieve the stable endpoint.

Check discovery and endpoint health

Begin with API discovery. It shows which versions the cluster advertises:

kubectl get --raw /apis/metrics.k8s.io/ | jq .

For a v1.37 migration, confirm that both v1 and v1beta1 are present while compatibility is required. Then inspect the version-specific APIService registrations:

kubectl get apiservice v1.metrics.k8s.io
kubectl get apiservice v1beta1.metrics.k8s.io
kubectl describe apiservice v1.metrics.k8s.io
kubectl describe apiservice v1beta1.metrics.k8s.io

An Available=False condition points to the aggregated API path, commonly the Service reference, endpoint, or serving certificate configuration. The precise cause is in the APIService condition message and related events; do not guess from the condition alone. A healthy APIService still does not prove that the kubelet collection is current, so test the resource endpoints next.

kubectl get --raw /apis/metrics.k8s.io/v1/nodes | jq '.items[0]'
kubectl get --raw /apis/metrics.k8s.io/v1/namespaces/default/pods | jq '.items[0]'
kubectl get --raw /apis/metrics.k8s.io/v1beta1/nodes | jq '.items[0]'

Compare timestamp, window, usage, and the object names in returned records. If discovery lists a version but a request fails, check the APIService, aggregation path, and implementation logs. If requests return data but timestamps lag the expected collection cadence, inspect the implementation’s collection status and kubelet access rather than blaming API-version negotiation.

The resource quantities are not a substitute for reading a full monitoring time series. kubectl top is a convenient point-in-time inspection tool; it does not show retained history, a service-level objective, or the causal context needed for a capacity incident. For sustained investigations, correlate the API snapshot with a monitoring system that retains scrape history, control-plane metrics, and workload events.

Plan for client compatibility in Kubernetes 1.37

Kubernetes 1.37 has a transition detail that matters in production: kubectl top supports both versions, prefers metrics.k8s.io/v1 when available, and falls back to v1beta1 when v1 is not served. The HPA controller in Kubernetes 1.37 still consumes only metrics.k8s.io/v1beta1; support for selecting the version through discovery is planned but is not available in that release.

This means serving only the stable endpoint can break HPA resource metrics on 1.37 even though kubectl top works. Keep v1beta1 available until every important client has been checked for the exact Kubernetes and client-library versions it runs. Treat the HPA behavior as release-specific and verify it against the control-plane version during future upgrades rather than extrapolating from this 1.37 note.

Before changing a metrics implementation, inventory the consumers:

kubectl top versions used by operators and automation.
HPA controllers in every cluster version and managed-control-plane release.
VPA components, where deployed.
Operators, adapters, dashboards, or scripts that call the aggregated API directly.
Client libraries with generated API types or hard-coded group-version paths.
Health checks and policy rules that assert an exact served version.

Do not assume that upgrading the control plane also upgrades a separately installed Metrics Server or custom implementation. The cluster administrator or managed Kubernetes provider may own different parts of the release. Confirm the supported API version matrix with the platform’s documentation and then test both data endpoints in a representative cluster.

Roll out both versions, then measure adoption

For a metrics implementation you operate, deploy a version that serves both v1 and v1beta1 during the Kubernetes 1.37 transition. Confirm its release supports Kubernetes 1.37 and that the registered APIService for each group-version reports available. If your provider manages the implementation, request its support matrix and migration guidance rather than editing provider-owned resources.

A safe rollout sequence is:

Record the current cluster version, implementation version, APIService conditions, and representative metric timestamps.
Update the implementation in a staging or canary cluster and preserve both API versions.
Confirm discovery advertises both versions and query Node and Pod metrics through both paths.
Exercise kubectl top with the client versions used by operators and automation.
Check a real HPA using CPU or memory metrics and verify it continues to receive fresh data and make expected scaling decisions.
Compare requests and errors for each version through implementation telemetry or API audit data where available.
Update clients that can safely use the stable version, but retain beta compatibility for clients that still require it.
Remove beta serving only after the cluster’s supported consumers and control-plane releases no longer depend on it.

The API’s schema is unchanged at graduation, which reduces migration risk, but version negotiation and APIService registration are still observable behavior. Use canary requests against both endpoints to detect a version-specific routing or serializer issue before rolling out to every cluster. Maintain a rollback path that restores the previous implementation and APIService configuration without losing the ability to serve resource metrics.

Keep autoscaling signals and observability distinct

An HPA using CPU utilization needs resource requests for the relevant containers, and it needs usable resource metrics. A missing or stale measurement can prevent the controller from calculating a scale recommendation. API version availability is only one part of the control loop: workload resource requests, metric freshness, HPA conditions, scaling policies, and spare cluster capacity still matter.

Keep alerts for metrics collection and API serving separate from application scaling objectives. Useful checks include APIService availability, successful collection from Nodes, age of returned sample timestamps, and HPA conditions such as inability to fetch metrics. A healthy kubectl top result for one Pod is not proof that every Node is reporting or that the HPA can read the endpoint it needs.

Similarly, do not route application latency or queue depth through the Resource Metrics API. Those signals require a custom or external metrics API implementation and are usually sourced from an application or monitoring system. Keeping the API boundaries clear avoids designing an autoscaling policy around a field that the resource metrics API does not expose.

Troubleshoot by layer

When a resource metric is missing, follow the data path in order:

Check API discovery to see which group-versions are actually served.
Check the relevant APIService condition and its reported message.
Query the exact Node or Pod endpoint through the API server.
Inspect timestamp and window to distinguish missing data from stale data.
Verify the implementation can reach and collect from kubelets on affected Nodes.
Check component logs and version compatibility for the implementation and kubelets.
Test the actual consumer, including the Kubernetes 1.37 HPA beta-version requirement.

This layered process separates negotiation failures from collection failures. For example, kubectl top may be using v1 while a controller still requests v1beta1; a missing endpoint can therefore affect one consumer but not another. Conversely, both endpoints can respond while reporting old values because the collector is not updating. Record the failing request path and the source timestamps before changing versions or restarting components.

Production acceptance checklist

Cluster version and metrics implementation support are documented and tested together.
The implementation serves the API versions required by the installed clients.
APIService objects for required versions report Available=True.
Node and Pod requests return expected objects with plausible timestamp and window values.
kubectl top works with the operator’s actual client version.
HPA works with the exact control-plane version; on Kubernetes 1.37, v1beta1 remains served.
API availability, collection health, and stale samples have distinct signals and runbooks.
Resource Metrics API use is not confused with Prometheus retention or custom/external metrics.
The rollout has a tested rollback route that preserves compatible API endpoints.

The Kubernetes 1.37 graduation makes metrics.k8s.io/v1 the stable contract for resource usage. Operationally, success comes from serving that contract through a healthy aggregation path while keeping every actual consumer compatible and every scaling signal fresh. Validate the endpoints and the autoscaler independently before removing the legacy version.

Related:

Sources: