Flux Kustomizations: Reconciliation Order, Health Gates, and Pruning

Flux Kustomization resources describe how the kustomize-controller builds and reconciles a source path into a Kubernetes cluster. Despite the shared name, a Flux Kustomization is not simply the kustomization.yaml file consumed by Kustomize. It is a controller-managed deployment unit with a source reference, reconciliation interval, pruning policy, health checks, dependency conditions, decryption and substitution options, and a status record that operators can inspect.

Production GitOps depends on making ordering and ownership explicit. A controller must install CRDs before custom resources that use them; platform services must become healthy before applications depend on them; pruning must not delete resources that another process owns; and a failed reconciliation must remain visible rather than looking like a successful Git commit. Flux provides these controls, but they only work when the repository is divided along meaningful lifecycle boundaries.

Separate the source from each reconciliation unit

A Flux source such as GitRepository or OCIRepository fetches an artifact. A Flux Kustomization refers to that source and a path within its artifact, then applies the built objects. Multiple Kustomizations can point to different paths in one repository. This separation allows source authentication, artifact revision, and apply status to be diagnosed independently.

Choose paths with clear ownership and deletion boundaries. A platform/crds unit, a platform/controllers unit, and an apps/checkout unit have different dependency and rollback semantics. Avoid one root Kustomization that owns every namespace, operator, and workload if a failed resource should not block unrelated applications. Conversely, do not create hundreds of tiny reconciliation units whose dependencies obscure the actual system graph.

The controller’s inventory tracks objects it applied. With pruning enabled, objects removed from the source may be deleted from the cluster. This is useful for convergence but makes path ownership critical. Do not include generated resources, manually managed resources, or another controller’s owned objects in a pruned path unless the ownership handoff is deliberate. Before turning on pruning for an existing tree, inspect the inventory and preview what the first reconciliation could delete.

Model dependency and readiness as separate gates

.spec.dependsOn orders Flux Kustomizations by their readiness condition. If a dependency is not Ready, the dependent Kustomization waits. This is appropriate for controller installation followed by custom resources, or a shared service followed by workloads. A dependency should represent a real prerequisite, not a convenient way to force every application into a single serial release chain.

Readiness depends on health checking. A dependency can be Ready because its resources were applied, but the operator may require a controller Deployment to become available before applying custom resources. Configure .spec.healthChecks to wait for selected resources, or .spec.wait: true to wait on all reconciled resources; when wait is enabled, explicit healthChecks are ignored. Use the narrower option when only particular resources define meaningful readiness, and be deliberate about timeouts for slow but healthy workloads.

apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: cert-manager-controller
  namespace: flux-system
spec:
  interval: 10m
  timeout: 5m
  path: ./clusters/prod/platform/cert-manager/controller
  prune: true
  sourceRef:
    kind: GitRepository
    name: platform-config
  healthChecks:
    - apiVersion: apps/v1
      kind: Deployment
      name: cert-manager
      namespace: cert-manager
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: cert-manager-issuers
  namespace: flux-system
spec:
  interval: 10m
  timeout: 5m
  path: ./clusters/prod/platform/cert-manager/issuers
  prune: true
  dependsOn:
    - name: cert-manager-controller
  sourceRef:
    kind: GitRepository
    name: platform-config

This example uses a resource-specific health check for the controller and a dependency edge for issuer objects. Confirm that the source object exists in the Flux system namespace and that cert-manager is the actual Deployment name and namespace in the installed chart. CRDs may require their own readiness strategy depending on how they are installed. Validate against the Flux CRDs installed in the cluster; API versions can change as the project evolves.

Dependencies can create a directed acyclic graph of deployment units. Review the graph for cycles and unintended long chains. A dependsOn readiness condition is based on the dependency’s last applied status; it does not continuously prove that every external dependency remains available forever. Runtime health monitoring still belongs in the application and platform observability system.

Treat pruning as an ownership operation

When prune: true, the controller can remove previously applied resources that disappear from the current artifact inventory. This aligns the cluster with Git but can be destructive if the source path or object set is accidentally narrowed. A rename may appear as deletion plus creation unless the resource’s identity remains stable or an explicit migration is designed. For stateful data and cluster-wide platform resources, review deletion semantics before pruning.

Flux supports per-resource prune controls through documented annotations. Use them sparingly and explain why an object is intentionally retained. If you disable pruning at the whole Kustomization level, removed objects can remain unmanaged in the cluster, creating drift and future ownership ambiguity. For handoffs, make a separate plan to transfer the resource to a new controller or owner and verify which inventory is responsible after the change.

During incident response, suspension pauses reconciliation but is not a rollback. It prevents the controller from applying newer changes while the team investigates. Preserve the source artifact revision, Kustomization conditions, controller logs, and object inventory. Reverting Git may be appropriate, but a reconciliation of a previous manifest still needs to account for irreversible side effects and schema transitions.

Observe status, conditions, and retries

Inspect Ready, Reconciling, and Stalled conditions rather than relying only on a single kubectl get status column. The status message and reason distinguish source artifact failures, build errors, apply errors, health-check failures, dependency-not-ready conditions, and prune failures. A Kustomization can be retrying while also not Ready; a successful Git push only proves that the source changed, not that the target cluster applied it.

The controller retries failed reconciliations with backoff. Repeated retries are useful for transient dependency recovery but can hide a permanent configuration defect if no alert watches the condition. Route failure conditions to the team’s notification system, and alert on time spent not Ready for critical units. Do not alert on every ordinary reconciliation; focus on prolonged failure, stalled work, and unexpected revision drift.

When investigating a stuck resource, first verify the source artifact revision and path, then run the same Kustomize build locally with the repository’s pinned toolchain. Compare the built objects to live objects, inspect admission errors and field ownership conflicts, check dependency readiness and selected health resources, and inspect controller logs. If pruning is involved, identify the inventory and calculate exactly which object would be deleted before changing the prune annotation or source tree.

Use impersonation and decryption deliberately

Flux Kustomizations can be configured to reconcile using a service account, which provides a way to reduce the permissions available to an individual deployment unit. The service account must exist in the appropriate cluster and namespace context, and its RBAC policy should cover only the resources in that unit. Do not give every Flux controller cluster-admin simply because initial bootstrap was convenient; carve privileges by team and lifecycle boundary where operationally feasible.

Secret decryption and post-build substitution add more inputs to reconciliation. Keep decryption keys out of Git, limit which Kustomizations can decrypt a secret tree, and make the source of each substituted value observable. A manifest that resolves only because a secret exists in one cluster may fail elsewhere. Test a clean bootstrap path and document the ordering for keys, CRDs, controllers, and encrypted resources.

Build a recoverable rollout structure

Organize repository paths so that the smallest safe unit can be reviewed, applied, and rolled back independently. Put cluster bootstrap, CRDs, controllers, shared services, and application releases into separate Kustomizations when their dependencies or ownership differ. Add only dependencies that express hard ordering requirements. Use health checks for resources whose readiness determines whether dependent configuration can safely apply.

Before enabling a new production Kustomization, check its rendered object inventory, namespace and cluster scopes, service account, prune setting, timeout, source reference, and dependency graph. Test both normal reconciliation and deletion in a disposable cluster. For a CRD or controller upgrade, make the old/new compatibility window explicit and do not assume that reverting an image digest reverses a storage-version change.

Flux is a continuous controller, not a one-shot deployment command. The operational contract is the combination of Git state, source artifact, controller configuration, reconciliation inventory, admission result, and resource health. When those pieces are visible and boundaries are intentional, GitOps can provide repeatable change delivery without turning an accidental path edit into a cluster-wide delete operation.

Related:

Sources: