How to Set Up Centralized Logging for Kubernetes with the EFK Stack

Centralized logging pulls every pod’s logs into one searchable place — this walks through the EFK stack (Elasticsearch, Fluent Bit, Kibana), a common alternative to the original ELK stack that swaps in the lighter-weight Fluent Bit for log collection.

Step 1: deploy Elasticsearch

helm repo add elastic https://helm.elastic.co
helm install elasticsearch elastic/elasticsearch -n logging --create-namespace

Elasticsearch is the storage and search engine underneath everything else in this stack — logs ultimately land here as indexed, searchable documents.

Step 2: deploy Fluent Bit as a DaemonSet

helm install fluent-bit fluent/fluent-bit -n logging

Running as a DaemonSet means exactly one Fluent Bit instance runs per node, each responsible for collecting logs from every pod on that specific node — the standard pattern for cluster-wide log collection.

Step 3: configure Fluent Bit to find and parse container logs

[INPUT]
    Name              tail
    Path              /var/log/containers/*.log
    Parser            docker
    Tag               kube.*

[OUTPUT]
    Name  es
    Host  elasticsearch-master
    Port  9200
    Index kubernetes_cluster

This tails the standard container log path every node exposes and ships parsed entries to Elasticsearch.

Step 4: enrich logs with Kubernetes metadata

[FILTER]
    Name                kubernetes
    Match                kube.*
    Kube_URL             https://kubernetes.default.svc:443

The Kubernetes filter attaches pod name, namespace, and labels to each log entry — without this, you’d have raw log text with no easy way to filter by which specific workload produced it.

Step 5: deploy Kibana for querying and visualization

helm install kibana elastic/kibana -n logging
kubectl port-forward -n logging svc/kibana-kibana 5601:5601

Step 6: create an index pattern in Kibana

Kibana → Stack Management → Index Patterns →
  Create index pattern → "kubernetes_cluster*"

Step 7: search logs across the entire cluster

Kibana → Discover →
  kubernetes.namespace_name: "production" AND
  kubernetes.labels.app: "myapp"

This is the actual payoff — searching and filtering logs from every relevant pod at once, rather than running kubectl logs against each one individually and manually correlating results.

Step 8: set an index lifecycle policy to control storage growth

Kibana → Stack Management → Index Lifecycle Policies

Logs accumulate quickly at cluster scale — configuring automatic rollover and deletion of old indices prevents Elasticsearch storage from growing unbounded.

Step 9: build a dashboard for recurring queries

Kibana → Dashboard → Create new dashboard →
  add saved searches and visualizations

Saving common queries (error rates by service, a specific namespace’s recent activity) as a dashboard turns one-off investigation queries into an always-available operational view.

Why centralized logging becomes necessary past a certain cluster size

kubectl logs works fine for a handful of pods, but becomes genuinely unusable once a workload runs across dozens of replicas or a cluster hosts many services — there’s no way to search across pods, correlate a request across services, or retain history past a pod’s own lifecycle without something like this stack aggregating logs into one durable, searchable store.