Teams move secrets out of the cluster into an external store — and the first thing they create is a new secret: the token that grants access to that store. A Vault token in a ConfigMap, an AWS key in an operator variable — «a secret to access the secrets» puts the problem right back where it started. According to the Verizon DBIR 2025, more than 60 % of cloud incidents begin with leaked credentials. The working combination for 2026 closes the loop entirely: External Secrets Operator syncs secrets from the external store, and workload identity removes the last static key — the one the operator uses to authenticate to the store.
Why a native Secret is not a source of truth
A Kubernetes Secret is stored in etcd as base64 — that is encoding, not encryption. Until encryption at rest is enabled explicitly (--encryption-provider-config, with a KMS provider in production), reading etcd or one of its backups directly bypasses RBAC altogether. But even an encrypted native Secret makes a poor source of record: it has no versioning, no audit trail, no automatic rotation, it is tied to one cluster — multi-cluster turns into copy-paste — and putting it into Git under GitOps is out of the question: plaintext lives in commit history forever.
The conclusion: the «true» secret lives in an external store. The default is Vault or its BSL-free fork OpenBao (API-compatible, under the Linux Foundation); the alternatives are AWS Secrets Manager, GCP Secret Manager and Azure Key Vault. Cloud-managed stores hide a pricing trap: at $0.40 per secret per month, 500 secrets turn into roughly $2,400 a year before API charges — self-hosted Vault can be cheaper at scale.
ESO: one CRD facade over 50+ backends
External Secrets Operator (a CNCF project) is the universal glue between the store and the cluster. A ClusterSecretStore describes the backend connection; an ExternalSecret is a declarative request: «sync this path into a regular K8s Secret». The application knows nothing about Vault — it reads the secret through the familiar valueFrom: secretKeyRef. Switching backends means editing the SecretStore, not the applications.
On top of the basic pair sit ClusterExternalSecret (fanning one secret out into many namespaces for multi-tenant platforms) and PushSecret (the reverse direction: from the cluster into the store). What lives in Git is only the ExternalSecret — a reference to a path, never the value.
Workload identity: no credentials for the credentials
One bootstrap secret remains: what does the operator itself use to authenticate to the store? The answer is — nothing static. The mechanism is identical across clouds: the pod receives a projected ServiceAccount token, audience-bound to the OIDC endpoint of the cloud, and the cloud IAM validates it directly. On EKS this is IRSA — the eks.amazonaws.com/role-arn annotation on the ServiceAccount, with an IAM trust policy pinned to the exact namespace and serviceaccount pair — or the more modern EKS Pod Identity. On GKE it is Workload Identity, on AKS it is Azure Workload Identity: the same pattern with a different annotation.
For Vault, the same role is played by the kubernetes auth method: the Vault role is hard-bound to the ServiceAccount name and namespace (bound_service_account_names plus bound_service_account_namespaces), and the token lives for an hour. Compromising a pod in a neighbouring namespace yields nothing. The one caveat: the federated role must be least-privilege. Granting it AdministratorAccess «for the demo» reassembles every risk of a static key — just without the key itself.
Rotation is a process, not an event
refreshInterval on an ExternalSecret sets how often the store is polled: 1–5 minutes means fast rotation at the cost of backend load; 1–24 hours is gentle on the backend, but the secret goes stale. A changed value in the store reaches the K8s Secret automatically, yet pods read secrets at startup — a restart is required. That gap is closed by stakater/reloader: the reloader.stakater.com/auto annotation on a Deployment, and the restart happens by itself whenever the Secret changes.
The top tier is dynamic secrets in Vault: database credentials are generated per request, each with its own TTL and automatic revocation. Rotation stops being a problem at all — every instance of the application holds its own short-lived credentials.
Checklist and pitfalls
Vault KV v2: the remoteRef.key must contain the /data/ segment — for example secret/data/prod/app/db. Without it ESO reports SecretSynced: True while the Secret stays empty: the nastiest failures are the ones that look like success.
creationPolicy: Owner on the target — otherwise an ownership conflict appears on a manual kubectl edit. A long refreshInterval for short-lived credentials defeats the purpose of rotation. One shared Secret for twenty applications makes RBAC coarse: revoking access for a single service becomes impossible.
Env vars versus files: environment variables are visible in /proc, in crash dumps and in kubectl describe; the 2026 trend is file-mounted secrets via the CSI driver for the most sensitive material. Encryption at rest in etcd is the base layer ESO does not replace: the synced secret still lands in etcd.
The bottom line: the store answers «where do secrets live», ESO answers «how they reach the pod», and workload identity answers «how all of this runs without a single long-lived key». The layers cannot be removed from the bottom: without the first two, workload identity has nothing left to protect. And how a secret avoids landing in the repository along the way — from secret scanning to admission policies.