Kubernetes 'couldn't find key in Secret': How to Fix It

What you see

kubectl get pods -l app=web

NAME                   READY   STATUS                       RESTARTS   AGE
web-7b5c9d6f4c-qz8lm   0/1     CreateContainerConfigError   0          40s

The container never starts, so kubectl logs shows nothing useful. The reason is in the Events.

Step 1: read the Events

kubectl describe pod -l app=web

Warning  Failed  5s (x4 over 40s)  kubelet  Error: couldn't find key API_KEY in Secret default/app-secret

This tells you three things: the Secret name (app-secret), its namespace, and the key the pod asked for (API_KEY). The Secret itself was found - otherwise the error would be secret "app-secret" not found.

Step 2: list the keys the Secret really has

kubectl describe secret app-secret

Data
====
api-key:  12 bytes

describe shows key names and sizes but hides the values. To print all keys as JSON:

kubectl get secret app-secret -o jsonpath='{.data}'

Here the Secret has api-key, while the pod asks for API_KEY. Keys are case-sensitive, and - and _ are different characters.

Step 3: correct secretKeyRef.key

The env var name and the Secret key are two separate things. The env var name is what your app reads; the key is what the Secret stores:

env:
- name: API_KEY              # the env var the app reads - keep it
  valueFrom:
    secretKeyRef:
      name: app-secret       # the Secret
      key: api-key           # was: API_KEY - must match a real key

Apply and wait for the new pod:

kubectl apply -f deployment.yaml
kubectl rollout status deploy/web --timeout=90s

Step 4: verify inside the container

kubectl exec deploy/web -- printenv API_KEY

If it prints the value, the reference is correct. Do not paste real secret values into shared chat or tickets while you debug.

Should you change the Deployment or the Secret?

Either side can be wrong. Change the Deployment when the Secret is shared or managed elsewhere (other apps may use the same key). Change the Secret when it was created by hand with a typo and nothing else uses it:

kubectl create secret generic app-secret \
  --from-literal=API_KEY='value' \
  --dry-run=client -o yaml | kubectl apply -f -

Why apply does not catch this

Kubernetes does not check Secret names or keys when you apply a Deployment. The kubelet checks them when it creates the container. So a wrong key applies cleanly and only fails on the pod. After every apply, check pod status and Events, not only the apply output.

Making a key optional

If the app can run without the value, mark the reference optional:

secretKeyRef:
  name: app-secret
  key: api-key
  optional: true

The pod starts and the env var is simply not set when the key is missing. Use this only for truly optional settings.

Prevent it in the future

• Use the same key names in every environment (dev, staging, production).
• When a tool such as External Secrets or Sealed Secrets generates the Secret, check which key names it writes.
• If you want every key as an env var with the same name, use envFrom with secretRef instead of listing keys one by one.