← All cheatsheets

Containers

Kubernetes Troubleshooting — Pending, CrashLoopBackOff, and friends

Diagnose stuck and crashing pods and broken Services: confirm context and namespace, read events, pull logs, inspect workload state, and restore Service routing.

Docs-checkedkubernetestroubleshootingcrashloopbackoffpendingprobes

Quick reference — reach for these first

Describe podkubectl describe pod POD -n NSContainer state + Events at the bottom — start here
Previous logskubectl logs POD -n NS --previousStdout/stderr from the container's last (crashed) run
Namespace eventskubectl get events -n NS --sort-by=.lastTimestampScheduler and kubelet reasons, newest last
Service endpointskubectl get endpoints SVC -n NSEmpty means the selector matches no ready pods
DNS test podkubectl run dnstest --rm -it --image=busybox:1.36 --restart=Never -n NS -- nslookup SVCResolve a Service name from inside the cluster

Troubleshooting path: context first, recovery last

Kubernetes troubleshooting pathFive left-to-right stages from confirming context and namespace, through events, logs, and workload state, to Service recovery.1. Context + NScurrent-context2. Eventsdescribe / get events3. Logslogs --previous4. Workload statePending / Crash / Pull5. Service recoveryendpoints / selectorconfirm target before any change

Text equivalent: A left-to-right flow of five stages. Stage 1, Context + namespace: kubectl config current-context and set the namespace. Arrow to stage 2, Events: kubectl get events and describe pod. Arrow to stage 3, Logs: kubectl logs with --previous. Arrow to stage 4, Workload state: inspect Pending, CrashLoopBackOff, ImagePullBackOff, and probe failures. Arrow to stage 5, Service recovery: check endpoints, fix the selector or label mismatch, then roll the deployment. A note under the first box reads: confirm the target before any change.

Common tasks

Task · Command · Confirm

TaskCommandConfirm
Confirm which cluster you are pointed at before touching anythingkubectl config current-contextPrints the cluster you expect (e.g. minikube, not a prod context)
Confirm the active namespacekubectl config view --minify -o jsonpath='{..namespace}'; echoPrints NS; empty output means the 'default' namespace
Pin the namespace so later commands target itkubectl config set-context --current --namespace=NSkubectl config view --minify | grep namespace shows NS
Snapshot pod health in the namespacekubectl get pods -n NS -o wideSTATUS and RESTARTS columns reveal which pods are unhealthy and on which node
Pending pod: read the scheduler's reasonkubectl describe pod POD -n NS | sed -n '/Events/,$p'FailedScheduling line names the blocker: Insufficient cpu/memory, a taint, or an unbound PVC
Pending pod: check node capacity and taintskubectl describe nodes | grep -E 'Taints|Allocated resources' -A6Confirms whether any node has room / tolerable taints for the pod
Pending pod: check for an unbound PVCkubectl get pvc -n NSA claim stuck in Pending blocks the pod until a PV or StorageClass satisfies it
CrashLoopBackOff: read the crash output from the last runkubectl logs POD -n NS --previous --tail=50The exit message or stack trace from before the restart is visible
CrashLoopBackOff: read the exit code and reasonkubectl get pod POD -n NS -o jsonpath='{.status.containerStatuses[0].lastState.terminated.exitCode}{" "}{.status.containerStatuses[0].lastState.terminated.reason}'; echoExit code is non-zero (137=OOMKilled, 1/2=app error, 127=bad command)
ImagePullBackOff/ErrImagePull: see the pull errorkubectl describe pod POD -n NS | grep -A3 -iE 'Failed|ErrImagePull'Message shows the cause: 'manifest unknown' (bad tag) or '401 Unauthorized' (auth)
ImagePullBackOff: verify the imagePullSecret exists and is referencedkubectl get pod POD -n NS -o jsonpath='{.spec.imagePullSecrets}'; echo; kubectl get secrets -n NSA referenced secret of type kubernetes.io/dockerconfigjson is present for private registries
Probe failures: find why a Running pod is not Readykubectl describe pod POD -n NS | grep -iE 'liveness|readiness|unhealthy'Unhealthy events show the probe failure: 'HTTP probe failed with statuscode: 500' or 'connection refused'
Missing endpoints: compare the Service selector against pod labelskubectl get endpoints SVC -n NS; kubectl get svc SVC -n NS -o jsonpath='{.spec.selector}'; echo; kubectl get pods -n NS --show-labelsSelector key/values match at least one Ready pod's labels; mismatch = empty ENDPOINTS
[CHANGES WORKLOAD] After the fix, roll the deployment and watch it settlekubectl rollout restart deploy/DEPLOY -n NS && kubectl rollout status deploy/DEPLOY -n NSrollout status reports 'successfully rolled out'; new pods reach Running/Ready

A Service returns no response — empty endpoints traced to a label mismatch

endpoints

1
Confirm the target context and namespace before changing anything
kubectl config current-context
# minikube
2
curl to the Service hangs; check whether it has any endpoints
kubectl get endpoints web -n shop
# NAME   ENDPOINTS   AGE
# web    <none>      10m
3
Endpoints are empty — read what labels the Service selects on
kubectl get svc web -n shop -o jsonpath='{.spec.selector}'; echo
# {"app":"web"}
4
List the pods' labels — they say app=frontend, not app=web
kubectl get pods -n shop --show-labels
# NAME          READY  STATUS   LABELS
# web-7d9f...   1/1    Running  app=frontend,pod-template-hash=7d9f
5
Fix the mutable side: point the Service at the pods' real label. (A Deployment's selector is immutable — you cannot re-label its pods in place.)
kubectl patch svc web -n shop -p '{"spec":{"selector":{"app":"frontend"}}}'
# service/web patched
6
The Service now matches the pods — endpoints populate and it routes
kubectl get endpoints web -n shop
# NAME   ENDPOINTS        AGE
# web    10.1.2.3:8080    12m

Troubleshooting — symptom → first check → next check

SymptomFirst checkEstablishesNext check
Pod stuck in Pending, never scheduleskubectl describe pod POD -n NS (read Events)Whether the scheduler rejected it for resources, a taint, node selector, or an unbound PVCkubectl describe nodes (Allocated resources + Taints) and kubectl get pvc -n NS
CrashLoopBackOff — container restarts in a loopkubectl logs POD -n NS --previousThe app exits on startup; the previous-run log holds the actual errorCheck the terminated exitCode (137=OOM, 127=bad cmd) and the container's env/config/mounts
ImagePullBackOff / ErrImagePullkubectl describe pod POD -n NS (grep Failed)Image name/tag is wrong, registry is unreachable, or auth failedFix the tag, or verify the imagePullSecret and registry credentials
Pod Running but never Ready; traffic not routedkubectl describe pod POD -n NS (liveness/readiness events)The readiness probe is failing, so the pod is excluded from endpointsExec in and curl the probe path/port; fix the path or loosen initialDelay/timeout thresholds
Service reachable but returns nothing; curl hangs or refuseskubectl get endpoints SVC -n NS<none> endpoints = selector matches no Ready pod (label mismatch) or all pods unreadyCompare spec.selector to pod labels (kubectl get pods --show-labels); align them
In-cluster name resolution fails (service DNS not found)kubectl run dnstest --rm -it --image=busybox:1.36 --restart=Never -n NS -- nslookup SVCWhether DNS itself is broken vs the app's own networkingkubectl get pods -n kube-system -l k8s-app=kube-dns and kubectl logs for CoreDNS errors

References

kubectl v1.31, any conformant cluster (minikube/EKS/GKE), bash shell; replace CAPS placeholders.