Opscraft
Deployment strategies · Lesson 3 of 10 · 4 min read

Zero-downtime mechanics

A rolling update with the default settings can still drop requests. Zero downtime needs the whole request path to cooperate.

Getting traffic only to ready pods

  • Readiness probe — removes a pod from Service endpoints until it can serve (warm caches, DB connections established). Without it, traffic hits pods that are still starting → 502/503s.
  • Startup probe — for slow starters, so liveness doesn't kill them during boot.
  • With the AWS Load Balancer Controller and target-type: ip, use pod readiness gates (label the namespace elbv2.k8s.aws/pod-readiness-gate-inject=enabled) so a pod isn't considered ready until the ALB target group health check passes — otherwise old pods may be removed before new ones are healthy in the ALB.
Enlarge diagramService / ALBpod v2: Readyin endpointspod v2: startingnot Ready: no traffic
Figure 1. Traffic only goes to pods that pass their readiness probe. With the ALB, readiness gates also wait for the target to be healthy.
Swipe sideways to see all of it, or tap to open it full screen.

Graceful shutdown: the termination sequence

When a pod is deleted during a rollout, two things happen in parallel:

  1. The kubelet runs the preStop hook, then sends SIGTERM to the containers.
  2. The endpoint controller removes the pod from Service endpoints; kube-proxy on every node and the ALB (via the LB controller) update — this takes seconds.

If the app exits immediately on SIGTERM, requests still being routed to it during step 2 fail. The fix:

spec:
  terminationGracePeriodSeconds: 45
  containers:
  - name: orders
    lifecycle:
      preStop:
        exec:
          command: ["sh", "-c", "sleep 15"]     # let endpoints/ALB deregister first

Then the app should, on SIGTERM: stop accepting new connections, finish in-flight requests, close DB connections, exit. Total time must stay under terminationGracePeriodSeconds (default 30 s), after which SIGKILL arrives.

Enlarge diagrampod deletedendpointskube-proxy + ALB catch upcontainerpreStop: sleep 15, still servingSIGTERM: drainexitKeep the total under terminationGracePeriodSeconds (e.g. 45 s).
Figure 2. Why preStop sleep matters: endpoint removal and SIGTERM start together. The sleep keeps the old pod serving until load balancers stop sending to it.
Swipe sideways to see all of it, or tap to open it full screen.

On the ALB, the target group deregistration delay (default 300 s) controls how long draining connections are allowed; tune it to your request durations (e.g. 30–60 s) so rollouts aren't needlessly slow.

Other pieces

  • PodDisruptionBudgets protect availability during node drains and cluster upgrades (not during Deployment rollouts — those use maxUnavailable).
  • Enough capacity for surge pods: cluster autoscaler/Karpenter headroom, or rollouts stall in Pending.
  • Session state outside pods (Redis, JWT) — sticky sessions pin users to old pods and break when they disappear.
  • Long-lived connections (WebSockets, gRPC streams): longer grace periods, server-sent "go away"/reconnect signals, clients that reconnect with backoff.
  • Brief 502s during rollouts almost always mean one of: no readiness probe, no preStop delay, app not handling SIGTERM, keep-alive timeout mismatch between ALB and app, or deregistration slower than pod termination.
Enlarge diagramNo readiness probetraffic before the app is readyNo preStop delaypod exits while still in endpointsSIGTERM ignoredin-flight requests cutKeep-alive mismatchapp timeout shorter than ALB idleSlow deregistrationpod gone before the ALB stops
Figure 3. Brief 502s during a rollout almost always come from one of these.
Swipe sideways to see all of it, or tap to open it full screen.