TTL, failover and load balancing on AWS
DNS failover is bounded by TTL
Route 53 failover routing flips the answer when a health check fails (after the check's failure threshold — e.g. 3 × 30 s). Then clients switch only when their cached answer expires:
- With TTL 300, clients may keep using the failed endpoint for up to 5 minutes after Route 53 switches — plus detection time.
- With TTL 60, about a minute.
So failover records should use ~60 s TTLs, health checks with fast intervals (10 s) and sensible thresholds, and clients that retry and re-resolve on connection failure.
Why ELB IPs change and why aliases matter
Load balancers scale out and replace their nodes, so the IP addresses behind an ALB's DNS name change over time. Two consequences:
- Never hardcode an ALB's IPs in an A record — use an alias (Route 53 tracks the changes) or a CNAME to the ELB name.
- Clients that cache resolved IPs too long (JVM
-1, long-lived connections that never reconnect) eventually talk to IPs that no longer belong to your load balancer → failures after the LB scales.
DNS switch vs load balancer switch
For blue-green cutovers and rollbacks, switching at the load balancer (listener rules, weighted target groups) takes effect in seconds for all clients. A DNS switch waits on caches and TTL-ignoring clients, and rollback waits again. DNS-based weighted routing also shifts traffic slowly and unevenly because a few large resolvers serve many users and all their users move together.
When you need fast global failover without DNS caching delays, AWS Global Accelerator gives static anycast IPs and shifts traffic between regional endpoints in seconds.
Lessons from incidents
"We changed DNS during the incident but users kept failing for 30 minutes" is a common postmortem finding. Action items: inventory critical records and pre-set appropriate low TTLs, document cutover steps, and prefer LB-level or Global Accelerator failover for critical paths.