Why and where to cache
Caching is the cheapest performance and resilience win in most systems, and the source of some of the strangest bugs: users seeing old pages, one user seeing another's data, or a database falling over the moment a cache restarts. Interviewers ask about it in system design rounds and in troubleshooting questions. This topic covers the layers of caching, the HTTP headers that control browsers and CDNs, how CDNs such as CloudFront work, application caching with Redis or Valkey, and the failure modes you need to design against.
The layers of caching
A request can be answered from a cache at several points before it reaches the database:
| Layer | Example | Typical TTL | Who controls it |
|---|---|---|---|
| Browser | HTTP cache, service worker | Seconds to a year | Cache-Control from your server |
| CDN edge | CloudFront, Cloudflare, Fastly | Seconds to days | Headers plus CDN cache policy |
| Reverse proxy | Nginx proxy_cache, Varnish |
Seconds to minutes | Proxy config |
| Application | Redis, Valkey, Memcached, in-process maps | Seconds to hours | Your code |
| Database | Buffer pool, query cache, read replicas | Automatic | The database |
The closer to the user, the faster and cheaper the hit, and the harder it is to take back a wrong answer: you can delete a Redis key instantly, but a response cached in a million browsers for a year stays there.
Hit ratio, freshness and what to cache
- Hit ratio = hits ÷ (hits + misses). A CDN at 95% sends one request in twenty to the origin; dropping to 80% quadruples origin traffic. Track it per path type.
- TTL (time to live) decides how long a copy counts as fresh. After that it is stale: the cache must check with the origin (revalidate) or fetch again.
- Cache what is read often, changes rarely, and is the same for many users: images, scripts, product catalogues, rendered public pages, configuration, the results of expensive queries.
- Don't share-cache personalised or sensitive responses (account pages, anything after login) unless the user is part of the cache key, and never cache
Set-Cookieresponses at a shared layer. - Every cache is a copy, so every cache is a consistency decision: how stale can this data be before it hurts someone?