Caching: Where Caches Live
A cache keeps a copy of data somewhere faster to reach than its source. Learn the layers where caches live, from the browser to the database, what to cache at each, and how to measure whether a cache works.
In a nutshell
- A cache trades freshness for speed: it answers from a nearby copy instead of the slower source.
- Caches live at every layer: browser, CDN, reverse proxy, application (in-process or Redis/Memcached) and database.
- Cache data that is read often and changes rarely, and decide how stale each item may be.
- Measure the hit ratio: a cache that rarely hits only adds latency and complexity.
Why caching works
Real traffic is uneven: a small share of items gets most of the requests (the home page, today's top products, a celebrity's profile). Keep those few items in fast memory and most requests never reach the database. Memory is also far faster than disk or a network round trip to a busy database, as the latency numbers topic shows.
- ~100 nsread from main memory
- ~0.5 msround trip within a data centre
- ms to sa database query under load
The layers
Caches at every layer
Diagram as text
- Browser → CDN: miss
- CDN → Reverse proxy: miss
- Reverse proxy → App server: miss
- App server → Redis: lookup
- App server → Database: miss
| Layer | What to cache | Controlled by |
|---|---|---|
| Browser | Static assets, some API responses | Cache-Control, ETag headers |
| CDN | Images, scripts, whole pages for anonymous users | Headers, CDN rules, purges |
| Reverse proxy | Rendered pages, API responses (Varnish, Nginx, LiteSpeed) | Proxy config, headers |
| In-process | Small, very hot, rarely changing data (config, feature flags) | Your code; each server has its own copy |
| Distributed cache | Query results, sessions, computed objects (Redis, Memcached) | Your code; shared by all servers |
| Database | Recently used pages of tables and indexes | The database itself (buffer pool) |
What to cache, and for how long
- Good candidates: read-heavy and expensive data (product pages, profile summaries, results of heavy queries, rendered HTML fragments, responses from slow third-party APIs).
- Poor candidates: data that changes on almost every read, data that must be exactly current (account balances during a payment), and data read only once.
- Staleness budget: for each item, decide how old it may be. A product description can be an hour old; stock levels maybe seconds; a price at checkout must be checked against the source.
Track the hit ratio (hits ÷ lookups), latency and memory. A 95% hit ratio means only 1 in 20 requests reaches the database; at 50% the cache may cost more than it saves.
Common mistakes
- Caching everything without measuring the hit ratio.
- Caching data with no expiry and no invalidation, so it is wrong forever.
- Caching personalised responses at a shared layer (CDN, proxy), showing one user's data to another.
- Assuming the database can absorb the full load if the cache disappears.
Interview questions
Try answering out loud before you open each one.
Where would you add caching in this design?
Walk from the client inwards: HTTP caching and a CDN for static and public content; a reverse proxy or CDN for anonymous pages; Redis for hot objects and expensive query results with TTLs; and keep the database's working set in memory. For each, say what is cached, how stale it may be, and how it is invalidated.
What is a good hit ratio?
It depends on the workload, but read-heavy caches commonly aim for 90% or more. What matters is the effect: database load and latency at the 99th percentile. Low hit ratios suggest the wrong keys, TTLs that are too short, or a cache too small for the working set.
Check yourself
Related topics
Further reading
- Caching best practices AWS
- HTTP caching MDN Web Docs
Your notes
Saved in this browser as you type.
Finished this topic?
Mark it read to track your progress, then carry on with the next one.