SYSTEM DESIGN · QUESTION 28
How would you design a distributed cache?
On this page
I would first clarify the read/write pattern, freshness needs, object sizes, and acceptable behavior during cache failure. A typical cache-aside design checks the cache, loads misses from a durable store, and populates the cache with a bounded lifetime. Then I would address eviction, hot keys, invalidation, stampedes, and availability. A cache reduces repeated work but introduces stale-data and operational trade-offs.
LET'S MAKE IT SIMPLE
First, the idea in plain English
A cache keeps copies of frequently used data somewhere faster to reach than the main database. When an application needs an item, it can reuse the cached copy instead of repeating an expensive lookup. In a distributed system, many servers may access the cache, and cached values can become stale or disappear. The design is therefore about deciding what can be cached and how the system behaves when that copy is wrong or missing.
A USEFUL COMPARISON
Picture it this way
A kitchen keeps commonly used ingredients near the cook rather than walking to a distant storeroom for every dish. If an ingredient changes in the storeroom, the nearby copy must be refreshed or discarded. A computer cache faces the same freshness problem, plus concurrent requests and failures across machines.
FOLLOW THE JOURNEY
Cache-aside serves hits quickly and loads misses from storage
Trace reads, misses, writes, expiration, and failure behavior between an application, cache, and durable store.
Cache-aside is one pattern, not a universal choice. Write-through, write-behind, and invalidation events change ordering and failure semantics.
An application reads the cache; a hit returns the value, while a miss loads it from durable storage and populates the cache with an expiry policy.
How to read this visual
Cache-aside is one pattern, not a universal choice. Write-through, write-behind, and invalidation events change ordering and failure semantics.
UNDER THE HOOD
How it works, step by step
Clarify constraints
Ask about access patterns, staleness tolerance, data size, latency, and failure expectations.
Why it matters: Cache policy depends on what the product can safely serve.
Choose a cache pattern
Select cache-aside or another read/write strategy and define the source of truth.
Why it matters: The pattern determines when data is loaded and how updates propagate.
Control growth and hotspots
Set expiration and eviction, and plan for hot keys and concurrent misses.
Why it matters: Unbounded or uneven demand can overload the cache or origin.
Define invalidation and outages
Specify refresh behavior, fallback, and consistency guarantees.
Why it matters: Stale data and cache failure are normal design cases, not exceptions.
OUTSIDE THE TEXTBOOK
A practical example
A product catalog page is read frequently and changes less often. With cache-aside, the service checks a shared cache, loads the product from the database on a miss, then stores it with a freshness policy. Product updates invalidate or refresh affected keys. If the cache is unavailable, the service may fall back to storage with safeguards so a traffic surge does not overwhelm the database.
GO A LITTLE DEEPER
Technical details that matter
- TTL expiration bounds staleness but does not guarantee immediate invalidation after a write.
- A cache stampede occurs when many requests miss the same key at once; request coalescing, jittered expiry, or controlled refresh can reduce duplicated origin work.
- Hot keys can overload a cache shard; replication or local caching may help but complicates consistency.
- Cache failure policy is workload-specific: fail open to the database, serve stale data, or reject requests with load shedding.
MAKE A GOOD CHOICE
When is this useful?
- Repeated reads of data that is expensive to calculate or fetch.
- Reducing load on a database or remote service when stale results are acceptable within a defined bound.
- Serving low-latency reads for popular, reasonably cacheable objects.
If data must always be strongly current, read directly from the source of truth or use a design with explicit consistency guarantees. Precomputation or database indexing may address the bottleneck without a separate cache.
WATCH OUT FOR
Common misunderstandings
- Treating TTL as complete cache invalidation.
- Caching sensitive data without tenant isolation, access controls, or suitable retention.
- Assuming the cache will always reduce load even when keys are high-cardinality or miss-heavy.
REAL-WORLD TRADE-OFFS
Limitations and failure cases
- Cached values can be stale or inconsistent across regions.
- The cache adds another dependency, cost, and operational failure mode.
- A sudden cache flush or outage can create a thundering herd against the origin.
WHAT TO REMEMBER
Key takeaways
- 1Start with access patterns and acceptable staleness.
- 2Define the durable source of truth and cache update strategy.
- 3Plan for eviction, hot keys, stampedes, and cache outages.
- 4Caching trades repeated work for freshness and operational complexity.
PRACTICE THE FOLLOW-UP
What might an interviewer ask next?
How do you prevent a cache stampede?
Coalesce concurrent loads for the same key, refresh before expiry, add jitter, or limit origin concurrency. The right combination depends on acceptable staleness.
How do you invalidate cached data after a write?
Options include deleting or updating keys, versioned keys, short TTLs, or change events. Each has ordering and failure trade-offs.
What happens if the cache goes down?
Define whether to fall back, serve stale values, or shed load. A naive full fallback can overload the database, so use limits and recovery planning.
SAY IT OUT LOUD
How to explain this in an interview
“I would clarify the freshness and load requirements, then consider cache-aside: check the cache, load from the source of truth on a miss, and populate with an expiry policy. The important design work is invalidation, hot keys, stampedes, and cache failure. I would specify whether stale data is acceptable and protect the origin during fallback, because caching adds consistency and operational trade-offs.”
Connect this question to the roadmap topic it supports.