Cache Failure Modes

A cache improves latency only if misses, expiries, and failures are controlled. Once the cache becomes part of the critical path, it can amplify traffic spikes, hide stale data, or push too much load into the database. Most cache incidents come from a few recurring failure modes.

1. Thunder herd on mass expiry

A thunder herd happens when many cached keys expire at roughly the same time and a large population of callers falls through to the backing store together. A common cause is setting identical TTLs for a whole batch of entries during deployment or warm-up.

The database now sees a sudden wave of concurrent misses instead of the smoother load profile the cache was meant to provide. If recomputation is expensive, the application can spend seconds rebuilding data while user latency climbs.

The usual mitigations are TTL jitter, staggered refresh, and request coalescing. TTL jitter spreads expiries over time. Stale-while-revalidate lets one worker refresh while others temporarily serve slightly stale values. Request coalescing ensures only one miss per key reaches the database at a time.

2. Cache penetration on non-existent keys

Cache penetration happens when callers repeatedly ask for keys that do not exist in the cache or the database. That may be caused by bad input, bots, or a product surface where users probe unknown IDs. Because the object is missing everywhere, every request becomes a miss and keeps hitting storage.

Negative caching is the simplest defence. Store a short-lived null marker so repeated misses do not keep reaching the database. A Bloom filter can help even earlier by cheaply answering “definitely not present” for many impossible keys.

The tradeoff is false positives and invalidation. Bloom filters can say “maybe present” for a missing key, so they reduce load rather than guaranteeing correctness. Negative cache entries also need shorter TTLs than real data, or newly created objects may stay invisible for too long.

3. Cache breakdown on a hot key

Cache breakdown, often called a dogpile on a hot key, is more concentrated than a thunder herd. One extremely popular key expires, then a large fraction of traffic stampedes the database for that single record. Product home pages, exchange rates, feature flags, and celebrity profiles are typical examples.

The fix is to treat hot keys differently from ordinary keys. Some teams never let them expire automatically and refresh them in the background instead. Others use mutex locking per key so only one caller repopulates the value. Replication can help read scale, but it does not remove the rebuild problem if all replicas still have to query the same origin.

4. Cache crash or partial outage

When the cache cluster is unavailable, the danger is not only slow reads from the source of truth. It is the sudden removal of a protective buffer. A database sized for a 5 percent miss ratio may fail quickly if 100 percent of requests bypass the cache.

This is why cache clients often need degraded-mode behaviour. Circuit breakers can stop endless retries to the dead cache. Rate limits can protect the database. Serving stale local copies for selected endpoints can keep the product usable while the shared cache recovers. High-availability cache clusters reduce the chance of a full outage, but they do not remove the need for a fallback plan.

What good cache design looks like

A healthy cache design assumes misses are normal, expiries are noisy, and the cache itself can fail. That leads to practical rules: spread TTLs, guard the origin with admission control, deduplicate rebuild work, cache negative results carefully, and decide in advance which paths may serve stale data.

The point of a cache is not just speed. It is controlled load. When the load-control part is missing, the fast path becomes the failure path.