Cache Eviction Policies
Cache eviction policies through TTL, LRU, LFU, FIFO, and workload fit.
A cache only helps while it has room for the right data. Once memory fills up, the eviction policy decides what stays hot and what gets discarded. That choice matters because different workloads exhibit different locality patterns. A policy that is excellent for one access shape can be wasteful for another.
The simplest policy is TTL, or time-to-live. Every item expires after a configured duration regardless of how often it is used. TTL works well when data naturally becomes stale after a known interval, such as session state, DNS records, or short-lived API responses. Its weakness is bluntness. A heavily used item can disappear at the worst possible moment just because the clock ran out.
LRU, least recently used, assumes temporal locality. If something was accessed recently, it is more likely to be needed again soon. This matches many application workloads, which is why LRU is so popular. The downside is scan resistance. A large one-time scan can push out useful items simply because it touched many entries once.
LFU, least frequently used, keeps items with the strongest long-term popularity. It is useful when some keys are consistently hotter than others, such as product metadata or configuration lookups. The challenge is bookkeeping. Frequency counters consume space, need ageing logic, and can make it hard for newer items to compete with historical favourites.
MRU, most recently used, sounds backwards but has legitimate use cases. In some sequential or streaming workloads, the most recently touched item is the least likely to be needed again soon. Evicting recent entries can therefore outperform LRU when the access pattern is effectively marching forward.
Segmented LRU adds a protection mechanism. New entries start in a probationary segment and only move into a protected segment after proving they are genuinely useful. This helps distinguish one-hit wonders from items with sustained reuse. Many production caches use variations on this idea because pure LRU often promotes the wrong data too eagerly.
In practice, eviction is rarely the only policy that matters. Admission policy, expiration, object size, and write amplification matter too. A tiny popular key and a huge rarely used blob should not necessarily be treated as equals. That is why modern systems often combine strategies rather than implementing textbook policies in isolation.
The best way to choose is to examine the workload. Is reuse short-term or long-term? Are scans common? Is staleness bounded by time? Do large objects crowd out more valuable small ones? Cache design is full of heuristics, but the core principle is straightforward: your eviction policy should reflect why items become valuable in the first place. Otherwise the cache spends memory preserving the wrong history.