Consistent Hashing

Consistent hashing is a placement strategy for distributed systems where servers are expected to join, leave, or fail over time. The classic alternative is simple modulo hashing: choose a node with hash(key) mod N. That works only while N stays constant. The moment you add or remove a node, almost every key is remapped, cache hit rates collapse, and large amounts of data must move.

Consistent hashing changes the objective. Instead of trying to compute a fixed bucket number directly, it places both nodes and keys on the same logical ring. A key belongs to the first node encountered when moving clockwise around that ring. When a new node is added, only the keys that fall between it and its predecessor need to move. Most other keys remain untouched. That limited remapping is the main benefit.

The ring itself is only a mental model. In implementation, both node identifiers and keys are hashed into a large numeric space. Because real hash functions are not perfectly even at small scale, production systems usually add virtual nodes. Rather than placing each physical server on the ring once, they place it many times under different derived identities. This smooths the distribution and reduces the chance that one unlucky server owns a disproportionately large segment.

Replication is often layered on top. Instead of storing a key on only the first node clockwise, the system stores it on the next few nodes as well. That improves durability and availability, but it also introduces placement rules about rack awareness, zone diversity, and replica repair. Consistent hashing solves where data should start; it does not solve all the operational questions that follow.

The technique is widely used in caches, key-value stores, CDNs, and some load balancers because these systems care deeply about minimising churn. A cache cluster is the clearest example. If adding one node caused every cached object to move, the cluster would effectively cold-start on every scale event. Consistent hashing limits the blast radius so only a slice of the cache is repopulated.

It is not a magic answer to every distribution problem. Hot keys can still overload a node because placement fairness does not guarantee traffic fairness. If one key receives ten thousand times more traffic than others, the owning node becomes hot regardless of how elegant the ring looks. Systems often need request replication, load-aware routing, or explicit hot-key mitigation on top.

You should also remember that consistent hashing makes topology changes cheaper, not free. Rebalancing still consumes network, disk, and CPU. Large stateful systems may stream data for hours after node changes. Good operational tooling therefore matters: rate limiting for rebalance traffic, visibility into ownership changes, and guardrails that prevent too many nodes changing at once.

The core idea is simple enough to explain on a whiteboard, but its value is operational. Consistent hashing is about keeping most of the system stable while the cluster changes around it. In distributed infrastructure, that stability is often more important than the elegance of the algorithm itself.