6 Load Balancing Algorithms

A load balancer makes a scheduling decision every time a request arrives. That decision affects latency, cache locality, failure handling, and how quickly a weak backend hurts the rest of the fleet. These six algorithms are common because each one favours a different tradeoff.

1. Round robin

Round robin sends each new request to the next backend in a fixed rotation. If there are four healthy instances, traffic is distributed A, B, C, D, then the cycle repeats.

It is simple and works well when backends are stateless and similarly sized. The weakness is that equal turn taking does not mean equal work. One request may be trivial while the next starts an expensive report query, so the algorithm assumes a level of homogeneity that may not exist.

2. Sticky round-robin

Sticky round-robin adds session affinity. The first request is assigned in round-robin order, but later requests from the same client, cookie, or session key keep returning to the same backend.

This helps when a service keeps local session state, in-memory cache entries, or WebSocket context. The tradeoff is imbalance. A few heavy users can overload one instance while others sit idle, and failover becomes more visible because a lost backend usually means a lost session.

3. Weighted round-robin

Weighted round-robin gives some backends more turns than others. A node with weight 4 receives roughly twice as many requests as a node with weight 2.

That is useful when the fleet mixes hardware sizes, container limits, or canary pools. The limitation is that weights reflect expected capacity, not live behaviour. If a large node is degraded, the balancer may still send it extra traffic because the static configuration says it should handle more.

4. Hash-based routing

Hash-based routing applies a hash function to an attribute such as client IP, session ID, or URL path and maps the result to a backend. The main benefit is deterministic placement.

Requests with the same key land on the same instance, which helps cache locality and can reduce the need for central session storage. The hard part is rebalancing. When nodes are added or removed, many keys may move at once unless the system uses consistent hashing or a related scheme.

5. Least connections

Least connections is a dynamic algorithm. The balancer tracks how many active connections each backend holds and sends new work to the least busy one.

This usually handles mixed request durations better than plain rotation, especially with long-lived connections. Even so, connection count is only a proxy for real load. Ten idle connections may be cheaper than two CPU-heavy requests, so the metric is useful but imperfect.

6. Least response time

Least response time sends traffic towards the backend that has been replying fastest, sometimes combined with connection count or outstanding request count.

The advantage is faster adaptation when one instance slows down because of garbage collection, disk stalls, or dependency latency. The risk is oscillation. If the policy chases the fastest node too aggressively, it can create feedback loops and overload the newest favourite.

There is no universally best algorithm. Stateless fleets with uniform capacity often do fine with round robin, while bursty or uneven workloads need dynamic signals. The right choice depends on traffic shape, backend behaviour, and which failure modes you are willing to accept.