Load Balancers

A load balancer is a system that sits in front of a pool of servers and decides where each incoming request should go. Its basic job is to spread traffic so one machine does not get overloaded while others sit idle. In practice, a load balancer often becomes a control point for availability, security, and operational policy as well.

The simplest balancing strategy is round robin, where requests rotate through the available backends. That works when requests are broadly similar. Real systems often need more context. Least connections can help when request duration varies. Hash based routing can keep traffic from the same client, session, or key on the same backend. At layer 4, the balancer mainly uses transport details such as IP addresses and ports. At layer 7, it can inspect HTTP headers, paths, host names, or cookies and route based on application logic.

Health checks are what make the system resilient rather than merely distributive. A balancer continuously probes backends and stops sending traffic to nodes that fail. That lets a service survive instance crashes, failed deployments, or planned maintenance with far less visible downtime. Many balancers also terminate TLS, apply rate limits, block malformed traffic, and emit metrics, which is why they are often the first operational component added as a service grows.

The tradeoffs start once state enters the picture. If an application stores session state only in local memory, requests from the same user may need to hit the same server. Sticky sessions can achieve that, but they reduce balancing freedom and make failure handling awkward. A cleaner design is usually to externalise session state into a shared store so any healthy backend can serve the next request.

Retries are another subtle area. If the balancer retries a failed request automatically, that may hide transient faults and improve perceived reliability. It can also duplicate non idempotent operations such as payments or form submissions unless the application has idempotency controls. Queueing and connection pooling have similar tradeoffs. They smooth bursts, but if the backend is saturated for a prolonged period, a balancer can become a place where failure is delayed rather than prevented.

A load balancer is therefore not just a traffic splitter. It is part scheduler, part health monitor, and often part security edge. The best deployments are boring because they rely on simple routing rules, aggressive health checks, and backends designed to be interchangeable. When a system depends on request affinity, hidden local state, or opaque retry behaviour, the load balancer stops being a protective layer and starts becoming a source of hard to explain outages. Used well, it gives a service room to scale and fail gracefully instead of catastrophically.