load balancing

Picture a supermarket with one queue feeding many checkout lanes. A coordinator at the head of the line sends each shopper to an open register so no single cashier is swamped while others sit idle. Load balancing is that coordinator for network traffic: a device or service that spreads incoming requests across a pool of servers, so the work is shared, no one server is overwhelmed, and the whole system can handle far more than any single machine.

A load balancer sits in front of a group of identical servers and, for each arriving connection or request, picks one to handle it using a server selection strategy. Simple strategies include round robin (rotate through the servers in turn) and random; smarter ones use least connections (send to the server with the fewest active connections) or weighted choices (give more traffic to bigger servers). Crucially, a load balancer also does health checks — it stops sending traffic to a server that fails to respond — so a crashed or overloaded machine is quietly taken out of rotation. The balancing can happen at layer 4 (the transport layer), where it forwards TCP/UDP connections by IP and port without looking inside, or at layer 7 (the application layer), where it reads the actual HTTP request and can route by URL, cookie, or host.

Why it matters: load balancing is what turns one server's worth of capacity into a fleet's, and it is the foundation of both scalability (add more servers behind the balancer to grow) and availability (one server can die without taking the site down). It is the engine inside reverse proxies and CDN edges, and inside cloud load balancers. The honest caveats: the balancer itself must not become a single point of failure (so it is usually replicated), and balancing is only easy when servers are interchangeable — if a user's session lives on one specific server, you need session affinity (sticky sessions) or shared state, which complicates the simple picture.