Load balancer
The traffic cop that spreads incoming requests across many servers.
A load balancer sits in front of a group of servers and distributes incoming requests among them. Users all connect to one address (the balancer's); behind it, the same website or API runs on two, ten or a thousand identical machines, and each request is sent to one of them. The user never knows which.
It solves two problems at once. Capacity: one server can only handle so many requests per second, so you add more and split the load. Availability: the balancer checks each server's health every few seconds and stops sending traffic to one that fails, so a crashed machine means a slightly busier fleet instead of an outage. It also lets you update servers one at a time with no downtime.
Balancers choose servers with simple rules (round robin, fewest connections, by region) and many also terminate HTTPS, so the TLS certificate lives in one place. In the cloud this is a managed service: AWS has the Application Load Balancer and the Network Load Balancer, and platforms like Kubernetes build one in. Combined with autoscaling (add servers when busy, remove them when quiet), a load balancer is what lets a site survive being on the news.
Keep in mind
- One public address in front, many identical servers behind.
- Health checks pull a broken server out of rotation automatically.
- It is what makes zero-downtime deploys and autoscaling possible.
- Often also terminates HTTPS, so certificates live in one place.
