Load Balancers
A load balancer spreads requests across many servers, skips unhealthy ones and lets you add capacity without users noticing. Learn layer 4 vs layer 7, health checks, sticky sessions and how to keep the load balancer itself up.
In a nutshell
- A load balancer gives many servers one address, spreads the load, and routes around failed servers with health checks.
- Layer 4 balances TCP/UDP connections (fast, any protocol); layer 7 reads HTTP (routing by path, headers, cookies).
- Avoid sticky sessions where possible: keep servers stateless so any server can take any request.
- Run load balancers in pairs or use a managed one, so the balancer is not a single point of failure.
What a load balancer does
Spreading load and skipping failures
All steps Health checks run all the time; a server that fails them stops getting traffic until it recovers.
- All users connect to one address: the load balancer.
- It sends each request to one of the healthy servers.
- Server C fails its health checks, so it is taken out of rotation until it passes again.
Diagram as text
- All users connect to one address: the load balancer.
- It sends each request to one of the healthy servers.
- Server C fails its health checks, so it is taken out of rotation until it passes again.
- One address, many servers: add or remove servers without changing anything users see.
- Health checks: regularly calls each server (for example
GET /health) and stops sending traffic to failures. - Even load: no server is overwhelmed while others sit idle.
- Zero-downtime deploys: drain one server, update it, put it back, repeat.
Layer 4 vs layer 7
The names come from the OSI network model. Layer 4 is the transport layer (TCP/UDP); layer 7 is the application layer (HTTP).
| Layer 4 | Layer 7 | |
|---|---|---|
| Sees | IP addresses and ports | Full HTTP requests: path, headers, cookies |
| Decides by | Connection (source, destination) | Content (/api → API servers, /images → image servers) |
| Speed | Very fast, little processing | More work per request |
| Can do | Balance any TCP/UDP protocol (databases, games) | TLS termination, routing, header rewrites, caching, retries |
| Example | Cloud network load balancers | Nginx, HAProxy in HTTP mode, cloud application load balancers |
Sticky sessions
If a server keeps a user's session in its own memory, the user must keep reaching the same server. Load balancers can arrange this with sticky sessions (affinity), usually through a cookie.
Keeping the load balancer up
A single load balancer in front of ten servers is a single point of failure. Common fixes:
- An active-passive pair sharing a floating IP: the standby takes over the address if the active one fails.
- Managed cloud load balancers, which the provider runs redundantly across zones.
- DNS across several load balancers, often one per region, with health-checked DNS in front.
Common mistakes
- Health checks that only test "is the process up" rather than "can it serve a real request".
- Health checks so strict that one slow shared dependency takes every server out of rotation at once.
- Relying on sticky sessions instead of making servers stateless.
- Forgetting the load balancer's idle timeout when using WebSockets or long requests.
Interview questions
Try answering out loud before you open each one.
Layer 4 or layer 7 for a web app with an API and a website on one domain?
Layer 7: it can route /api to API servers and everything else to web servers, terminate TLS and add headers. Layer 4 cannot see paths. A layer 4 balancer is still useful in front of non-HTTP services or as a very fast first tier.
How do health checks work and what should they check?
The balancer periodically calls a health endpoint on each server, removes servers that fail several checks in a row and adds them back after several successes. A good check verifies the server can do real work (for example reach its database) without being so strict that one shared dependency's hiccup removes every server at once.
Check yourself
Related topics
Further reading
- What is load balancing? Cloudflare Learning Center
- HAProxy documentation HAProxy
Your notes
Saved in this browser as you type.
Finished this topic?
Mark it read to track your progress, then carry on with the next one.