A load balancer distributes incoming requests to application servers, databases, or other computing resources. It returns the selected resource's response to the client. Health checks keep traffic away from unhealthy nodes. Distribution prevents one node from taking all traffic and removes a single server as the only point of failure.
- Client
The client sends a request to the load balancer.
- Health
Node C is unhealthy in the example and receives no traffic.
- Policy
The balancer chooses healthy Node A or B according to policy: round robin, least loaded, weighted, random, or a session key.
- Response
The illustrated response path returns through the load balancer. Actual network topology can differ.
Routing policy applies among eligible healthy nodes.
How to read this model
A traffic selector feeding three independently gated workers. The object is a teaching analogy. Inspecting a part highlights its role; it does not execute or simulate the software. Cabinet trays are records, workpieces are data, and latches are checks. Unless an explanation states a quantity, compartments and sample workpieces do not denote data counts. Part dimensions are not a quantitative scale.
Concept redrawn from Scalable system design patterns.
- Layer 4
Use transport information such as IP address and port to choose S1 or S2.
- Layer 7
Inspect HTTP information such as path and headers to route to an API pool or static-content pool.
The available routing information determines the policy you can apply.
How to read this model
A transport address selector beside an HTTP-aware routing selector. The object is a teaching analogy. Inspecting a part highlights its role; it does not execute or simulate the software. Cabinet trays are records, workpieces are data, and latches are checks. Unless an explanation states a quantity, compartments and sample workpieces do not denote data counts. Part dimensions are not a quantitative scale.
Hardware load balancers exist, but software such as HAProxy and NGINX can fill the role. Common routing policies include random choice, least loaded, round robin, weighted round robin, and a session or cookie key.
Load balancers can terminate TLS. They decrypt client traffic, so backend nodes do not each need an X.509 certificate or the CPU cost of a TLS handshake. Protect the hop from the load balancer to the backend when that network is not trusted.
Session persistence routes one client's requests to the same instance. It can help legacy applications that keep sessions in process memory. A shared session store, such as a database or cache, usually makes horizontal scaling less fragile.
Layer 4 and layer 7
Layer 4 load balancing uses transport information such as source and destination IP addresses and ports. It forwards TCP or UDP packets, often with network address translation, without reading the application payload. It is fast and protocol-agnostic, but cannot route on a URL or cookie.
Layer 7 load balancing terminates the connection and parses application data. It can route by HTTP path, headers, message fields, or cookies. For example, it can send /video/* to video nodes and billing requests to a hardened pool. Parsing and terminating connections use more CPU and memory.
| Choice | Can inspect | Strength | Cost |
|---|---|---|---|
| Layer 4 | IP, port, transport state | Low overhead and works for many protocols | Cannot make content-aware decisions |
| Layer 7 | HTTP path, headers, cookies, payload metadata | Precise routing and TLS termination | More processing and protocol coupling |
Horizontal scaling
Scaling out adds commodity nodes. It often costs less and provides more failure isolation than scaling up one large machine. A horizontally scaled application node should be stateless. Store sessions, profile pictures, and other user data in a shared system rather than local disk or process memory.
Each added application node also creates more downstream connections. Size the database, cache, and connection pools for the new concurrency.
Costs and failure modes
A load balancer can become a bottleneck if its capacity or configuration is wrong. One load balancer is still a single point of failure. Multiple balancers, active-passive or active-active, improve availability but add health checks, routing, and failover state. See availability patterns and reverse proxies for adjacent choices.