Load Balancing: Algorithms, Types, and Best Practices

Load balancing distributes incoming network traffic across a pool of backend servers to improve application availability, scalability, and performance. By preventing any single server from becoming a bottleneck and rerouting around failed servers, load balancers are essential components of fault-tolerant, horizontally scaled architectures. The global load balancing market exceeded $7 billion in 2024 (MarketsandMarkets), driven by cloud adoption and the demand for always-on applications.

Why Load Balancing Matters

Without load balancing, traffic hitting a single server past its capacity causes degraded performance (high latency, dropped connections) or outright failure. Load balancers enable horizontal scaling — adding more servers to handle increased demand — and provide failure isolation by continuously monitoring server health and removing unhealthy nodes from the pool.

Availability and Fault Tolerance

A load balancer acts as a single virtual IP (VIP) behind which a pool of real servers operates. If any server fails, the load balancer stops sending it traffic and distributes the load among the remaining healthy servers. Combining this with multi-availability-zone deployment provides high availability: an entire data center can fail while the load balancer continues directing traffic to the remaining zones.

Scalability

Load balancers decouple the client-facing endpoint from the backend capacity. Operators add or remove servers during peak or off-peak periods without changing the endpoint DNS record or client configuration. Auto-scaling groups (AWS Auto Scaling, Google MIGs, Azure VMSS) integrate with cloud load balancers to automatically provision servers based on CPU utilization, request count, or custom metrics.

Performance Optimization

Beyond request distribution, load balancers perform SSL termination (offloading TLS decryption from backend servers), TCP buffering (reducing the impact of slow clients), compression (gzip/brotli of response bodies), and caching (serving frequently requested static content directly).

Load Balancing Algorithms

Round Robin

The simplest algorithm: requests are distributed sequentially to each server in the pool. Round robin works well when servers have identical capacity and request processing times are uniform. It does not account for current load — a server already handling slow requests continues to receive new traffic. Weighted round robin assigns a numeric weight to each server; higher-weight servers receive proportionally more requests.

Least Connections

New requests are sent to the server with the fewest active connections. Least connections is more intelligent than round robin because it accounts for varying request durations. A server handling a database query that takes five seconds accumulates more concurrent connections than one serving a static file. Least connections naturally compensates for this imbalance.

IP Hash

A hash of the client IP address (and optionally port) determines which server receives the request. IP hash ensures that a given client consistently reaches the same backend server — essential for session persistence when the application stores session state locally. The downside: adding or removing servers rehashes client assignments, potentially breaking persistence for many users until their sessions expire.

Least Response Time

Requests are sent to the server with the lowest average response time, accounting for both active connections and observed latency. This algorithm adapts to varying server performance due to garbage collection, cache warming, or resource contention. NGINX Plus and HAProxy support least-time variants.

Random with Backoff

A request is assigned to a randomly selected server. If that server is overloaded, the load balancer tries another. This reduces thundering-herd problems compared to deterministic algorithms during failover events.

Layer 4 vs. Layer 7 Load Balancing

Layer 4 load balancers (network layer) forward traffic based on the destination IP address and TCP/UDP port. They see IP packets and transport segments but not application payload. Advantages: simple, fast, low overhead, supports any TCP/UDP protocol. Examples: AWS Classic Load Balancer, Linux Virtual Server (LVS).

Layer 7 load balancers (application layer) inspect the full HTTP request — URL path, headers, cookies, query parameters. They can route requests based on content: /api/* to one pool, /static/* to another, Host: admin.example.com to a restricted pool. Layer 7 enables gRPC load balancing, rate limiting per URL, and request rewriting. Examples: NGINX, HAProxy, AWS Application Load Balancer.

Load Balancer Deployment Types

Hardware Appliances

F5 BIG-IP, Citrix ADC (formerly NetScaler), and A10 Thunder are dedicated appliances optimized for throughput and feature depth. They provide hardware-accelerated SSL, DDoS protection, and global server load balancing. Hardware dominates in on-premises data centers with existing vendor relationships but carries high upfront cost and operational overhead.

Software Load Balancers

NGINX (and NGINX Plus), HAProxy, and Apache Traffic Server run on commodity hardware and cloud instances. They offer flexibility, open-source editions, and deep configurability. HAProxy powers approximately 60 percent of the top 100,000 websites according to W3Techs surveys. Software load balancers are suitable for the majority of production workloads.

Cloud Managed Load Balancers

AWS Elastic Load Balancing (ALB, NLB, GLB), Google Cloud Load Balancing, and Azure Load Balancer are fully managed services. They handle provisioning, scaling, SSL certificate management, and health check configuration. Cloud load balancers are priced per hour plus data processing fees and integrate natively with their respective cloud ecosystems.

Health Checks

Health checks determine server availability. Active checks send probes (HTTP GET, TCP connect, ICMP ping) at configurable intervals. A server that fails a configurable number of consecutive checks is marked unhealthy and removed from the pool. Passive checks observe actual traffic behavior — if a server returns multiple consecutive 5xx errors, it is automatically taken out of rotation. The combination of active and passive checks provides robust failure detection. Unhealthy servers are retried periodically to detect recovery.

SSL Termination and End-to-End Encryption

SSL termination decrypts incoming TLS connections at the load balancer, sending unencrypted traffic to backends. This offloads CPU-intensive decryption from application servers and allows the load balancer to inspect requests for content-based routing. For security-sensitive workloads, SSL passthrough forwards encrypted traffic to backends, preserving end-to-end encryption. SSL re-encryption terminates at the load balancer, then initiates a new encrypted connection to backends — offering both inspection and backend encryption.

Session Persistence

When an application stores session data in local memory (not a shared store like Redis), the load balancer must ensure that all requests from a given user go to the same server. Methods include: sticky cookies (the load balancer sets a cookie identifying the backend), IP hash (deterministic mapping), and cookie insertion. Session persistence reduces load balancing effectiveness and complicates failover — if a server fails, all its sessions are lost. Stateless application design (storing session state in an external cache) eliminates this constraint.

Global Server Load Balancing

GSLB distributes traffic across geographically distributed data centers. DNS-based GSLB returns different IP addresses to clients based on their geographic region or the current health and load of each site. More sophisticated solutions use anycast (the same IP prefix announced from multiple locations) or HTTP redirects. GSLB is the foundation of active-active multi-region architectures.

FAQ

Q: What is the difference between a load balancer and a reverse proxy?
A: A reverse proxy is a type of load balancer that terminates client connections and proxies them to backend servers. All load balancers provide distribution; reverse proxies additionally offer URL rewriting, caching, and authentication. The terms overlap — NGINX is both a reverse proxy and a load balancer.

Q: Should I use Layer 4 or Layer 7 load balancing?
A: Use Layer 4 if you need raw throughput for non-HTTP protocols or minimal latency overhead. Use Layer 7 if you need content-based routing, gRPC load balancing, or HTTP-specific features like rate limiting and sticky sessions.

Q: How many back-end servers can a single load balancer handle?
A: It depends on the software/hardware. HAProxy can proxy hundreds of thousands of connections and balance across thousands of servers. NGINX handles 500,000+ concurrent connections per instance. Cloud load balancers scale transparently — AWS ALB handles millions of requests per second.

Q: What happens if all back-end servers are unhealthy?
A: Well-designed load balancers serve a configurable “sorry page” or fail open (forward to a degraded-but-alive server). The application should degrade gracefully. The worst behavior is returning 502 Bad Gateway to every request without explanation.

Q: Do load balancers support WebSocket and gRPC?
A: Yes. NGINX, HAProxy, and cloud load balancers support WebSocket upgrade headers. gRPC requires Layer 7 HTTP/2 load balancing. AWS ALB supports gRPC natively; NGINX requires grpc_pass directives.

Internal Links

• CDN Guide — how CDNs complement load balancers at the edge
• HTTP and HTTPS Guide — TLS termination and HTTP/2/3 considerations with load balancers
• Network Troubleshooting — diagnosing load balancer health check failures

References

• RFC 7230, “HTTP/1.1 Message Syntax and Routing” (reverse proxy semantics)
• HAProxy. “Configuration Manual.” HAProxy Documentation. https://docs.haproxy.org/
• NGINX. “NGINX Load Balancing — HTTP and TCP Load Balancer.” NGINX Documentation. https://docs.nginx.com/
• AWS. “Elastic Load Balancing — Amazon Web Services.” AWS Documentation. https://docs.aws.amazon.com/elasticloadbalancing/
• Kurose, J. F. and Ross, K. W., Computer Networking: A Top-Down Approach, 8th ed., Pearson, 2021, Section 2.2.5 (“Web Caching”)
• Tanenbaum, A. S. and Wetherall, D. J., Computer Networks, 6th ed., Pearson, 2021, Section 7.1.3 (“The Web”)

For a comprehensive overview, read our article on Cabling Standards.

For a comprehensive overview, read our article on Cdn Guide.