Load balancing is the fundamental technique for distributing network traffic across multiple servers to ensure no single server becomes overwhelmed, improving application availability, scalability, and fault tolerance. Operating at various layers of the OSI model—from network to application—load balancers act as intelligent traffic directors that continuously monitor backend server health and route requests using sophisticated algorithms. In 2026, load balancing has evolved beyond simple traffic distribution to encompass AI-driven optimization, eBPF-accelerated kernel-level forwarding, and deep service-mesh integration via Istio, Linkerd, and the Kubernetes Gateway API. What makes load balancing essential is that it transforms individual servers into resilient, horizontally scalable systems capable of handling millions of requests per second while maintaining sub-second response times and near-perfect uptime.
What This Cheat Sheet Covers
This topic spans 16 focused tables and 108 indexed concepts, 108 flashcards. Below is a complete table-by-table outline of this topic, spanning foundational concepts through advanced details.
A jump-to index of every table row in this cheat sheet.
An interactive map of every table and concept in this topic.
Table 1: Core Load Balancing Algorithms
The algorithm is the brain of every load balancer — it decides which backend gets the next request. The choices here run from the dead-simple (Round Robin, Random) to the load-aware (Least Connections, Least Response Time) to the consistency-preserving (IP Hash, Consistent Hashing). Which one fits depends on whether your servers are equal, whether request costs vary wildly, and whether a client needs to keep landing on the same machine.
| Algorithm | Example | Description | |
|---|---|---|---|
Request1 → Server1Request2 → Server2Request3 → Server1 | • Distributes requests sequentially in rotation • simplest algorithm, best when servers have equal capacity and request cost. | ||
server backend1 weight=3;server backend2 weight=1; | • Assigns different weights so higher-weight servers receive proportionally more requests • use when servers have unequal capacity. | ||
Server1: 5 connServer2: 2 conn→ Route to Server2 | • Routes to the server with the fewest active connections • ideal when request processing times vary significantly. | ||
Server1: 8 pendingServer2: 3 pending→ Route to Server2 | • Counts in-flight requests (not just connections) per target • default AWS ALB algorithm since 2023, better than least connections for HTTP/2. | ||
hash(192.168.1.50) → Server2 | • Uses a hash of the client IP to consistently route a client to the same server • provides session persistence without cookies. | ||
Server1: 50ms avgServer2: 20ms avg→ Route to Server2 | • Selects the server with the fastest response time and fewest active connections • optimizes for user-perceived latency. | ||
hash(key) mod 360 → Servervirtual_nodes = 150 | • Maps servers and requests onto a virtual ring • only K/n keys remapped when a server is added or removed — used in CDN and distributed caches. | ||
Server1: 10 conn, weight=2Server2: 3 conn, weight=1 | • Combines least connections with capacity weights • calculates connections-to-weight ratio to pick the optimal target. | ||
pick 2 random servers→ route to less-loaded one | • Randomly selects two candidates and routes to the less-loaded one • Envoy's LEAST_REQUEST policy — O(1) yet near-optimal distribution. | ||
check CPU & memoryif load < 70% → route | • Routes based on real-time server metrics (CPU, memory, disk I/O) • requires agent or monitoring integration on each server. | ||
Server1: 50 MbpsServer2: 15 Mbps→ Route to Server2 | • Routes to the server currently transmitting the least data (Mbps) • useful in bandwidth-constrained or media-streaming environments. | ||
random() → Server3 | • Selects a backend randomly • simple but can produce uneven distribution with small sample sizes. |