# Load Balancing

> How rws distributes traffic across backends — round-robin everywhere, plus random, IP-hash, and least-connections in config-driven mode.

Source: https://rws8.tech/docs/proxy/load-balancing/ · Last updated 2026-10-27

Every proxy component in `rust-web-server` — `ReverseProxy`, `H2ReverseProxy`, `GrpcProxy`, `TcpProxy`, `UdpProxy`, `WsProxy`, and `DynamicProxy` (config-driven mode) — uses the same underlying selection mechanism: a **lock-free atomic round-robin counter**.

## How round-robin works

Each proxy struct holds an `AtomicUsize` counter initialised to `0`. On every request the counter is incremented with `Ordering::Relaxed` and the backend index is computed as:

```rust
let idx = counter.fetch_add(1, Ordering::Relaxed) % backends.len();
```

No mutex is acquired; the operation is a single CPU instruction on every major architecture. Under concurrent load requests are distributed evenly across all live backends in cyclic order.

## Library API

```rust
use rust_web_server::proxy::{LoadBalancing, ReverseProxy};

// Explicit — same as the default.
let proxy = ReverseProxy::new(["http://a:8080", "http://b:8080", "http://c:8080"])
    .strategy(LoadBalancing::RoundRobin);
```

The `.strategy()` method exists to allow future strategies to be added without a breaking API change.

## Config-driven mode

In config-driven mode, `[[upstream]]`'s `strategy` field selects between four strategies:

```toml
[[upstream]]
name     = "api"
backends = ["api-1:3000", "api-2:3000", "api-3:3000"]
strategy = "least_connections"   # "round_robin" (default) | "random" | "ip_hash" | "least_connections"
```

| Strategy | Behavior |
|---|---|
| `round_robin` (default) | The same lock-free atomic counter described above. |
| `random` | Uniform random selection, mixing a nanosecond timestamp with the round-robin counter — no new dependency. |
| `ip_hash` | Hashes the client IP (`DefaultHasher`) to pick a backend — the same client IP is consistently routed to the same backend as long as the live backend list doesn't change, giving basic session affinity without a session store. |
| `least_connections` | Routes to whichever live backend currently has the fewest in-flight requests, tracked via a `ConnectionGuard` (RAII, decrements on `Drop`) around each proxied request. |

An unrecognized or omitted `strategy` value falls back to `round_robin`.

`weights = [3, 1, 1]` (one per backend) makes `round_robin`, `random` and `least_connections` proportional — smooth weighted round-robin, so 3:1:1 goes `a b a c a` rather than `a a a b c` — and `[upstream.sticky]` pins a client to a backend with a cookie. See [Weights, stickiness, retries and the circuit breaker](/proxy/config-driven/#weights-stickiness-retries-and-the-circuit-breaker).

### DynamicProxy and live backend lists

When `[upstream.health_check]` is configured, the health-checker thread maintains a shared `Arc<RwLock<Vec<String>>>` containing only the currently healthy backends. `DynamicProxy` (the config-driven proxy adapter) reads from this live list on every request:

```
All backends:  [api-1, api-2, api-3]
                          ↓  health checker removes api-2
Live backends: [api-1, api-3]
                          ↓  round-robin counter % 2
Request N+0 → api-1
Request N+1 → api-3
Request N+2 → api-1
```

The live list is protected by `RwLock` — reads are concurrent; the health-checker thread takes a brief write lock once per probe round to publish the list.

## Failover behaviour

`ReverseProxy` tries backends in order starting from the round-robin cursor position. If a backend fails to connect, the next backend in the list is tried. This continues until either a backend succeeds or all backends have been tried (returning `502`).

```
backends = [A, B, C]
counter = 7 → idx 7 % 3 = 1 → start at B
B fails → try C → C succeeds → return response
```

## Canary / weighted round-robin

`CanaryLayer` implements weighted round-robin by pre-expanding the rotation: a backend with `weight = 3` appears three times in the rotation array. The same lock-free atomic counter selects slots.

```rust
// 3 out of 4 slots → stable; 1 out of 4 → canary.
CanaryLayer::new(vec![
    WeightedBackend::new("http://stable:8080", 3),
    WeightedBackend::new("http://canary:8080", 1),
])
```

See [Canary / Traffic Splitting](/proxy/canary/) for full details.

:::note[Strategy selection is config-driven-proxy only]
The library API (`ReverseProxy::strategy(LoadBalancing::RoundRobin)` above) only has the one variant today. `random`/`ip_hash`/`least_connections` are implemented in `DynamicProxy`, the adapter backing `rws.config.toml`'s `[[upstream]]` mode — see [Config-driven mode](#config-driven-mode) above.
:::
