Reverse Proxy

ReverseProxy is a Middleware that forwards incoming HTTP/1.1 requests to one or more backends. It lives in src/proxy/mod.rs and requires no feature flags.

Basic usage

use rust_web_server::app::App;
use rust_web_server::core::New;
use rust_web_server::proxy::ReverseProxy;

// All requests forwarded round-robin across two backends.
let app = App::new()
    .wrap(ReverseProxy::new(["http://backend-1:8080", "http://backend-2:8080"]));

Selective proxying with path_prefix

By default every request is proxied. Use .path_prefix() to restrict proxying to a specific path prefix; all other requests fall through to the inner application.

// Only proxy /api/* — other paths are handled locally.
let app = App::new()
    .wrap(ReverseProxy::new(["http://api-service:3000"])
        .path_prefix("/api"));

Load balancing strategy

ReverseProxy accepts a .strategy() builder for future extensibility. The only active strategy is round-robin.

use rust_web_server::proxy::{LoadBalancing, ReverseProxy};

let proxy = ReverseProxy::new(["http://a:8080", "http://b:8080"])
    .strategy(LoadBalancing::RoundRobin); // default; explicit for clarity

The counter is a lock-free AtomicUsize incremented on every request. The backend index is counter % backend_count, giving a uniform cyclic distribution with no mutex overhead.

Timeouts

let proxy = ReverseProxy::new(["http://backend:8080"])
    .connect_timeout_ms(3_000)   // TCP connect timeout (default: 5 000 ms)
    .read_timeout_ms(60_000);    // Response read timeout (default: 30 000 ms)

The write timeout for the forwarded request is always 10 seconds (not configurable via the builder today).

Automatic failover

When a backend connection fails, ReverseProxy tries the next backend in round-robin order. Only after all backends have failed does it return 502 Bad Gateway.

request → backend-1 fails → backend-2 fails → 502 Bad Gateway
request → backend-1 fails → backend-2 OK    → response forwarded

Headers

Hop-by-hop headers stripped

The following headers are never forwarded to the upstream or back to the client, per RFC 7230:

• Connection
• Keep-Alive
• Proxy-Authenticate
• Proxy-Authorization
• TE
• Trailers
• Transfer-Encoding
• Upgrade

Headers added to forwarded requests

Header	Value
X-Forwarded-For	Client IP from ConnectionInfo
Via	1.1 rws
Host	Backend host (replaces the original Host header)
Connection	keep-alive (pooled path) or close (non-pooled callers)

Connection pooling

ReverseProxy ships with a built-in ConnPool that reuses idle TCP connections to backends. When a backend responds with Connection: keep-alive and a fixed-size body (Content-Length), the stream is returned to the pool and reused for the next request to the same backend.

Streaming responses (SSE, chunked, large downloads) are forwarded directly to the client without buffering — see Streaming responses.

Defaults: 8 idle connections per backend, 60-second idle timeout. No configuration needed.

Tuning the built-in pool

use rust_web_server::proxy::ReverseProxy;

let proxy = ReverseProxy::new(["http://backend:8080"])
    .max_idle_conns(32);   // up to 32 idle connections per backend

Sharing a pool across proxy instances

Useful when two ReverseProxy instances route to overlapping backends (e.g., an API proxy and an auth proxy both reach the same service).

use std::sync::Arc;
use std::time::Duration;
use rust_web_server::app::App;
use rust_web_server::core::New;
use rust_web_server::proxy::{ConnPool, ReverseProxy};

let pool = Arc::new(ConnPool::new(32, Duration::from_secs(120)));

let api  = App::new().wrap(
    ReverseProxy::new(["http://api:3000"]).with_pool(Arc::clone(&pool)));
let auth = App::new().wrap(
    ReverseProxy::new(["http://auth:4000"]).with_pool(Arc::clone(&pool)));

How pooling works

Backend response	Pool action
Connection: keep-alive + Content-Length ≤ 1 MB	Stream returned to pool after body is read
Transfer-Encoding: chunked	Streamed to client (raw passthrough); connection not pooled
Content-Type: text/event-stream	Streamed to client (chunked re-encoding); connection not pooled
Content-Length > 1 MB	Streamed to client; connection not pooled
Connection: close or EOF	Stream dropped; TCP connection closed
Idle stream older than idle_timeout	Evicted on next acquire()
Pool at max_idle for this backend	Extra stream dropped (TCP closed)

Tip

Connection pooling is most impactful when backends and the proxy share a network (low-latency link). On high-throughput routes, pooling eliminates per-request TCP handshake latency entirely.

Backend URL format

Backend strings accept any of these forms:

http://host:port      # scheme stripped, plain TCP
h2://host:port        # treated as plain TCP
host:port             # bare host:port
host                  # port defaults to 80

TLS to upstream

The programmatic ReverseProxy middleware connects over plain TCP. For HTTPS backends, use the config-driven proxy with https:// backend URLs — see Config-Driven Proxy. Alternatively, terminate TLS at the backend itself (e.g., a sidecar that unwraps TLS and proxies to the app on a localhost port).

Streaming responses

ReverseProxy detects streaming backend responses and forwards bytes to the client as they arrive, without buffering the full body in memory. A response is streamed when any of these conditions are true:

Condition	Use case
Content-Type: text/event-stream	SSE / Server-Sent Events
Transfer-Encoding: chunked	AI token streams (OpenAI, Anthropic, etc.)
Content-Length > 1 MB	Large file downloads

How it works

1. The proxy reads only the response headers from the backend.
2. It creates a Response with the parsed headers and sets Response::stream_pipe to a reader backed by the live backend TCP connection.
3. The server detects stream_pipe and calls Server::pipe_stream():
   • Chunked backend: raw chunk frames are forwarded byte-for-byte (client handles decoding).
   • SSE / plain body: bytes are re-encoded as Transfer-Encoding: chunked so the client receives each fragment immediately.

Streamed connections are never returned to the ConnPool — the backend socket is consumed by the pipe.

SSE through the proxy

use rust_web_server::app::App;
use rust_web_server::core::New;
use rust_web_server::proxy::ReverseProxy;

// All SSE responses from the backend are forwarded without buffering.
let app = App::new()
    .wrap(ReverseProxy::new(["http://event-service:9000"])
        .path_prefix("/events"));

The client connects with Accept: text/event-stream as usual. The proxy transparently forwards each SSE event as a chunked frame the moment it arrives from the backend.

AI token stream through the proxy

use rust_web_server::app::App;
use rust_web_server::core::New;
use rust_web_server::proxy::ReverseProxy;

// OpenAI / Anthropic APIs send Transfer-Encoding: chunked.
// The proxy forwards each token chunk without waiting for the full response.
let app = App::new()
    .wrap(ReverseProxy::new(["https://api.openai.com"])
        .path_prefix("/v1"));

Custom stream_pipe (application code)

Any application handler can use stream_pipe to stream arbitrary Read sources:

use rust_web_server::response::{Response, STATUS_CODE_REASON_PHRASE};

fn my_handler(_req: &_, _params: &_, _conn: &_, _state: &_) -> Response {
    let mut r = Response::new();
    r.status_code = *STATUS_CODE_REASON_PHRASE.n200_ok.status_code;
    r.reason_phrase = STATUS_CODE_REASON_PHRASE.n200_ok.reason_phrase.to_string();
    // Any std::io::Read + Send value works: files, cursors, sockets, generators …
    r.stream_pipe = Some(Box::new(std::io::Cursor::new(b"data: hello\n\n".to_vec())));
    r
}

502 Bad Gateway

ReverseProxy returns 502 Bad Gateway with Content-Type: text/plain when:

• No backends are configured.
• All backends fail to connect or return a network error.

Combining with other middleware

Because ReverseProxy implements Middleware, you can stack it with rate limiting, auth, rewriting, and any other middleware via .wrap().

use rust_web_server::app::App;
use rust_web_server::core::New;
use rust_web_server::proxy::ReverseProxy;
use rust_web_server::rate_limit::RateLimitLayer;

let app = App::new()
    .wrap(RateLimitLayer::new(100, 60))   // 100 req/min per IP
    .wrap(ReverseProxy::new(["http://backend:3000"])
        .path_prefix("/api"));

Middleware is applied outermost-first, so RateLimitLayer runs before ReverseProxy.