Response Caching

CacheLayer is a middleware that short-circuits the inner application for cacheable GET responses, serving them directly from an in-memory store until their TTL elapses. It lives in crates/rws-server/src/cache/mod.rs and implements the standard Middleware trait, so it composes with every application variant.

Basic setup#

use rust_web_server::app::App;
use rust_web_server::cache::CacheLayer;
use rust_web_server::core::New;

let app = App::new()
    .wrap(CacheLayer::memory(1000).ttl(60));

This caches up to 1 000 GET responses for 60 seconds each.

Constructor and builder methods#

CacheLayer::memory(capacity)#

Creates a new in-memory cache bounded to capacity entries. The default TTL is 60 seconds.

let layer = CacheLayer::memory(500);

CacheLayer::disk(dir, max_bytes)#

Keeps entries in files under dir (created if missing), bounded to max_bytes in total, oldest dropped first (since 18.11.0). Entries survive a restart: when the layer is built it indexes the directory and removes expired, unreadable and half-written files. Everything else (what is cached, for how long, Age, Vary) is the same as the memory cache.

let layer = CacheLayer::disk("/var/cache/rws/api", 512 * 1024 * 1024)?.ttl(600);

• One file per entry, named by a hash of the cache key; it is written to a temporary file and renamed, so a reader never sees a half-written entry.
• Reads and writes are file I/O, so a disk-cached route runs on the blocking pool rather than the event loop. Use it for responses that are expensive to produce, not for ones a memory cache would serve in microseconds.
• A disk error (full disk, permissions) is logged and the response is simply not cached.
• In a container, put dir on a writable volume (the image's /srv is read-only).

.ttl(secs)#

The longest an entry is kept. A response can ask for less with Cache-Control: s-maxage (meant for shared caches like this one), else max-age, is used when it's shorter than the ttl; a response that says neither is kept for the ttl. max-age=0 isn't stored at all.

let layer = CacheLayer::memory(500).ttl(300); // 5 minutes at most

Since 18.11.0. Before, every cacheable response was kept for the full ttl whatever its max-age.

.vary_by_header(name)#

Includes a request header in the cache key so that different values of that header produce separate cache entries. The name match is case-insensitive. Chain multiple calls to vary by more than one header.

let layer = CacheLayer::memory(500)
    .vary_by_header("Accept-Language")
    .vary_by_header("Accept");

Without .vary_by_header, all requests to the same URI share a single cache entry regardless of any request headers.

What is cached#

Cache key#

The cache key is the request URI combined with the values of every header named in .vary_by_header(). Two requests to the same path but with different Accept-Language values produce separate entries when vary_by_header("Accept-Language") is configured.

Age header#

On cache hits, the response receives an Age header indicating how many seconds have elapsed since the entry was inserted:

Age: 42

If the response already carries an Age header, its value is replaced.

Cache-Status header#

Every response that passes through a CacheLayer says what the cache did (RFC 9211, since 18.17.0):

A Cache-Status from further upstream is kept, with this cache's member after it. proxy_name (RWS_CONFIG_PROXY_NAME) renames rws; cache_status = false (RWS_CONFIG_CACHE_STATUS) leaves the header out.

curl -sI https://example.com/products | grep -i cache-status

Eviction#

A disk cache evicts by total size instead: expired entries first, then the oldest, until the files fit in max_bytes; an entry bigger than max_bytes on its own isn't stored.

When the memory store is at capacity and a new entry must be inserted, the oldest entry (by insertion time) is evicted. Expired entries are purged before every insert, so a full store of expired entries makes room before falling back to oldest-first eviction.

Cache statistics#

CacheLayer is cheaply Clone-able — all clones share the same backing store. Keep a handle to query statistics or invalidate the cache while the other clone is used as middleware:

let cache = CacheLayer::memory(1000).ttl(60);
let cache_handle = cache.clone(); // shared reference

let app = App::new().wrap(cache);

// Later, from a handler or admin endpoint:
let hits   = cache_handle.hits();   // u64
let misses = cache_handle.misses(); // u64
let size   = cache_handle.size();   // current entry count

cache.clear()#

Evicts all entries immediately:

cache_handle.clear();

Purging#

purge removes part of a cache (memory or disk) and returns how many entries went. A target matches the request path and query as cached, every Vary variant included:

use rust_web_server::cache::Purge;

cache_handle.purge(&Purge::Exact("/products/42".into()));
cache_handle.purge(&Purge::Prefix("/products/".into()));
cache_handle.purge(&Purge::All); // same as clear()

HTTP PURGE#

.purge_token(token) makes the layer answer the PURGE method itself, nginx/Varnish style (config file: purge_token_env):

curl -X PURGE -H "Authorization: Bearer $TOKEN" https://example.com/products/42
curl -X PURGE -H "Authorization: Bearer $TOKEN" 'https://example.com/products/*'
# {"purged":3}

A trailing * purges a prefix. A wrong or missing token gets 401; the request never reaches the application. Without a token set, PURGE passes through to the application like any other method. rws accepts PURGE as a request method since 18.11.0 (before, it was a 400).

Admin API and MCP#

Named caches go in cache::registry: the binary registers its own as app, and every config route's cache as route:<name> (rebuilt on each reload). The admin API lists them at GET /admin/api/v1/caches and purges with DELETE /admin/api/v1/caches/<name>?path=… or ?prefix=… (see Admin API); the admin UI has a purge field on its Controls page. The cache_clear MCP tool takes the same optional path / prefix. Library apps call cache::registry::register(name, layer.clone()) to add theirs.

Full example#

use rust_web_server::app::App;
use rust_web_server::cache::CacheLayer;
use rust_web_server::core::New;
use rust_web_server::response::{Response, STATUS_CODE_REASON_PHRASE};

let cache = CacheLayer::memory(200)
    .ttl(120)
    .vary_by_header("Accept-Language");

let cache_handle = cache.clone();

let app = App::with_state(cache_handle)
    .get("/api/products", |_req, _params, _conn, cache| {
        // This response will be cached for 120 s per Accept-Language value.
        let mut r = Response::new();
        r.status_code = *STATUS_CODE_REASON_PHRASE.n200_ok.status_code;
        r.reason_phrase = STATUS_CODE_REASON_PHRASE.n200_ok.reason_phrase.to_string();
        r
    })
    .get("/admin/cache/stats", |_req, _params, _conn, cache| {
        let body = format!(
            "hits={} misses={} size={}",
            cache.hits(), cache.misses(), cache.size()
        );
        let mut r = Response::new();
        r.status_code = *STATUS_CODE_REASON_PHRASE.n200_ok.status_code;
        r.reason_phrase = STATUS_CODE_REASON_PHRASE.n200_ok.reason_phrase.to_string();
        r
    })
    .wrap(cache.clone()); // middleware layer applied outermost

Caching values: fetch_or#

CacheLayer caches whole responses. cache::ValueCache<V> caches values your code computes — an expensive query, a rendered fragment, an API answer — each for its own time (18.35.0):

use std::time::Duration;
use rust_web_server::cache::ValueCache;

let cache: ValueCache<Vec<Product>> = ValueCache::new(10_000);   // at most 10 000 entries

let featured = cache.fetch_or("products:featured", Duration::from_secs(300), || load_featured(&db));
// async: cache.fetch_or_async(key, ttl, || async { … }).await
// fallible, errors not cached: cache.try_fetch_or(key, ttl, || db.query(…))?

When it's full, expired entries go first, then the oldest. A ValueCache is cheap to clone and all clones share the entries: put one in the app state. The value is computed outside the lock, so two requests missing the same key at the same moment both compute it. It's per process: replicas each have their own.