Caching

Cache Stampede & Hot Keys

Two distinct failure modes with a shared root cause: a stampede is many requests missing the same expired key at once; a hot key is one key so popular it overloads the single shard that owns it.

Cache stampede ("dog-piling"): a popular key expires, and hundreds of concurrent requests all miss at the same instant and hit the database simultaneously
Stampede mitigation — request coalescing: the first miss for a key locks and populates it while every other concurrent request for that key waits for the same result instead of also querying the database
Stampede mitigation — probabilistic early expiration: refresh a key slightly before its TTL, with jitter, so not every replica of a popular key expires at exactly the same moment
Hot key: one key (a viral post, a celebrity profile) receives disproportionate traffic, overloading the single node/shard that owns it even though the cluster overall has plenty of spare capacity
Hot-key mitigation: a small local (in-process) cache on top of the distributed cache for the hottest keys, or explicitly replicating a hot key across multiple nodes so no single one absorbs all its traffic

A stampede at the moment a key expires

Without coalescing, every concurrent miss for the same key independently re-does the same expensive work

Request coalescing with in-flight futures

class CoalescingCache<K, V> {
    private final Map<K, CompletableFuture<V>> inFlight = new ConcurrentHashMap<>();

    CompletableFuture<V> get(K key, Supplier<V> loadFromSource) {
        return inFlight.computeIfAbsent(key, k ->
            CompletableFuture.supplyAsync(loadFromSource)
                .whenComplete((v, err) -> inFlight.remove(k))   // only the first caller triggers the load
        );
    }
}

Every concurrent caller for the same key gets the same in-flight future — the source of truth is queried exactly once, not once per waiter

Stampede vs hot key
	Cache stampede	Hot key
Trigger	A key expiring	Sustained popularity of one key
Blast radius	The database, briefly	One cache node/shard, continuously
Primary fix	Request coalescing, jittered TTLs	Local caching or replicating the hot key