Concrete scenario
A product catalog key expires under 10,000 concurrent reads.
Mechanics
Singleflight coalesces duplicate work within one process; cross-process coordination needs a lock, lease, queue, or refresh owner with a bounded wait and failure policy. Probabilistic early refresh such as XFetch reduces synchronized expiry probability but does not guarantee permanent residency. Cache-aside writes still need race handling.
Failure mode
An old reader can load the database, race with an update-and-delete, then repopulate stale data. A refresh owner can crash, waiters can pile up, and a distributed lease can expire while work continues.
Trade-off
Coalescing protects the database but adds waiting and coordination. Versioned values, delayed double deletion, write-through, or stale-while-revalidate each exchange consistency, write latency, complexity, and availability.
Worked example: one hot key, 10,000 readers
A catalogue key takes 200 ms to rebuild from the database and receives 10,000 reads per second across 100 app servers. With no protection, when it expires every request in the 200 ms window misses, which is about 2,000 concurrent identical DB queries.
- Per-process singleflight: at most one load per server, so 100 concurrent loads. That is 20 times better, but still 100.
- Distributed rebuild lock (
SET lock:key token NX PX 3000): one load cluster-wide. The other callers either serve the stale value or poll briefly. - Probabilistic early refresh (XFetch): each reader refreshes early with a probability that rises as expiry approaches, weighted by rebuild cost. Typically one reader refreshes shortly before expiry, and nobody ever sees a miss.
The stale-set race and leases
Facebook's memcache issues a lease token on a miss. A delete invalidates outstanding leases, so the late set of v1 is rejected. The same token also rate-limits rebuilds: memcache hands out at most one token per key every 10 seconds, and other clients briefly wait and retry.
Technique comparison
| Technique | Stops stampede | Stops stale set | Cost |
|---|---|---|---|
| Local singleflight | Per process only | No | Trivial |
| Distributed lock with TTL | Yes | No | Extra round trip. Stale lock holder |
| Leases (memcache) | Yes | Yes | Needs cache support |
| Versioned compare-and-set | No | Yes | Needs version from the DB row |
| Stale-while-revalidate | Yes, readers never wait | No | Serves bounded staleness |
| XFetch early refresh | Mostly | No | Tiny. Probabilistic |
| Delayed double delete | No | Reduces the window | Heuristic. The delay is a guess |