Distributed cache design

Under the Hood: Redis Cluster vs. Memcached Multi-Node

When caching requirements exceed the memory capacity of a single server (typically > 128GB) or require high write availability, the cache must be distributed across a cluster. Two distinct architectural paradigms dominate:

1. Memcached: Dumb Server, Smart Client

Memcached nodes do not communicate with each other. The client library maintains a list of node IPs and uses Ketama Consistent Hashing client-side to calculate which node holds key K. The client connects directly to that server. If a node fails, the client marks it dead and remaps its keys onto the surviving nodes. Those keys start cold and the extra load can cascade, which is why Facebook sends failed hosts' traffic to a separate Gutter pool instead (see below).

2. Redis Cluster: Smart Server Hash Slots

Redis Cluster partitions its keyspace into 16,384 Hash Slots. Every key is assigned a slot via CRC16:

hash_slot = CRC16(key) mod 16384

Slots are divided among master nodes (e.g., Node A holds slots 0..5460, Node B holds 5461..10922, Node C holds 10923..16383). Each master has one or more replica nodes.

Redis Cluster Topology and Redirection Flow

Redirection Mechanics: MOVED vs. ASK

• MOVED Redirection: When a client queries a node for a hash slot owned by another node, the server returns -MOVED 8402 10.0.1.2:6379. The client updates its internal slot-to-node routing table and retries against the target node.
• ASK Redirection: Occurs during online slot migration. If slot 8402 is migrating from Node B to Node C, Node B returns -ASK 8402 10.0.1.3:6379. The client sends an ASKING command followed by the query to Node C, but does not update its permanent routing table.

Worked example: sizing a cache cluster

Suppose you need to cache 400 million user-profile objects averaging 1 KB, which is about 400 GB of values. Redis adds per-key overhead, typically around 50 to 100 bytes for small keys, so budget roughly 450 GB. Keep nodes at no more than about 70 percent memory use to leave room for fragmentation and replication buffers. That means about 640 GB of usable RAM. With 64 GB nodes that is 10 primaries, plus one replica each for availability, so 20 nodes.

For throughput, assume 1.2 million reads per second at peak. A single Redis primary commonly serves on the order of 100,000 or more simple GETs per second per core, depending on payload and pipelining. Ten primaries at about 120,000 each fit, but with no headroom for a hot slot. Either add primaries or serve reads from replicas where slightly stale reads are acceptable.

Memcached-style versus Redis Cluster

How real systems do it

• Facebook puts mcrouter in front of memcache pools, splits pools by access pattern, and uses a small Gutter pool to absorb traffic for failed hosts instead of rehashing onto healthy ones.
• Netflix EVCache writes to a copy in every availability zone and reads locally, falling back to another zone on a miss.
• Twitter ran twemproxy and later Pelikan to multiplex connections, because thousands of app servers each holding connections to every cache node exhausted file descriptors.

Failure modes

• Connection storms: N clients times M nodes connections. Put a proxy or connection pooling in between.
• Rehash cascade: when a dead node's keys move to its neighbours, the neighbours can overload in turn. Gutter pools or replicas avoid this.
• Split-brain after a partition: Redis Cluster may accept writes on a minority primary until cluster-node-timeout expires, and those writes are lost when it rejoins.