Designing Live Commentary (Twitch Chat)
Live commentary for events (sports, game streams) presents unique challenges: extreme write throughput concentrated on a single channel, and the need for sub-second read latency for millions of viewers simultaneously.
The Hot Key Dilemma
Standard database partitions group data by chat room ID. But if a popular streamer has 500,000 concurrent viewers chatting, that single partition will be overwhelmed, leading to massive lag or failure.
In-Memory Pub/Sub and Sampling
To handle this, live commentary completely bypasses persistent storage for the hot read path. Messages are routed through a highly distributed Pub/Sub system (for example Redis, where Redis 7 sharded pub/sub avoids classic Redis Cluster broadcasting every PUBLISH to all nodes, or a purpose-built layer such as Twitch's chat Edge and Pubsub services). The WebSocket servers subscribe to the channel and push messages directly to viewers.
When velocity is too high (e.g., 10,000 messages per second), human readability is impossible. The WebSocket servers intelligently drop or sample messages before sending them down the wire to prevent crashing the client's browser and to save bandwidth.
Asynchronous Persistence
While the live chat is served from memory, the messages are simultaneously sent to a Kafka topic. Background consumer workers batch these messages and write them in bulk to a cold storage database (like Cassandra) for later VOD (Video on Demand) playback, completely decoupling the live experience from disk I/O.
Worked example: 500,000 viewers, 2,000 messages per second
- Naive fan-out: 2,000 messages x 500,000 viewers = 1 billion deliveries per second. Impossible.
- Sample at the edge: show each viewer at most 20 messages per second (already more than a human can read) and prefer messages from friends, moderators and verified accounts. Deliveries drop to 10 million per second.
- Batch: flush every 200 ms, so each viewer receives 5 frames per second; with 250 edge servers holding 2,000 viewers each, that is 10,000 frames per second per server, which is comfortable.
- Central fan-out: the channel's pub/sub topic publishes each message once to 250 subscribed edge servers, 500,000 internal messages per second for this stream instead of 1 billion.
Controlling the firehose
| Technique | What it does | Trade-off |
|---|---|---|
| Sampling | Randomly keep a fraction of messages per viewer | Viewers see different subsets; fine for chat, not for scores |
| Prioritization | Always keep friends, hosts, moderators, paid highlights | Needs per-viewer context at the edge |
| Batching | Send one frame every 100 to 500 ms | Adds up to one batch interval of latency |
| Slow mode and rate limits | Limit each commenter to one message every N seconds | Product friction, but reduces spam and load at the source |
| Coalescing reactions | Send counts per emoji per interval, not individual taps | Loses per-user detail, which viewers do not need |
How real systems do it
- LinkedIn keeps an in-memory subscription table mapping each live video to the front-end servers with viewers, and a dispatcher publishes each event only to those servers, which then fan out to their connections.
- Twitch chat runs over WebSockets with edge servers holding viewer connections and a separate distribution layer between them.
- If you use Redis for the pub/sub layer, note that classic Redis Cluster PUBLISH is broadcast to every node; Redis 7 sharded pub/sub (SPUBLISH) keeps a channel's traffic on its own shard.