Designing a Video Streaming Platform
A video platform like YouTube or Netflix turns an uploaded file into media that viewers can play on different devices and varying network connections. Uploading is only the first step: the service must prepare suitable versions, make them discoverable, and keep delivering the next segment while playback continues.
The scale is staggering. YouTube processes hundreds of hours of video uploads every minute and serves billions of views daily. Netflix accounts for a massive percentage of the internet's downstream traffic.
The Upload Pipeline
When a creator uploads a video, we don't just store the raw file and serve it. The upload goes to an object storage service like Amazon S3.
Because video files are large, we use Multipart Uploads. The client breaks the file into smaller chunks and uploads them in parallel. If a chunk fails, only that chunk is retried, rather than restarting a 5GB upload from scratch. To do this securely directly to object storage, the backend generates Pre-signed URLs for each chunk.
Transcoding and Adaptive Bitrate Streaming
Raw video is huge and not universally playable. The platform must transcode the video into a ladder of codecs (H.264, VP9, AV1), resolutions (1080p, 720p, 480p) and bitrates, packaged in containers such as MP4/CMAF or WebM, to support everything from high-end 4K TVs to 10-year-old smartphones on 3G connections.
Modern streaming uses protocols like HLS (HTTP Live Streaming) or DASH (Dynamic Adaptive Streaming over HTTP). These protocols break the transcoded video into short segments (typically 2 to 6 seconds; Apple's HLS authoring spec recommends 6 seconds). The video player on the client's device downloads a manifest file listing all segments and quality levels.
The magic is Adaptive Bitrate Streaming (ABR). If the user's network speed drops, the video player automatically requests the next 2-second segment in 480p instead of 1080p, preventing playback buffering.
Delivering Content (CDN)
Serving video directly from centralized application servers would melt the servers and result in terrible latency for users far away. Video platforms rely heavily on Content Delivery Networks (CDNs).
When a user in Tokyo clicks play, they stream the video chunks from an edge server in Tokyo, not from the primary data center in Virginia. CDNs cache the static video segments globally. Netflix takes this a step further with Open Connect, placing custom cache appliances directly inside the data centers of Internet Service Providers (ISPs).
Metadata and Search
While the heavy video files live on CDNs and object storage, the video metadata (title, description, tags, view count) lives in a database (like MySQL or Cassandra). A search cluster (like Elasticsearch) indexes this metadata to power the search bar.
View counting is notoriously difficult at scale. Updating a database row every time someone watches a video would cause massive write contention. Instead, platforms use distributed counters or stream processing (like Kafka + Flink) to aggregate views asynchronously and write them to the database periodically.
A typical bitrate ladder (H.264)
| Rendition | Resolution | Approximate bitrate | GB per hour of video |
|---|---|---|---|
| 240p | 426x240 | 0.4 Mbps | 0.18 |
| 480p | 854x480 | 1.2 Mbps | 0.54 |
| 720p | 1280x720 | 3 Mbps | 1.35 |
| 1080p | 1920x1080 | 5 to 6 Mbps | 2.25 to 2.7 |
| 4K (HEVC or AV1) | 3840x2160 | 12 to 16 Mbps | 5.4 to 7.2 |
Netflix's per-title and per-shot encoding tunes the ladder to each piece of content. A cartoon needs far fewer bits than a grainy action film at the same quality, and AV1 needs roughly 30 to 50% fewer bits than H.264 at similar quality, at a much higher encoding cost.
Worked example: storage and egress
- YouTube has said that more than 500 hours of video are uploaded every minute, which is about 720,000 hours per day.
- If every upload got the full ladder above (about 10 GB per hour across renditions), that would be about 7 PB per day. Real platforms encode only low renditions for rarely watched videos, and add high and 4K renditions once a video becomes popular.
- Egress dominates cost. 10 million concurrent viewers at an average of 3 Mbps is 30 Tbps. That is why Netflix places Open Connect appliances inside ISPs, and why a CDN hit ratio of 95% or more is essential.
Processing pipeline
Segment length trade-off
| Segment length | Pro | Con |
|---|---|---|
| 2 s | Faster start and faster bitrate switching; lower live latency | More requests and manifest entries; slightly worse compression |
| 6 s (Apple's HLS recommendation) | Efficient, fewer requests | Slower adaptation; higher live latency |
| Low-latency HLS/DASH partial segments | 2 to 5 s glass-to-glass latency for live streams | More complex players and CDN configuration |