A video streaming platform like YouTube allows users to upload and watch videos seamlessly. But beneath this seemingly simple experience lies an engineering marvel that enables us to store, process, and stream gigabytes of video data to millions of users with minimal latency and buffering.
Functional Requirements
Design a video streaming platform that allows:
-
Users to upload videos from various devices. Since different devices may produce videos with different configurations (e.g., file formats, resolutions, frame rates, etc), the system should support a wide range of input formats.
-
Users to watch uploaded videos seamlessly across different devices (web, mobile, tablet, and smart TVs). The system should provide a smooth viewing experience regardless of network conditions.
Non-Functional Requirements
The system should meet the following non-functional requirements:
-
Support
~1Mvideo uploads and~100Mvideo watched per day. -
Prioritize availability over strict consistency. For example, if a data center in Europe goes down, the traffic should automatically be routed to another region so that users can continue watching videos with minimal disruption.
-
Handle for low latency streaming of videos, even in low bandwidth environments. Videos should start playing within a few seconds and continue smoothly with minimal buffering.
-
Videos may be uploaded in one country but viewed globally. Users from anywhere in the world should receive similar playback performance.
Design Rationale
Video streaming involves several key concepts that are essential to understand before designing an efficient and scalable video streaming system. Let's break down these concepts one by one.
Q. What is a video?
A video is essentially a sequence of images (frames) played over time, combined with audio.
For example, an .mp4 file is a container that packages different streams together. It typically
contains a video stream encoded using a codec such as H.264 and an audio stream encoded
using a codec such as AAC.
The video stream contains a sequence of compressed video frames, while the audio stream contains compressed audio data. The container also stores important information such as timestamps, duration, frame rate, resolution, and codec information, which helps the video player correctly decode and synchronize the audio and video during playback.
For a 30 FPS video, 30 frames are captured every second. A single 1920 x 1080 RGB frame
requires approximately 1920 x 1080 x 3 bytes ≈ 6 MB. At 30 FPS, a 10-minute video would
require 10 minutes x 60 seconds/minute x 30 frames/second x 6 MB/frame ≈ 110 GB of memory just
for the video frames. This is why video compression is so essential.
Q. How does a video player work in your local machine?
When a video player opens a local video file, it first demultiplexes (demuxes) the container to
separate the video and audio streams. The video decoder then converts the compressed H.264 data
into raw video frames that can be displayed. Similarly, the audio decoder converts the compressed
AAC data into PCM audio samples that can be played through the speakers.
The decoded video frames and audio samples are temporarily stored in buffers so that decoding and playback can happen smoothly. The player then uses a playback clock along with the timestamps associated with the audio and video streams to determine when each video frame should be displayed and when the corresponding audio samples should be played. This keeps the audio and video synchronized throughout playback.
Press Play or Step. play() invokes the PlaybackController, which drives every stage below.
Q. How does a video player work in a browser?
A browser-based video player follows essentially the same decode → buffer → synchronize → render pipeline
as a local video player, but the major difference is that the video data is obtained over the
network rather than directly from the local disk.
When a webpage contains a <video> element, the browser's media subsystem requests the video from
the specified URL. For a simple MP4 video, the browser
can use HTTP to download the media, often using HTTP Range Requests to retrieve only the required
portions of the file.
No request yet.
Waiting for the first response.
The downloaded data is placed into a buffer, after which the browser's media pipeline demultiplexes
the container, separates the audio and video streams, and sends them to the appropriate decoders.
The video decoder converts the compressed video stream, such as H.264, into raw video frames, while
the audio decoder converts the compressed audio stream, such as AAC, into PCM audio samples.
The browser then uses timestamps and a playback clock to synchronize the audio and video before sending the video frames to the rendering pipeline and audio samples to the audio subsystem.
When you move from this simple local-player scenario to YouTube, the fundamental playback mechanism stays similar. YouTube simply adds a huge infrastructure layer before the player receives the video data.
Q. How does the upload process work?
A 2 GB video can be divided into chunks:
┌──────┬──────┬──────┬──────┬──────┐
│ 100MB│ 100MB│ 100MB│ 100MB│ ... │
└──────┴──────┴──────┴──────┴──────┘
The chunks give us resumability. It should be asynchronous process. Once the upload is complete, the system should generate an event. The user shouldn't have to wait for the entire upload to complete.
Q. Why do we need to process videos?
The original uploaded video is not necessarily in a format that is suitable for streaming to
every user. For example, a creater might upload a 4k, 60 fps, 20Mbps video. But a user may have
different network speeds. A user on a slow connection can watch 360p, while someone on a fast
connection can watch 4k.
Different devices and browsers suppport different codecs (what??) and formats.
Transcoding generally means converting an already encoded video into another encoded
representation. For example, H.264 4K → H.264 1080p.
Q. Suppose you're watching 1080p and your network suddenly becomes slow from 20 Mbps to 2 Mbps. What happens?
The player can switch from high bitrate to low bitrate. To do this, the platform needs multiple versions of the video.
Assuming the average video size is 500MB:
1M video uploads per day means 100,000 / 86,400 ≈ 10 uploads per second:
= Storage Capacity: 100,000 x 500MB = 50,000,000MB = 50TB of data storage per day
= 50TB per day x 365 days ≈ 18PB of data storage per year.
= Average Ingress (upload bandwidth) per second = 10 uploads per second x 500MB = 5GBps
100M video watched per day means:
Q1. How to handle different video configurations during upload?
Once the user uploads a video, S3 will fire an event notification to a video processing service. This service will do the work to convert the original video into different formats. It will store each format as a file in S3. It will also update the video metadata record with the file URLs representing the different formats.
Q2. How can we provide low-latency video streaming to users worldwide?
Transcoding... post processing...
different devices require different video formats in order to play back video.
This approach fails to anticipate the need to store small segments of video for streaming later. If we store the entire video, there's no way for the client to download "part" of a video. As we will see later, downloading "part" of a video is really important for streaming for various reasons.
Post-processes videos by splitting them into small segments (each a playable unit that's a few seconds in length) and then converts each segment into different formats playable on different devices. This approach introduces some complexity. Firstly, it makes our post-processing service more complex, turning it into a "pipeline." It first must split up the video into segments, and then generate video formats per segment. In addition, the system needs to store references to these segments in a sane way and use them downstream in our streaming flow effectively.
Watch a Video
If the entire video needs to be downloaded before playback, the user could be waiting to watch the video for a long time. For example, a 10GB video would take 13+ minutes to download on 100 MBPS internet, which is an unreasonable amount of time.
If the client requests the video in a single HTTP request and experiences a network disruption during that request, the download could fail and any download progress would be lost, resulting a lot of time wasted on a re-attempt.
Better Approach: Rather than forcing a full video download all at once, the system can instead download video segments to properly "stream" the video. The client would choose a video format based on the user's device, bandwidth, and preferences (e.g. if the user specified HD video, the client would stream 1080p video). The client would then load the first segment for the video, which would be a few seconds in length. This would allow the user to start watching the video quickly without excess loading. In the background, the client would start loading more segments so that it could continue playing the video seamlessly. However, (drawback) if a 1080p video is streamed and network conditions get worse, loading 1080p segments might get slower, resulting in buffering for the user.
Best Approach: Adaptive bitrate streaming relies on having stored segments of videos in different formats. It also relies on a manifest file being created during video upload time, which references all the video segments that are available in different formats. It is used by the client to stream segments of video as network conditions vary. The client will execute the following logic when streaming the video:
-
The client will fetch the
VideoMetadata, which will have a URL pointing to the manifest file in S3. -
The client will download the manifest file.
-
The client will choose a format based on network conditions / user settings. The client retrieves the URL for this segment in its chosen format from the manifest file. The client will download the first segment.
-
The client will play that segment and begin downloading more segments.
-
If the client detects that network conditions are slowing down (or improving), it will vary the format of the video segments it is downloading. If network conditions get worse (e.g. the bitrate is lower), the client will attempt to download more compressed, lower resolution segments to avoid any interruption in streaming.
API Design
VideoPlayingService
Data Model Design
High-Level Design
Playing Service
When play() is called:
play()
│
▼
PlaybackController
│
├── Start PlaybackClock
│
├── Demux video/audio packets
│
├── Decode video
│
├── Decode audio
│
├── Put decoded data into buffers
│
├── Synchronizer checks timestamps
│
├── VideoRenderer displays correct frame
│
└── AudioRenderer plays corresponding audio
Similar Case Studies
- Design Spotify
The audio is also sampled thousands of times per second. For example, CD-quality audio uses
44,100 samples/second. So a 10-second video would require 10 seconds x 44,100 samples/second x 2 bytes/sample = 0.882 GB
of memory.
| Component | 10 seconds |
|---|---|
| Uncompressed 1080p video @ 30 FPS | ~1.86 GB |
| Uncompressed stereo audio (44.1 kHz, 16-bit) | ~1.76 MB |
| Total | ~1.862 GB |
-
Design Live Streaming Platform
-
Design Netflix