Art of Codeartofcode
Lesson 6 of 16

40. Design Video Upload and Streaming (YouTube)

1
The interview prompt
2
40.1 What do you ask first?
3
40.2 How big is this, really?
4
40.3 What are the doors?
5
40.4 What do we store?
6
40.5 What does the whole system look like?
7
40.6 How does the upload pipeline work?
8
40.7 How does transcoding work?
9
40.8 How does adaptive bitrate work?
10
40.9 How does CDN delivery work at this scale?
11
40.10 How do you count views?
12
40.11 What would you say in the last five minutes?
13
Score yourself
14
Variations to practise
15
Sources
16
Code lab
40. Design Video Upload and Streaming (YouTube)

40.5 What does the whole system look like?

3 min read+10 XP

Step 4: High-level design (10 minutes)

the creator uploads to a raw store; a job queue drives transcode workers that write segments and playlists to the video store, served through the CDN; viewers get the playlist from the watch API and stream segments; view events flow to counters.

Figure: the final design. Upload and processing on top; watching through the CDN; counting on the side.

Upload and processing, step by step:

  1. The creator's app asks for an upload URL and sends the file in resumable chunks directly to the raw upload store (40.6).
  2. On completion, a job goes on the queue (Chapter 22). The processing pipeline validates the file, extracts metadata, makes thumbnails, runs safety and copyright checks, and transcodes in parallel into the ladder (40.7).
  3. Segments and playlists (HLS/DASH) go to the video store; the video's status becomes ready, lowest qualities first.

Watching, step by step:

  1. The player asks the watch API for the video; it gets metadata and the master playlist URL.
  2. The player fetches playlists and segments from the nearest CDN edge, choosing a quality for each segment (40.8).
  3. Popular segments are already in edge caches; rare ones are fetched from the video store through a shield (Chapter 26).
  4. The player reports view events; counters de-duplicate and add them up (40.10).

Interviewer pushback:

  • "A transcode worker dies halfway." → Each segment is a separate job; the queue redelivers unfinished ones (Chapter 22). Outputs are written under deterministic names, so a retry just overwrites.
  • "A video goes viral right after upload." → Its segments are pulled into every edge on first request; the shield collapses misses (Chapter 26); higher qualities and better codecs get queued with high priority.

Memory code: "Upload once, process in pieces, watch from the edge."

Offer: "The interesting parts are the upload pipeline, transcoding, adaptive bitrate, CDN delivery and view counts. Which would you like?"