High Performance Browser Networking

Ideas you will meet

HTTP and QUIC

HTTP versions, QUIC, caching headers and web performance.

WebSockets

Persistent bidirectional connections for real-time features.

CDN

Serve content from edge locations close to users.

What we already know about this

Our own lessons on these ideas. Read them here before you start, so it makes sense.

• System Design Foundations · 11 minNetworking, APIs and CDNsWhat happens between a click and your server: DNS, HTTP, API styles, and how a CDN moves content closer to users.Read hereClose

  The journey of a request

  Before your code runs, a lot happens. When a browser loads https://shop.example.com:

  100%

  Figure SDA-FND-09-1 · Request journey: DNS, CDN, origin · © forcsguy.com

  1. DNS turns the name into an IP address. Answers are cached at several levels for a time given by the record's TTL, so changes spread gradually.
  2. A connection is opened (TCP) and secured (TLS). Each step costs round trips, so long distances hurt.
  3. HTTP carries the request and response.

  Latency is mostly physics: a round trip across an ocean costs over 100 ms no matter how fast your code is. The only fixes are fewer round trips and moving data closer to the user.

  Image: Syced · CC0 · Source · Wikimedia Commons: Fujikura optic fiber at Science Museum

  Name lookup is a short chain of questions, each answered by a more specific server:

  100%

  Figure SDA-FND-09-2 · How a name becomes an address · © forcsguy.com

  Image: Howe's Adventures & Achievements of Americans · Public domain · Source · Wikimedia Commons: Transatlantic submarine cable map

  CDNs

  A content delivery network is a worldwide set of servers (edges) that keep copies of your content close to users.

  • Great for static files: images, video, scripts, stylesheets, and often whole pages that rarely change.
  • Cuts latency, absorbs traffic spikes and shields your origin from load.
  • Controlled by cache headers (Cache-Control: max-age=...) and purge APIs when content changes.
  • Use a versioned file name (app.3f9a.js) so you can cache for a year and still ship updates instantly.

  100%

  Figure SDA-FND-09-3 · Content delivery in tiers · © forcsguy.com

  Before: everyone goes to the origin

  100%

  After: a CDN serves copies near users

  100%

  What changed

  • Users get files from a nearby location
  • The origin handles far fewer requests
  • Changed files need invalidation or versioned names

  Choosing an API style

  Style	Strength	Watch out for
  REST over HTTP/JSON	Simple, cacheable, universal	Over-fetching, many round trips for complex screens
  GraphQL	Client asks for exactly the fields it needs	Query cost control, caching is harder
  gRPC	Fast binary protocol, strong contracts	Needs tooling, awkward in browsers
  SOAP	XML messages with a formal contract (WSDL) and built-in standards for security and transactions, still common in banks and older enterprise systems	Verbose, heavier tooling, rarely chosen for new public APIs
  WebSockets / SSE	Server can push live updates	Long-lived connections need extra capacity and care

  A good default is REST for public APIs, gRPC between internal services if you need speed, and WebSockets only where live push is truly needed.

  Designing good APIs

  • Version them so old clients keep working.
  • Paginate list endpoints (cursor-based pagination stays stable as data changes).
  • Make writes idempotent with idempotency keys, so a retry does not double-charge.
  • Return clear errors with correct status codes.
  • Authenticate and rate limit at the edge, often in an API gateway that also handles routing, logging and TLS.

  100%

  Figure SDA-FND-09-4 · API gateway in front of services · © forcsguy.com

  More on this site

  Related reading

  • High Performance Browser NetworkingIlya Grigorik · O'ReillyBooks
  • HTTP/3: the past, the present, and the futureCloudflareBlogs
  • MDN: HTTP cachingMDN contributors · MozillaDocumentation
  • Cloudflare BlogCloudflare engineers · CloudflareBlogs

  Show 6 moreShow fewer

  • The Road to QUICCloudflareBlogs
  • Flannel: An Application-Level Edge Cache to Make Slack ScaleSlack Engineering · SlackCase Studies
  • Cloudflare Learning Center: What is a CDN?CloudflareDocumentation
  • Cloudflare EngineeringCloudflare engineers · CloudflareBlogs
  • Justin.tv's Live Video Broadcasting Architecture - High Scalability -High Scalability engineers · High ScalabilityCase Studies
  • Service Workers at SlackSlack engineers · SlackCase Studies

  Open this lesson with its quiz and exercise

• Classic System Designs · 13 minDesign a video streaming serviceUpload, process and stream video to millions: transcoding pipelines, adaptive bitrate and using a CDN without going bankrupt.Read hereClose

  The problem

  A video streaming service lets users upload videos and lets many others watch them smoothly on any device and connection.

  Requirements

  • Upload large files reliably.
  • Playback starts quickly and does not buffer.
  • Works on phones, TVs and browsers, on fast and slow networks.
  • Huge scale, with traffic dominated by viewing, not uploading.
  • Keep cost under control.

  Estimates that shape the design

  • Video files are big: a few GB uploaded, then several versions stored.
  • Views outnumber uploads by orders of magnitude, and bandwidth is the main bill.

  High-level design

  Two separate flows: upload and process, and stream.

  100%

  Figure SDA-CSD-10-1 · Upload-and-process and streaming flows · © forcsguy.com

  Upload

  • The client asks the API for an upload URL and sends the file directly to object storage, in chunks that can resume after a network drop. Your application servers never carry the video bytes.
  • When the upload finishes, storage emits an event that starts processing.

  Processing pipeline

  Raw video cannot be streamed to everyone. The pipeline:

  1. Splits the video into short segments, so they can be processed in parallel.
  2. Transcodes each into several resolutions and bitrates (for example 240p to 4K) and several formats and codecs.
  3. Creates thumbnails, subtitles, and sometimes watermarks.
  4. Runs checks: content moderation, copyright matching, virus scanning.
  5. Writes a manifest listing the available versions.
  6. Updates the metadata (title, status) so the video becomes visible.

  Model it as a graph of tasks run by a workflow engine over a queue and a pool of workers, so steps parallelise, retry independently and scale with load.

  100%

  Figure SDA-CSD-10-2 · Transcoding task graph · © forcsguy.com

  Inside the transcoding pipeline

  Transcoding is heavy and varies by creator (some want watermarks, some supply thumbnails, some upload very high resolution). So define the work as a task graph (a DAG): the original is split into video, audio and metadata; the video stage fans out into encodings at several resolutions and bitrates, a thumbnail and optionally a watermark; the audio stage encodes separately; a final stage merges. Independent branches run in parallel.

  The parts that make it run:

  • Pre-processor splits the video into short independently playable chunks (aligned to groups of pictures), builds the graph from configuration, and keeps the chunks in temporary storage so a failed step can retry without starting over.
  • Graph scheduler cuts the graph into stages and puts the tasks on a queue.
  • Resource manager keeps a priority queue of tasks, a priority queue of worker availability and a list of what is running, and pairs the most important task with the best-suited worker.
  • Workers run specific tasks; temporary storage holds metadata in memory and media in blob storage and is cleaned when the job ends.

  Two vocabulary points: a container (.mp4, .mov) is the wrapper holding video, audio and metadata; a codec (H.264, VP9, HEVC) compresses the content. Raw footage can be hundreds of GB per hour, which is the first reason to transcode; the others are device compatibility and adapting quality to bandwidth.

  Cloud services instead of building everything

  100%

  Figure SDA-CSD-10-3 · Two paths: video and everything else · © forcsguy.com

  100%

  Figure SDA-CSD-10-4 · Upload flow with completion events · © forcsguy.com

  100%

  Figure SDA-CSD-10-5 · Metadata is updated in parallel with the upload · © forcsguy.com

  Streaming

  • The player downloads the manifest, then fetches small segments over plain HTTP (the HLS and MPEG-DASH formats work this way).
  • Adaptive bitrate: the player measures its throughput and switches between quality levels segment by segment, so playback adapts to the connection instead of stopping.
  • Segments are static files, ideal for CDN caching.

  100%

  Figure SDA-CSD-10-6 · One video, many encodings · © forcsguy.com

  100%

  Figure SDA-CSD-10-7 · Popular videos from the CDN, the rest from storage servers · © forcsguy.com

  The CDN and cost

  CDN traffic is expensive. Practical tactics:

  • Serve popular videos from edge caches; keep the long tail only at the origin or in a regional cache.
  • Pre-position new releases that you expect to be popular.
  • Choose bitrates carefully, and use efficient codecs for popular content (extra encoding cost is repaid by lower bandwidth).
  • Large services run their own cache servers inside ISPs to cut transit cost.

  Before: serve from the origin

  100%

  After: serve from a CDN

  100%

  What changed

  • Viewers stream from a nearby edge, so less buffering
  • The origin handles only cache misses
  • Bandwidth cost falls because edges absorb the traffic

  Metadata and other parts

  • A database for video metadata, ownership, comments and counts, with caches for the hot reads.
  • Search and recommendations as separate services fed by events.
  • View counts are written to a queue and aggregated, not updated row by row on every view.
  • Security: signed URLs for private videos, and DRM for premium content.

  Speed, safety and cost, in detail

  • Parallel and resumable uploads: the client cuts the file into chunks and uploads them independently, so a drop resumes at the last chunk. Put upload points near users (a CDN works as an upload edge).
  • Loose coupling: a queue between download, encode and publish steps lets each stage work as soon as items arrive instead of waiting for the previous stage to finish a whole file.
  • Pre-signed upload URLs: the API issues a short-lived address that authorises one upload to one location, so clients never hold storage credentials.
  • Protecting content: DRM, encrypted streams with an authorisation check at playback, and visible watermarks.
  • Cost: a rough sum shows CDN delivery alone could cost on the order of a hundred thousand dollars a day at this scale, and views follow a long tail. So serve only the popular videos from the CDN, serve the rest from your own storage servers, keep fewer encoded versions of rarely watched videos (encode short ones on demand), keep regional favourites in their region, and at very large scale partner with internet providers or run your own delivery network. Always check your own viewing data first.

  Errors. Separate recoverable errors (a chunk fails to encode: retry a few times) from non-recoverable ones (malformed file: stop the job and return a clear error). A playbook per component: retry uploads and encodes; if an old client cannot chunk, chunk on the server; regenerate a graph or reschedule a task if the pre-processor or scheduler fails; use replicas for queues, caches and databases; promote a replica if a primary database fails.

  Live streaming differs in needing lower latency (a different protocol), less parallelism (chunks are small and processed as they arrive) and error handling that cannot afford long retries. Takedowns need both upload-time checks and user flagging.

  100%

  Figure SDA-CSD-10-8 · Chunked, resumable and regional uploads · © forcsguy.com

  100%

  Figure SDA-CSD-10-9 · Pre-signed upload URL · © forcsguy.com

  Failure handling

  Every processing task is idempotent and tracked, so a failed or retried segment does not corrupt the output. Failures go to a dead-letter queue and set a visible error status. Use multiple regions for storage and CDN origins so viewing survives a regional outage.

  Worked numbers: why a CDN is not optional

  Suppose 5 million viewers are watching at the same time, each at an average of 3 Mbit/s.

  • Egress: 5,000,000 x 3 Mbit/s = 15 Tbit/s. No single data centre network, and no sensible origin, can send that. Video is served from edge caches close to viewers; the origin only sends each popular segment to each edge once.
  • Why caching works so well: video segments are the same files for everyone watching, and demand is highly skewed. A popular show may be 90%+ cache hits at the edge, so the origin sees a small fraction of the traffic.
  • Upload side: if 1 million videos are uploaded a day at an average of 300 MB, that is 300 TB a day of originals, before the several encoded versions of each (often 5 to 8 renditions, which together can take as much space as the original or more). The long tail of rarely watched videos is the main storage cost, which is why some systems encode extra renditions only when a video becomes popular.

  100%

  Figure SDA-CSD-10-10 · Fast transcoding by splitting a video into segments · © forcsguy.com

  How the player picks the quality

  Streaming uses adaptive bitrate (ABR): the video exists in several renditions (for example 240p to 1080p), each cut into the same short segments, and the player decides per segment which one to request.

  1. The player downloads the playlist (manifest) listing the renditions and segments.
  2. It starts with a conservative rendition so playback begins quickly.
  3. For each segment it measures how fast the download went and how full its buffer is.
  4. If the buffer is healthy and the network fast, it steps up a rendition; if the buffer is draining, it steps down before the video stalls.

  Because every rendition has identical segment boundaries, switching is seamless. Viewers see a slightly softer picture instead of a spinning wheel.

  100%

  Figure SDA-CSD-10-11 · The player adapts to the network segment by segment · © forcsguy.com

  Mistakes people make

  • Encoding every rendition for every upload. Most videos are barely watched. Encode a base set first and add more when a video proves popular.
  • Serving from the origin. Without a CDN the origin saturates and far-away viewers buffer.
  • Cold caches on a launch. A new popular release hits empty edges. Pre-warm the CDN for expected hits.
  • Uploads that cannot resume. Large files over mobile networks fail often; use chunked, resumable uploads.
  • Forgetting access control. Public segment URLs are easily shared. Use signed, expiring URLs, and DRM where licences require it.
  • Ignoring start-up time. Viewers leave if playback takes more than a couple of seconds. Short first segments, a good first rendition and a nearby edge matter more than peak quality.

  Try it yourself

  Open the System Builder's viral link challenge to see how much traffic a CDN takes off your servers, and the WebRTC lesson for when low-latency live video needs a different approach than segment-based streaming.

  More on this site

  Related reading

  • HTTP/3: the past, the present, and the futureCloudflareBlogs
  • Cloudflare BlogCloudflare engineers · CloudflareBlogs
  • The Road to QUICCloudflareBlogs
  • Cloudflare EngineeringCloudflare engineers · CloudflareBlogs

  Show 6 moreShow fewer

  • High Performance Browser NetworkingIlya Grigorik · O'ReillyBooks
  • MDN: HTTP cachingMDN contributors · MozillaDocumentation
  • Meta Engineering BlogMeta Engineering · MetaBlogs
  • Flannel: An Application-Level Edge Cache to Make Slack ScaleSlack Engineering · SlackCase Studies
  • Netflix ResearchNetflix engineers · NetflixBlogs
  • Nvidia Engineering BlogNvidia engineers · NvidiaBlogs

  Open this lesson with its quiz and exercise

• Core Concepts: How Things Actually Work · 7 minPolling, long polling, SSE and WebSocketsFour ways to get live updates to a browser, and how to choose between them.Read hereClose

  The problem

  HTTP is request and response: the client asks. How does the server tell a browser something now?

  The four options

  Technique	How	Good	Weak
  Short polling	Client asks every N seconds	Trivial, works everywhere	Wasteful, delayed by N
  Long polling	Client asks; the server holds the request until there is news (or a timeout), then the client asks again	Near real time, plain HTTP	A held request per client, awkward at scale
  Server-sent events (SSE)	One long HTTP response streams events	Simple, auto-reconnect, text over HTTP	One-way only
  WebSockets	HTTP upgrades to a two-way connection	Low latency both ways	Stateful connections, more infrastructure

  100%

  Figure SDA-CORE-08-1 · The four options · © forcsguy.com

  Before: polling: keep asking

  100%

  After: WebSocket: server pushes

  100%

  What changed

  • No wasted requests when nothing has changed
  • Updates arrive the moment they happen
  • The server must hold many open connections

  Choosing

  • Updates are rare or delays are fine: polling.
  • The server pushes, the client mostly listens (feeds, scores, notifications, progress): SSE.
  • Both sides send often (chat, games, collaborative editing): WebSockets.

  Running persistent connections

  Connections are long-lived, so capacity is about how many connections a server can hold, not requests per second.

  • Put connection handling in servers that are separate from your stateless API.
  • Use heartbeats to detect dead connections, and reconnect with backoff and jitter.
  • Track which server holds which user so messages can be routed (a presence or session store), and use a pub/sub system between servers.
  • When a client reconnects, let it ask for what it missed (events after id X), because connections drop.
  • Authenticate on connect and again when tokens expire.

  More on this site

  Related reading

  • RFC 6455: The WebSocket ProtocolIETFDocumentation
  • MDN: WebSockets APIMDN Contributors · MozillaDocumentation
  • Uber's Real-Time Push PlatformUber Engineering · UberCase Studies
  • How Discord Scaled Elixir to 5,000,000 Concurrent UsersDiscordCase Studies

  Show 6 moreShow fewer

  • The WhatsApp Architecture Facebook Bought for $19BHigh ScalabilityCase Studies
  • Erlang at Facebook (slides)Eugene Letuchy · FacebookCase Studies
  • Slack EngineeringSlack engineers · SlackBlogs
  • Real-time MessagingSlack Engineering · SlackCase Studies
  • How Uber Scales Their Real-time Market Platform - High Scalability -High Scalability engineers · High ScalabilityCase Studies
  • Real-Time Predictions at DoorDashDoorDash engineers · DoorDashCase Studies

  Open this lesson with its quiz and exercise