Performance Optimization on Image, Video, Page Load

Ideas you will meet

Observability

Metrics, logs, traces, SLOs and alerting.

What we already know about this

Our own lessons on these ideas. Read them here before you start, so it makes sense.

• System Design Foundations · 9 minObservability and running a systemMetrics, logs and traces, how to define 'healthy' with SLOs, and how to ship changes without breaking things.Read hereClose

  You cannot fix what you cannot see

  A system you cannot observe is a system you cannot trust. Observability is the ability to understand what is happening inside from the outside. Three kinds of signal work together.

  Signal	What it is	Best for
  Metrics	Numbers over time (requests per second, error rate, CPU)	Dashboards, alerts, trends
  Logs	Timestamped records of events	Details of what happened
  Traces	The path of one request through many services	Finding where time is spent

  100%

  Figure SDA-FND-11-1 · From alert to root cause · © forcsguy.com

  What to measure

  A simple checklist, the four golden signals:

  1. Latency: how long requests take (use percentiles).
  2. Traffic: how much demand there is.
  3. Errors: how many requests fail.
  4. Saturation: how full the busiest resource is (CPU, memory, connections, disk, queue depth).

  Averages hide pain. If the average is 100 ms but p99 is 4 seconds, one user in a hundred is waiting four seconds, and with many requests per page that is most of your users sometimes.

  Defining healthy: SLIs, SLOs, error budgets

  • An SLI (indicator) is a measurement, like "fraction of requests answered successfully in under 300 ms".
  • An SLO (objective) is the target, like "99.9% over 30 days".
  • The error budget is what is left over: 0.1% of requests may fail. While there is budget, ship features; when it runs out, focus on reliability.

  Alert on symptoms users feel (error rate, latency) more than on causes (one machine's CPU). Page a human only for things that need action now.

  Ship changes safely

  Most outages come from changes. Reduce the risk:

  • Small, frequent releases are easier to understand and undo than big ones.
  • Canary release: send a few percent of traffic to the new version and watch the metrics before going wider.
  • Feature flags: deploy code turned off, then enable it gradually, and switch it off instantly if needed.
  • Blue-green: keep the old version running so you can switch back in seconds.
  • Backwards-compatible changes: database and API changes should work with both old and new code during rollout.
  • Fast rollback must be a rehearsed, boring button.

  When things go wrong

  Have an incident process: someone leads, someone communicates, mitigate first and investigate later. Afterwards write a blameless postmortem that asks "how did our system and process allow this?" rather than "who did this?". That is how reliability actually improves.

  A worked example: error budgets and burn rate

  Your service has an SLO of 99.9% of requests succeed over 30 days, and it serves 10 million requests a month.

  • Error budget: 0.1% of 10,000,000 = 10,000 failed requests may happen in the month before you break the promise.
  • Burn rate is how fast you spend it. A burn rate of 1 uses exactly the budget over 30 days. If 5% of requests are failing now, the error rate is 50 times the allowed 0.1%, a burn rate of 50: the whole month's budget is gone in 30 / 50 = 0.6 days, about 14 hours.
  • Alert on burn rate, not on a single bad minute. A common pattern watches two windows together: page someone if the burn rate over the last hour is above about 14 and the last 5 minutes agree (a fast burn: 2% of the monthly budget gone in an hour). Open a ticket for a slower burn, say rate above 6 over 6 hours. Two windows avoid paging for blips and avoid missing slow leaks.

  Why this beats "alert when error rate is above X": it ties alerts to what users were promised, it pages only when the budget is in danger, and it gives the team a rule for trade-offs: when the budget is healthy, ship faster; when it is nearly spent, slow down and fix reliability.

  100%

  Figure SDA-FND-11-2 · Alerting on how fast the error budget burns · © forcsguy.com

  Finding the cause with all three signals

  A worked investigation shows why you want metrics, traces and logs together.

  1. A metric alerts: the checkout success rate dropped and p99 latency rose. Metrics tell you that something is wrong and how bad, cheaply and over time.
  2. Narrow it down by dimension: break the metric down by region, version and endpoint. Only the new version in one region is affected, a strong hint it is a deploy.
  3. A trace shows where the time goes: one slow checkout request shows 4.8 of its 5 seconds in a call to the payment provider's tokenisation endpoint.
  4. Logs give the detail: filter logs by the trace id from that request and read the exact error: a timeout talking to a new tokenisation URL.
  5. Fix and verify: roll back the version, and watch the metric recover.

  100%

  Figure SDA-FND-11-3 · From an alert to a root cause · © forcsguy.com

  The glue is a correlation id (trace id) carried on every request and written into every log line, so you can jump from one signal to the others.

  Mistakes people make

  • Alerting on causes instead of symptoms. "CPU is 85%" may be fine; "users are seeing errors" never is. Page on symptoms users feel, and use cause metrics for diagnosis.
  • Averaging percentiles. The mean of two hosts' p99 is meaningless; merge distributions instead (see the t-digest lesson).
  • High-cardinality labels. A label such as user_id on a metric creates a new time series per user, which can multiply storage and cost by millions. Put such detail in logs or traces.
  • Logs without structure or ids. Free text is hard to query. Log JSON with a trace id, service and version.
  • Dashboards no one owns. A wall of graphs nobody reads is noise. Build a small dashboard per service around the golden signals: latency, traffic, errors and saturation.
  • No runbook. An alert that does not say what to check first wastes the first ten minutes of every incident.

  Try it yourself

  Run any test in the System Builder and read its charts as if they were production graphs: which signal moves first, and which part would you look at next? Build the search metrics lab to practise percentile maths, and read the t-digest lesson for how real systems summarise latency at scale.

  Real-world reading

  Related reading

  • The Site Reliability WorkbookGoogle SRE team · O'Reilly / GoogleBooks
  • Feature Toggles (aka Feature Flags)Pete Hodgson · Martin FowlerBlogs
  • InfluxData BlogInfluxData engineers · InfluxDataBlogs
  • MetricsDB: TimeSeries Database for storing metrics at TwitterTwitter engineers · TwitterCase Studies

  Show 6 moreShow fewer

  • The Architecture Twitter Uses to Deal with 150M Active Users, 300K QPS, a 22 MB/S Firehose, and Send Tweets in Under 5 SecondsHigh Scalability engineers · High ScalabilityCase Studies
  • Steady State: Always Put Logs on Separate DiskMicrosoft engineers · MicrosoftCase Studies
  • XCMetrics: All-in-One Tool for Tracking Xcode Build Metrics at SpotifySpotify engineers · SpotifyCase Studies
  • Honeycomb BlogHoneycomb engineers · HoneycombBlogs
  • BookKeeper: Distributed Log Storage at YahooYahoo engineers · YahooCase Studies
  • Analyzing V8 Garbage Collection Logs at AlibabaAlibaba engineers · AlibabaCase Studies

  Open this lesson with its quiz and exercise

• Messaging and Microservice Patterns · 13 minObservability: metrics, SLIs, SLOs and alertingTurning raw signals into targets people can act on: the three pillars, the four golden signals, SLIs, SLOs, error budgets and alerts that wake you only when it matters.Read hereClose

  Monitoring versus observability

  Monitoring watches known questions ("is the error rate high?"). Observability is the ability to ask new questions of a running system from the data it emits, which matters when failures are unexpected. It rests on three signal types, often called the pillars:

  Signal	What it is	Good for
  Metrics	Numbers over time (counts, rates, durations)	Dashboards, alerts, trends; cheap to store
  Logs	Timestamped event records	Detail about what happened in one place
  Traces	A request's path across services	Finding where time and errors come from

  Together with context (deploy markers, config changes) they let you move from "something is wrong" to "this is why". (Collection architecture is in the metrics monitoring lesson; instrumentation in the previous lesson.)

  What to measure

  Two handy checklists:

  • The four golden signals (for services): latency (separate successful from failed), traffic (demand), errors (rate of failures) and saturation (how full the most constrained resource is).
  • RED for request-driven services (Rate, Errors, Duration) and USE for resources (Utilisation, Saturation, Errors).

  Use percentiles (p50, p95, p99) rather than averages for latency, because averages hide the slow tail that users feel (see the latency lesson). Keep label cardinality low (do not label metrics with user ids).

  SLIs, SLOs and SLAs

  100%

  Figure SDA-MSV-06-1 · From measurement to promise · © forcsguy.com

  • SLI (indicator): a ratio of good events to valid events, such as successful requests / valid requests or requests under 300 ms / all requests.
  • SLO (objective): a target for the SLI over a window. Few, meaningful SLOs beat many.
  • SLA (agreement): the contractual promise and what happens if missed; keep it looser than the SLO so you have room to react.
  • Error budget = 1 - SLO. With 99.9 percent over 30 days you may be "down" about 43 minutes. Common reference points: 99 percent ≈ 7.2 hours a month, 99.9 percent ≈ 43 minutes, 99.99 percent ≈ 4.3 minutes.

  Using the budget: while budget remains, ship features and take risks; when it is nearly spent, prioritise reliability work. This turns arguments between "ship faster" and "be safer" into a shared, measurable policy.

  Alerting that deserves a human

  Alert fatigue is a reliability problem: people ignore noisy pages.

  • Alert on symptoms, not causes: "users see errors" rather than "one pod restarted". Keep cause metrics for dashboards and diagnosis.
  • Every page must be urgent, actionable and with a runbook; if not, make it a ticket or delete it.
  • Use burn-rate alerts: page when the error budget is being consumed much faster than allowed (for example 2 percent of a 30-day budget in an hour), ticket for slow burns. Use two windows (short and long) to be fast but not flappy.
  • Group and deduplicate related alerts; route by ownership; add severity.
  • Review alerts after incidents and prune them regularly.

  100%

  Figure SDA-MSV-06-2 · From signal to action · © forcsguy.com

  The tools, in one table

  Need	Common tools
  Metrics collection and alert rules	Prometheus (pull model), cloud metrics, Datadog
  Dashboards	Grafana, vendor dashboards
  Logs	ELK or OpenSearch stack, Loki, Splunk, cloud logging
  Traces	Jaeger, Zipkin, Tempo, vendor APM
  Instrumentation standard	OpenTelemetry
  Incident paging	PagerDuty, Opsgenie and similar

  Dashboards that help

  One overview per service showing the golden signals and SLO burn; drill-down panels for dependencies and resources; deploy and config markers on graphs; and consistent layout across services so anyone on call can read any of them.

  Incidents and learning

  Have on-call rotations with clear escalation, runbooks for known failures, and blameless post-incident reviews that produce concrete follow-ups. Practise with game days. Reliability is a process as much as a technology: measure, set targets, spend the budget wisely, and learn from every failure.

  Real-world reading

  Related reading

  • The Site Reliability WorkbookGoogle SRE team · O'Reilly / GoogleBooks
  • Google SRE Book: Service Level ObjectivesGoogle SRE team · GoogleBlogs
  • Alerting on Service-Level Objectives (SLOs) at SoundCloudSoundCloud engineers · SoundCloudCase Studies
  • PrometheusPrometheus · CNCF / PrometheusGitHub

  Show 6 moreShow fewer

  • Prometheus overviewPrometheus · CNCF / PrometheusDocumentation
  • Google SRE Book: Monitoring Distributed SystemsGoogle SRE team · GoogleBlogs
  • Grafana LabsGrafana Labs engineers · Grafana LabsBlogs
  • Monitoring and Alert System using Graphite and Cabot at HackerEarthHackerEarth engineers · HackerEarthCase Studies
  • Kotlin at DoorDashDoorDash engineers · DoorDashCase Studies
  • ThirdEye: Monitoring Platform at LinkedInLinkedIn engineers · LinkedInCase Studies

  Open this lesson with its quiz and exercise

• Classic System Designs · 11 minDesign a web crawlerDiscover and download billions of pages politely and efficiently: the URL frontier, deduplication and avoiding traps.Read hereClose

  The problem

  A web crawler starts from some known URLs, downloads each page, extracts the links and follows them. Search engines use crawlers to build their index, and others use them for archiving, price tracking or monitoring.

  Requirements

  • Scale: about a billion pages a month.
  • Polite: do not overload sites, follow robots.txt.
  • Robust: bad HTML, huge files, slow servers and traps must not break it.
  • Extensible: later add new content types (images, PDFs).
  • Avoid fetching the same page twice.

  Estimates

  1 billion pages a month is about 400 pages a second on average. At roughly 500 KB each (HTML plus metadata), that is around 500 TB a month, so storage and network are the big costs.

  High-level design

  100%

  Figure SDA-CSD-05-1 · Crawler pipeline · © forcsguy.com

  1. The URL frontier hands out URLs to crawl next.
  2. Fetchers download pages (resolving names with a local DNS cache, since DNS lookups can dominate time).
  3. The parser validates and processes the HTML.
  4. A content-seen check drops pages identical to ones already stored.
  5. Links are extracted, normalised, filtered, and checked against URL-seen before going back to the frontier.

  The web as a graph

  100%

  Figure SDA-CSD-05-2 · Crawling is graph traversal · © forcsguy.com

  The URL frontier

  The frontier is the brain. It decides what to fetch and when.

  • Priority: important or frequently changing pages first. Priority queues ordered by a score (for example link popularity or freshness).
  • Politeness: one queue per host, and each host is handled by one worker that waits between requests (for example one request per second to a given site).

  100%

  Figure SDA-CSD-05-3 · URL frontier: priority and politeness · © forcsguy.com

  Because the frontier is huge, keep most of it on disk with an in-memory buffer at the front.

  Keeping one polite worker per host

  100%

  Figure SDA-CSD-05-4 · Politeness queues · © forcsguy.com

  Not doing the same work twice

  • URL seen: before queueing a link, check it. A Bloom filter (or a hash set partitioned across servers) keeps this small and fast. Normalise URLs first (lowercase host, remove fragments, sort query parameters).
  • Content seen: pages often exist under many URLs. Compare a hash of the content, or a similarity fingerprint for near duplicates, and skip repeats.

  Staying out of trouble

  • robots.txt: fetch and cache it per host, and obey it.
  • Spider traps: endless generated URLs. Cap URL length, link depth and pages per host.
  • Timeouts and size limits so one slow or giant response cannot stall a worker.
  • Dynamic pages: content built by JavaScript needs a rendering step, which is much more expensive.
  • Harmful content: filter spam or illegal material before storing or indexing.

  More ways to make it fast and robust

  • Two-stage frontier. Front queues carry priority (each queue has a score band and is picked with a bias toward the higher bands); back queues carry politeness (one host per queue). A URL passes through both, so you get "important first" and "gentle per site" together.
  • Freshness. Recrawl by observed change history, and re-visit important pages more often. Recrawling everything is too costly.
  • DNS is a hidden bottleneck. Lookups can take tens to hundreds of milliseconds and many resolver interfaces block a thread, so keep your own DNS cache and refresh it on a schedule.
  • Locality and timeouts. Put crawl servers near the sites they fetch from, and give each fetch a short timeout so one slow host cannot stall a worker.
  • Robots rules are fetched once and cached, then obeyed before each fetch.
  • Resilience. Spread URLs over downloaders with consistent hashing, checkpoint the crawl state so a crash resumes instead of restarts, and validate pages before storing them.
  • Extensible by modules. New content types (images, a copyright monitor) plug in after the downloader without redesigning the pipeline.
  • Noise. Skip ad blocks, spam and low-value pages with a quality filter; cap URL length and per-site page counts to limit spider traps, and have a person exclude sites that still trap you.
  • Beyond search. The same engine serves archiving, mining and brand or copyright monitoring.

  100%

  Figure SDA-CSD-05-5 · Plugging new modules into the crawler · © forcsguy.com

  100%

  Figure SDA-CSD-05-6 · Distributing the crawl · © forcsguy.com

  Scaling and reliability

  • Add many fetcher workers, partitioned by host (consistent hashing keeps assignments stable).
  • Make workers stateless and checkpoint the frontier, so a crashed worker's URLs are re-queued.
  • Spread crawlers across regions to be closer to the sites being fetched.
  • Re-crawl by schedule: pages that change often (news) are revisited more often than static ones.

  Worked numbers: why it is about many hosts, not one fast fetcher

  Target: 1 billion pages a month, about 400 pages per second, at about 500 KB each.

  • Bandwidth: 400 x 500 KB = 200 MB/s, about 1.6 Gbit/s sustained, before retries and robots.txt fetches. Many machines in several places, not one big link.
  • Politeness sets the shape. If you allow one request per second per host, then to fetch 400 pages a second you must be talking to at least 400 different hosts at the same moment. A single site with 100,000 pages would take 100,000 seconds, nearly 28 hours, to crawl at that pace. So a crawler is a scheduler that interleaves thousands of hosts, which is exactly what the per-host queues do.
  • Seen-URL memory. With 10 billion known URLs, an exact set of 8-byte hashes is 80 GB. A Bloom filter at about 10 bits per URL (1% false positives) needs 12.5 GB, and a false positive only means skipping a page you could have crawled.
  • Storage: 1 billion x 500 KB = 500 TB a month raw. Compress and deduplicate, and keep only what you will index.

  100%

  Figure SDA-CSD-05-7 · Throughput comes from interleaving many hosts · © forcsguy.com

  Staying fresh: not every page needs the same schedule

  Re-crawling everything monthly wastes effort on pages that never change and misses news that changes hourly. Practical schedules:

  • Keep the observed change history per page (did the content hash change since the last visit?). Visit pages that change often more frequently, and stable pages less often, within limits.
  • Use hints: Last-Modified, ETag and conditional requests (If-None-Match) let the server answer "not modified" in a few bytes.
  • Give importance a vote: popular pages, and pages that many others link to, are revisited sooner.
  • Keep a small fast lane for news and feeds, driven by sitemaps and feed updates.

  100%

  Figure SDA-CSD-05-8 · Recrawl by how often a page changes · © forcsguy.com

  Walking through a spider trap

  A calendar page links to "next month", which links to the next month, forever. A naive crawler follows it, fetching millions of near-identical pages from one host. Defences, layered: cap URL length and path depth; cap pages per host per period; detect repeating patterns (the same path with only a number changing); skip pages whose content hash you have seen; and let an operator add a rule when a site still traps you. None is perfect, so budgets per host are the safety net.

  Mistakes people make

  • Ignoring robots.txt and crawl-delay. It gets you blocked, and it is rude.
  • Not normalising URLs. HTTP://Example.com/a?b=1&a=2#top and http://example.com/a?a=2&b=1 are the same page. Lowercase the host, drop fragments, sort parameters, remove tracking parameters.
  • One slow host stalling workers. Always use timeouts and size limits.
  • Treating the frontier as a plain FIFO. It ignores priority and politeness.
  • Fetching JavaScript-built pages as plain HTML. The content may be missing; rendering is a separate, costlier pipeline.
  • No deduplication of content. The same article under ten URLs wastes storage and pollutes the index.

  Try it yourself

  Build the Bloom filter lab and measure its false-positive rate and memory against an exact set, then read the seen filtering lesson. The consistent hash ring lab shows how to assign hosts to crawler workers so assignments stay stable.

  Real-world reading

  Related reading

  • System Design (Karan Pratap Singh)Karan Pratap Singh · karanpratapsinghGitHub
  • CAP Twelve Years Later: How the Rules Have ChangedEric Brewer · InfoQBlogs
  • Tech Dummies: System Design Case StudiesNarendra L · Tech DummiesYouTube
  • Tumblr Architecture - 15 Billion Page Views a Month and Harder to Scale than Twitter - High Scalability -High Scalability engineers · High ScalabilityCase Studies

  Show 6 moreShow fewer

  • Cloudflare Learning Center: What is a CDN?CloudflareDocumentation
  • Scaling Pinterest - From 0 to 10s of Billions of Page Views a Month in Two Years - High Scalability -High Scalability engineers · High ScalabilityCase Studies
  • The Site Reliability WorkbookGoogle SRE team · O'Reilly / GoogleBooks
  • Performance Optimization on Image, Video, Page LoadGoogle engineers · GoogleCase Studies
  • Site Reliability Engineering at Dropbox - Tammy Butow, Site Reliability Engineering Manager at DropboxDropboxYouTube
  • The WhatsApp Architecture Facebook Bought For $19 Billion - High Scalability -High Scalability engineers · High ScalabilityCase Studies

  Open this lesson with its quiz and exercise