Scaling with Performance at Facebook - Bill Jia, VP of Infrastructure at Facebook

Ideas you will meet

Distributed Systems

Consensus, time, failure and the fundamentals of scale-out.

What we already know about this

Our own lessons on these ideas. Read them here before you start, so it makes sense.

• Interview Preparation · 14 minPractice problem bankSixty prompts grouped by theme, each pointing to the lessons that teach its ingredients, so practice builds on what you have learned.Read hereClose

  How to use this bank

  1. Pick a prompt, set a timer (45 minutes for system design, 30 for low-level design).
  2. Work the method from the previous lesson: clarify, estimate, sketch, deep-dive, wrap up.
  3. Compare with the lessons listed for that prompt. Note what you missed.
  4. Repeat later without notes.

  Prompts are grouped by core challenge, because most problems are variations on a handful of patterns. "Lessons" point to this site's courses: FND foundations, CSD classic designs, CORE core concepts, RWS real-world systems, BDP big data, SDE software design, LLD low-level design, ADS advanced distributed systems, DDV databases, NSD networking and security, MSV messaging and microservices.

  1. Simple services at scale

  Prompt	Focus	Lessons
  A link-shortening service	Key generation, redirects, read-heavy caching	FND URL shortener; CSD unique ids
  A paste-sharing service	Storage of large text, expiry, abuse limits	FND databases; RWS large uploads
  A file-sharing service with expiring links	Object storage, signed URLs	CSD cloud file storage; RWS uploads
  A public status page	High availability for read-only content, CDN	FND networking and CDN; ADS cloud architecture
  A feature-flag service	Low-latency reads, safe defaults, targeting	MSV operating microservices
  A configuration distribution service	Consistency, versioning, propagation	ADS consensus; MSV discovery

  2. Rate, limits and counters

  Prompt	Focus	Lessons
  A rate limiter for a public API	Algorithms, shared counters, races	CSD rate limiter
  A global limiter across regions	Eventual consistency and over-admission	CSD rate limiter; DDV replication
  Usage metering and billing per customer	Accurate counting, aggregation, idempotency	RWS metrics monitoring; MSV reliable messaging
  A system to detect and block abusive traffic	Signals, thresholds, false positives	NSD proxies and attacks; CSD rate limiter
  A leaderboard for millions of players	Sorted sets, sharding, ranking	CORE Redis; ADS Redis internals

  3. Storage and data

  Prompt	Focus	Lessons
  A distributed key-value store	Partitioning, replication, quorums, repair	CSD key-value store; ADS storage engines
  A distributed cache	Eviction, consistency, hot keys	FND caching; CORE Redis
  A cloud file storage and sync service	Chunking, dedup, conflicts, notifications	CSD cloud file storage
  An object storage service	Durability, erasure coding, metadata	CORE erasure coding; RWS storage
  A time-series metrics database	Write-heavy ingestion, downsampling	DDV choosing a database; RWS metrics monitoring
  A multi-tenant SaaS database	Isolation models, noisy neighbours	DDV replication; FND replication and sharding
  A search service for a catalogue	Inverted index, relevance, freshness	CSD autocomplete; DDV choosing a database
  A deduplication system for uploaded media	Hashing, similarity, storage	CSD cloud storage; CORE Bloom filters

  4. Feeds, social and communication

  Prompt	Focus	Lessons
  A news feed or timeline	Fan-out on write or read, caching	CSD news feed; RWS Twitter
  A photo-sharing app	Media pipeline, feed, CDN	CSD news feed; CSD video streaming
  A chat application with presence	Persistent connections, ordering, offline	CSD chat; CORE realtime
  A notification system for push, SMS and email	Queues, retries, dedupe, preferences	CSD notification system; MSV reliable messaging
  Nested comments at scale	Tree storage, pagination, hot threads	DDV normalisation; DDV indexes
  Live score updates	Push at scale, fan-out, caching	CORE realtime; ADS load balancing
  A video conferencing service	Real-time media, signalling, scaling rooms	CORE realtime communication
  A collaborative whiteboard or editor	Merging concurrent edits	RWS collaborative editing

  5. Location and matching

  Prompt	Focus	Lessons
  Nearby places search	Geohash, quadtree, indexing	RWS nearby places
  A ride-hailing service	Matching, location updates, trip state	LLD cab booking; RWS nearby places
  A maps and navigation service	Tiles, routing, traffic	RWS maps and navigation
  A food-delivery platform	Orders, courier assignment, status	LLD food delivery
  A delivery route planner	Optimisation heuristics, batching	RWS maps; LLD cab booking

  6. Media

  Prompt	Focus	Lessons
  A video streaming platform	Upload, transcoding, adaptive streaming, CDN	CSD video streaming
  A music streaming service	Catalogue, playback, recommendations	LLD video catalogue; CSD video streaming
  A large file upload service	Chunking, resume, integrity	RWS large uploads
  A live-streaming platform	Low latency, ingest, fan-out	CSD video streaming; CORE realtime

  7. Commerce and money

  Prompt	Focus	Lessons
  A payment processing system	Idempotency, ledger, reconciliation	RWS payment system; CORE payments
  A cross-border money transfer	FX, compliance, settlement	RWS cross-border payments
  A flash sale or ticket drop	Hot items, queues, oversell prevention	RWS flash sale; LLD movie ticket booking
  An e-commerce inventory service	Stock reservation, consistency	LLD shopping cart; CORE locking
  A stock exchange or order matching engine	Order book, determinism, latency	RWS stock exchange
  A subscription billing system	Recurring charges, retries, proration	RWS payment system; MSV reliable messaging
  A fraud detection service	Features, streaming, latency	BDP batch and stream; MSV event-driven
  A hotel or seat booking system	Availability, holds, double-booking	LLD hotel management; LLD movie booking

  8. Infrastructure and platform

  Prompt	Focus	Lessons
  A web crawler	Frontier, politeness, dedup	CSD web crawler
  A distributed job scheduler or cron	Leader election, retries, exactly-once effects	CORE distributed locks; ADS consensus
  A distributed lock service	Leases, fencing, consensus	CORE distributed locks; ADS consensus
  A unique id generator	Time-based ids, clocks	CSD unique id generator
  A load balancer or API gateway	Algorithms, health, L4 versus L7	ADS load balancing
  A metrics and alerting platform	Collection, storage, alerts, SLOs	RWS metrics; MSV observability
  A log aggregation pipeline	Collection, indexing, retention	MSV operating microservices
  A CI/CD and deployment system	Pipelines, strategies, rollbacks	CORE deployment strategies; RWS delivery
  A secrets management service	Encryption, access control, rotation	NSD OWASP; MSV operating microservices
  An uptime monitoring service	Probes, global vantage points, alerting	MSV observability
  An email delivery service	Queues, reputation, bounces	RWS email system; CSD notification

  9. Machine learning and data

  Prompt	Focus	Lessons
  A recommendation system	Offline training, online serving, features	BDP batch and stream; LLD video catalogue
  Serving model predictions at scale	Latency, batching, caching	ADS cloud architecture; RWS AI coding assistant
  A real-time analytics dashboard	Streaming aggregation, storage	BDP batch and stream; DDV choosing
  A data pipeline for clickstream events	Ingest, stream processing, warehouse	BDP lakes and warehouses; MSV brokers
  An AI coding assistant	Context building, latency, privacy	RWS AI coding assistant

  10. Low-level design prompts

  Practise these as classes and code; see the LLD course for worked solutions: parking lot, library, hotel, elevator, ATM, cab booking, food delivery, movie booking, expense sharing, shopping cart, chess, snake, chat, social feed and video catalogue. Extra prompts: a vending machine, a ticketing helpdesk, a task scheduler, a rate limiter as a class library, a text editor with undo, an in-memory file system, a logging framework and a cache with pluggable eviction.

  After each attempt

  • Did I clarify and estimate?
  • Is the core flow clear and correct?
  • Did I choose storage with a reason?
  • Did I handle scale, failure, consistency, security and monitoring?
  • Where did I hesitate? Review that lesson, then retry the problem later.

  Real-world reading

  Related reading

  • How we built rate limiting capable of scaling to millions of domainsCloudflare engineering · CloudflareCase Studies
  • Designing robust and predictable APIs with idempotencyBrandur Leach · StripeBlogs
  • Maglev: A Fast and Reliable Software Network Load Balancer (paper)Eisenbud et al. · GooglePapers
  • H3: Uber's Hexagonal Hierarchical Spatial IndexUber Engineering · UberCase Studies

  Show 6 moreShow fewer

  • Stripe's payments APIs: the first 10 yearsStripeCase Studies
  • Scaling Your API with Rate LimitersPaul Tarjan · StripeBlogs
  • TAO: Facebook's Distributed Data Store for the Social Graph (paper)Bronson et al. · Meta (Facebook)Papers
  • Rate Limiting with NGINXNGINX · F5 / NGINXBlogs
  • Solving Large-scale Data Center and Cloud Interconnection Problems - Ihab Tarazi, CTO at EquinixEquinix engineers · EquinixCase Studies
  • My Scaling Hero - Jeff Atwood (a dose of Endorphins before your interview, JK)Codinghorror engineers · CodinghorrorBlogs

  Open this lesson with its quiz and exercise