Machine Learning (2 parts) at Condé Nast

Ideas you will meet

Distributed Systems

Consensus, time, failure and the fundamentals of scale-out.

Machine Learning Systems

Recommendations, ranking, feature stores and serving models at scale.

What we already know about this

Our own lessons on these ideas. Read them here before you start, so it makes sense.

• Core Concepts: How Things Actually Work · 6 minLatency, throughput and bandwidthThree numbers that are often confused, and why a fast network can still feel slow.Read hereClose

  Three different questions

  • Latency: how long does one request take from start to finish? Measured in milliseconds.
  • Bandwidth: how much could the pipe carry per second? A capacity, measured in bits per second.
  • Throughput: how much actually flows per second, measured in requests or bytes per second. It is at most the bandwidth, and often much less.

  A motorway analogy: bandwidth is the number of lanes, latency is the time one car needs to drive the length, and throughput is how many cars arrive per hour. Widening the road (more lanes) lets more cars through but does not make any single car arrive sooner.

  Why it matters in design

  Many speed problems are really latency problems, and more bandwidth will not fix them:

  Symptom	Likely cause	Lever
  Page makes 80 small requests and feels slow	Round-trip latency adds up	Fewer requests, bundling, HTTP/2, caching
  Users far away complain	Distance	CDN, regional servers
  Big downloads are slow	Throughput or bandwidth	Compression, parallel transfers, bigger pipe
  System is slow only at busy times	Throughput limit	Scale out, queue, shed load

  Image: Syced · CC0 · Source · Wikimedia Commons: Peabird Ethernet switch in 2003

  Where the time goes

  100%

  Figure SDA-CORE-01-1 · A latency budget for one page · © forcsguy.com

  Tail latency

  Averages hide the worst experiences. If 1% of requests take 3 seconds, a page that makes 50 calls will hit one of those slow calls most of the time. Track percentiles (p95, p99), and design to keep the slowest requests under control with timeouts and hedged requests.

  Latency budget

  Break a target (say 300 ms for a page) into a budget: network 80 ms, application 60 ms, database 50 ms, rendering 80 ms, slack 30 ms. When the total exceeds the target you know where to cut.

  Real-world reading

  Related reading

  • Optimizing Web Servers for High Throughput and Low Latency at DropboxDropbox engineers · DropboxCase Studies
  • Natural Language Processing and Content Analysis (2 parts) at Condé NastCondé Nast engineers · Condé NastCase Studies
  • Finding Kafka throughput limit in infrastructure at DropboxDropbox engineers · DropboxCase Studies
  • Scaling Gradient Boosted Trees for Click-Through-Rate Prediction at YelpYelp engineers · YelpCase Studies

  Show 6 moreShow fewer

  • Machine Learning (2 parts) at Condé NastCondé Nast engineers · Condé NastCase Studies
  • DataSift Architecture: Realtime Datamining at 120,000 Tweets Per Second - High Scalability -High Scalability engineers · High ScalabilityCase Studies
  • Deploying Brotli for Static Content at DropboxDropbox engineers · DropboxCase Studies
  • How Twitter Handles 3,000 Images Per Second - High Scalability -High Scalability engineers · High ScalabilityCase Studies
  • ESPN's Architecture at Scale - Operating at 100,000 Duh Nuh Nuhs Per Second - High Scalability -High Scalability engineers · High ScalabilityCase Studies
  • Architecture NotesArchitecture Notes · architecturenotes.coPodcasts & Newsletters

  Open this lesson with its quiz and exercise

• Low-Level Design Practice · 10 minATM machineA machine that behaves differently in each state: the State pattern, card and PIN checks, and cash dispensing with limits.Read hereClose

  The problem

  An ATM accepts a card, checks the PIN, then lets the customer check balance, withdraw cash or deposit, and returns the card. It talks to a bank to authorise operations and holds a limited amount of cash.

  States

  100%

  Figure SDA-LLD-06-1 · ATM states · © forcsguy.com

  In Idle, "enter PIN" makes no sense; in Authenticated, "insert card" makes no sense. The State pattern puts those rules inside state objects instead of if (state === ...) chains.

  interface AtmState {
    insertCard(atm: Atm, card: Card): void;
    enterPin(atm: Atm, pin: string): void;
    withdraw(atm: Atm, cents: number): void;
  }
  
  class Idle implements AtmState {
    insertCard(atm, card) { atm.card = card; atm.setState(new HasCard()); }
    enterPin() { throw new Error("insert a card first"); }
    withdraw() { throw new Error("insert a card first"); }
  }
  class HasCard implements AtmState {
    tries = 0;
    insertCard() { throw new Error("card already inserted"); }
    enterPin(atm, pin) {
      if (atm.bank.checkPin(atm.card!, pin)) atm.setState(new Authenticated());
      else if (++this.tries >= 3) { atm.retainOrEject(); atm.setState(new Idle()); }
    }
    withdraw() { throw new Error("enter the PIN first"); }
  }
  

  Who is responsible for what

  100%

  Figure SDA-LLD-06-2 · Withdrawal sequence · © forcsguy.com

  • Bank owns the balance and account rules. ATM owns hardware, cash on hand and the session.
  • Cash dispenser holds counts per denomination and decides which notes to give (greedy from the largest, with a fallback).
  • Every step is logged so a failure between "dispensed" and "confirmed" can be reconciled later (a mismatch report, or an automatic reversal).

  Design notes

  • Treat PINs as secrets: never store or log them; the bank verifies, the ATM only forwards an encrypted PIN block.
  • Idempotency: a retried authorise call must not debit twice; use a transaction id.
  • Timeouts: an idle session returns to Idle and ejects the card.
  • Extensions: deposits (count and hold until verified), fund transfers, receipts, and multiple card types.

  Real-world reading

  Related reading

  • Apple Machine Learning BlogApple engineers · AppleBlogs
  • AWS Machine Learning BlogAmazon Web Services engineers · Amazon Web ServicesBlogs
  • Optimizing Payments with Machine Learning at DropboxDropbox engineers · DropboxCase Studies
  • Machine Learning Platform at YelpYelp engineers · YelpCase Studies

  Show 6 moreShow fewer

  • PyKrylov: Accelerating Machine Learning Research at eBayeBay engineers · eBayCase Studies
  • Machine Learning (2 parts) at Condé NastCondé Nast engineers · Condé NastCase Studies
  • Machine Learning Platform at ZalandoZalando engineers · ZalandoCase Studies
  • Secret Detector: Preventing Secrets in Source Code at YelpYelp engineers · YelpCase Studies
  • Machine Learning at Jane StreetJane Street engineers · Jane StreetCase Studies
  • Waymo BlogWaymo engineers · WaymoBlogs

  Open this lesson with its quiz and exercise

• Low-Level Design Practice · 11 minElevator systemRequests, directions and scheduling: a state machine per car and a dispatcher that decides which car answers which call.Read hereClose

  The problem

  A building has several elevators. People press a button on a floor (a hall call with direction) and a floor button inside the car (a car request). The system moves cars, opens doors and decides which car should answer each call.

  Entities

  100%

  Figure SDA-LLD-07-1 · Elevator classes · © forcsguy.com

  One elevator as a state machine

  100%

  Figure SDA-LLD-07-2 · Elevator car states · © forcsguy.com

  A simple and effective scheduling rule is the elevator (scan) algorithm: keep two sorted sets of stops (above and below); continue in the current direction, serving stops in order; reverse only when none are left in that direction.

  class Elevator {
    floor = 0; dir: "UP" | "DOWN" | "IDLE" = "IDLE";
    up = new SortedSet<number>(); down = new SortedSet<number>();   // pending stops
  
    addStop(f: number) { (f >= this.floor ? this.up : this.down).add(f); if (this.dir === "IDLE") this.pick(); }
    private pick() { this.dir = this.up.size ? "UP" : this.down.size ? "DOWN" : "IDLE"; }
    step() {                                   // called once per tick
      if (this.dir === "UP") { this.floor++; if (this.up.deleteIfPresent(this.floor)) this.openDoors(); if (!this.up.size) this.pick(); }
      if (this.dir === "DOWN") { this.floor--; if (this.down.deleteIfPresent(this.floor)) this.openDoors(); if (!this.down.size) this.pick(); }
    }
  }
  

  Many cars: the dispatcher

  100%

  Figure SDA-LLD-07-3 · Assigning a hall call · © forcsguy.com

  Make the dispatcher a strategy so you can swap "nearest car" for "least loaded" or a smarter estimate without touching Elevator.

  Design notes

  • Concurrency: calls arrive while cars move. Use one event loop per elevator, or a queue per car, so state changes happen one at a time.
  • Safety: doors must not close on a blocked sensor; weight limits stop a car from moving.
  • Peak modes: morning (send idle cars to the lobby) and evening patterns are policies.
  • Extensions: express cars, destination dispatch (people enter their destination at the hall), emergency and maintenance modes.

  Real-world reading

  Related reading

  • Apple Machine Learning BlogApple engineers · AppleBlogs
  • AWS Machine Learning BlogAmazon Web Services engineers · Amazon Web ServicesBlogs
  • Machine Learning for Ranking Answers End-to-End at QuoraQuora engineers · QuoraCase Studies
  • Machine Learning (2 parts) at Condé NastCondé Nast engineers · Condé NastCase Studies

  Show 6 moreShow fewer

  • Google SRE Book: Distributed Periodic Scheduling with CronGoogle SRE team · GoogleBlogs
  • Please stop calling databases CP or APMartin Kleppmann · martin.kleppmann.comBlogs
  • Large-Scale Machine Learning Pipeline for Job Recommendations at IndeedIndeed engineers · IndeedCase Studies
  • Machine Learning Platform at YelpYelp engineers · YelpCase Studies
  • Engineering at QuoraQuora engineers · QuoraCase Studies
  • Distributed SchedulingUmbcPapers

  Open this lesson with its quiz and exercise