Machine Learning Platform at Yelp

Ideas you will meet

Distributed Systems

Consensus, time, failure and the fundamentals of scale-out.

Machine Learning Systems

Recommendations, ranking, feature stores and serving models at scale.

What we already know about this

Our own lessons on these ideas. Read them here before you start, so it makes sense.

• Low-Level Design Practice · 10 minATM machineA machine that behaves differently in each state: the State pattern, card and PIN checks, and cash dispensing with limits.Read hereClose

  The problem

  An ATM accepts a card, checks the PIN, then lets the customer check balance, withdraw cash or deposit, and returns the card. It talks to a bank to authorise operations and holds a limited amount of cash.

  States

  100%

  Figure SDA-LLD-06-1 · ATM states · © forcsguy.com

  In Idle, "enter PIN" makes no sense; in Authenticated, "insert card" makes no sense. The State pattern puts those rules inside state objects instead of if (state === ...) chains.

  interface AtmState {
    insertCard(atm: Atm, card: Card): void;
    enterPin(atm: Atm, pin: string): void;
    withdraw(atm: Atm, cents: number): void;
  }
  
  class Idle implements AtmState {
    insertCard(atm, card) { atm.card = card; atm.setState(new HasCard()); }
    enterPin() { throw new Error("insert a card first"); }
    withdraw() { throw new Error("insert a card first"); }
  }
  class HasCard implements AtmState {
    tries = 0;
    insertCard() { throw new Error("card already inserted"); }
    enterPin(atm, pin) {
      if (atm.bank.checkPin(atm.card!, pin)) atm.setState(new Authenticated());
      else if (++this.tries >= 3) { atm.retainOrEject(); atm.setState(new Idle()); }
    }
    withdraw() { throw new Error("enter the PIN first"); }
  }
  

  Who is responsible for what

  100%

  Figure SDA-LLD-06-2 · Withdrawal sequence · © forcsguy.com

  • Bank owns the balance and account rules. ATM owns hardware, cash on hand and the session.
  • Cash dispenser holds counts per denomination and decides which notes to give (greedy from the largest, with a fallback).
  • Every step is logged so a failure between "dispensed" and "confirmed" can be reconciled later (a mismatch report, or an automatic reversal).

  Design notes

  • Treat PINs as secrets: never store or log them; the bank verifies, the ATM only forwards an encrypted PIN block.
  • Idempotency: a retried authorise call must not debit twice; use a transaction id.
  • Timeouts: an idle session returns to Idle and ejects the card.
  • Extensions: deposits (count and hold until verified), fund transfers, receipts, and multiple card types.

  Real-world reading

  Related reading

  • Apple Machine Learning BlogApple engineers · AppleBlogs
  • AWS Machine Learning BlogAmazon Web Services engineers · Amazon Web ServicesBlogs
  • Optimizing Payments with Machine Learning at DropboxDropbox engineers · DropboxCase Studies
  • Machine Learning Platform at YelpYelp engineers · YelpCase Studies

  Show 6 moreShow fewer

  • PyKrylov: Accelerating Machine Learning Research at eBayeBay engineers · eBayCase Studies
  • Machine Learning (2 parts) at Condé NastCondé Nast engineers · Condé NastCase Studies
  • Machine Learning Platform at ZalandoZalando engineers · ZalandoCase Studies
  • Secret Detector: Preventing Secrets in Source Code at YelpYelp engineers · YelpCase Studies
  • Machine Learning at Jane StreetJane Street engineers · Jane StreetCase Studies
  • Waymo BlogWaymo engineers · WaymoBlogs

  Open this lesson with its quiz and exercise

• Low-Level Design Practice · 11 minElevator systemRequests, directions and scheduling: a state machine per car and a dispatcher that decides which car answers which call.Read hereClose

  The problem

  A building has several elevators. People press a button on a floor (a hall call with direction) and a floor button inside the car (a car request). The system moves cars, opens doors and decides which car should answer each call.

  Entities

  100%

  Figure SDA-LLD-07-1 · Elevator classes · © forcsguy.com

  One elevator as a state machine

  100%

  Figure SDA-LLD-07-2 · Elevator car states · © forcsguy.com

  A simple and effective scheduling rule is the elevator (scan) algorithm: keep two sorted sets of stops (above and below); continue in the current direction, serving stops in order; reverse only when none are left in that direction.

  class Elevator {
    floor = 0; dir: "UP" | "DOWN" | "IDLE" = "IDLE";
    up = new SortedSet<number>(); down = new SortedSet<number>();   // pending stops
  
    addStop(f: number) { (f >= this.floor ? this.up : this.down).add(f); if (this.dir === "IDLE") this.pick(); }
    private pick() { this.dir = this.up.size ? "UP" : this.down.size ? "DOWN" : "IDLE"; }
    step() {                                   // called once per tick
      if (this.dir === "UP") { this.floor++; if (this.up.deleteIfPresent(this.floor)) this.openDoors(); if (!this.up.size) this.pick(); }
      if (this.dir === "DOWN") { this.floor--; if (this.down.deleteIfPresent(this.floor)) this.openDoors(); if (!this.down.size) this.pick(); }
    }
  }
  

  Many cars: the dispatcher

  100%

  Figure SDA-LLD-07-3 · Assigning a hall call · © forcsguy.com

  Make the dispatcher a strategy so you can swap "nearest car" for "least loaded" or a smarter estimate without touching Elevator.

  Design notes

  • Concurrency: calls arrive while cars move. Use one event loop per elevator, or a queue per car, so state changes happen one at a time.
  • Safety: doors must not close on a blocked sensor; weight limits stop a car from moving.
  • Peak modes: morning (send idle cars to the lobby) and evening patterns are policies.
  • Extensions: express cars, destination dispatch (people enter their destination at the hall), emergency and maintenance modes.

  Real-world reading

  Related reading

  • Apple Machine Learning BlogApple engineers · AppleBlogs
  • AWS Machine Learning BlogAmazon Web Services engineers · Amazon Web ServicesBlogs
  • Machine Learning for Ranking Answers End-to-End at QuoraQuora engineers · QuoraCase Studies
  • Machine Learning (2 parts) at Condé NastCondé Nast engineers · Condé NastCase Studies

  Show 6 moreShow fewer

  • Google SRE Book: Distributed Periodic Scheduling with CronGoogle SRE team · GoogleBlogs
  • Please stop calling databases CP or APMartin Kleppmann · martin.kleppmann.comBlogs
  • Large-Scale Machine Learning Pipeline for Job Recommendations at IndeedIndeed engineers · IndeedCase Studies
  • Machine Learning Platform at YelpYelp engineers · YelpCase Studies
  • Engineering at QuoraQuora engineers · QuoraCase Studies
  • Distributed SchedulingUmbcPapers

  Open this lesson with its quiz and exercise