AWS Machine Learning Blog

Ideas you will meet

Machine Learning Systems

Recommendations, ranking, feature stores and serving models at scale.

What we already know about this

Our own lessons on these ideas. Read them here before you start, so it makes sense.

• Low-Level Design Practice · 10 minHow to approach a low-level design problemA repeatable six-step method: clarify, find the nouns, give each a responsibility, relate them, pick patterns and then code the core.Read hereClose

  Low-level design is about structure

  A high-level design decides servers, databases and queues. A low-level design (LLD) decides classes, methods and relationships inside one program. The questions are: what objects exist, who owns which rule, and how will this change next month?

  Interviewers use LLD problems (a parking lot, an elevator, a chess game) because a small problem shows how you think: do you clarify, do you model sensibly, do you keep rules in one place, do you notice concurrency?

  The six steps

  100%

  Figure SDA-LLD-01-1 · The six-step method for low-level design · © forcsguy.com

  1. Clarify requirements. Who uses it? What are the three to five main flows? What is the scale? What is deliberately out of scope?
  2. Find the nouns and verbs. Nouns become candidate classes; verbs become methods. Drop nouns that are only values (a price can be a number or a small value type).
  3. Assign responsibilities. Each class owns its data and the rules about that data. If a rule needs data from two classes, ask which one should own it or whether a third coordinating class is justified.
  4. Draw relationships. Has-a (composition), is-a (inheritance, used sparingly), uses (dependency). Prefer composition and interfaces to deep inheritance.
  5. Apply patterns where they earn their keep. A lifecycle with many states suggests the State pattern; swappable rules (pricing, matching) suggest Strategy; things that react to events suggest Observer; complex creation suggests Factory.
  6. Code the core flow, then stress it. Write the happy path, then ask: what if two users act at once? What if the input is invalid? What if a step fails halfway?

  Habits that earn trust

  • Names carry meaning. ParkingSpot, Ticket, calculateFee beat Manager and process.
  • Single responsibility. A class that "does everything" is the usual red flag.
  • Depend on interfaces at the points that will change (payment method, pricing rule), not everywhere.
  • Make illegal states hard to represent. Use enums or state objects instead of loose strings and booleans.
  • Say your trade-offs aloud. "I used a simple map here; if we needed millions of spots I would index by level."

  A tiny example of the method

  Requirement: "a vending machine that sells snacks for coins". Nouns: machine, product, slot, coin. Verbs: select, insert, dispense, refund. Responsibilities: Inventory knows what is in each slot; VendingMachine coordinates the flow; a state (idle, has money, dispensing) decides which actions are allowed. Only the state and the stock rules need real design; the rest is plain code.

  The following lessons apply this method to well-known problems. Each ends with the extensions an interviewer is likely to ask for.

  Real-world reading

  Related reading

  • Apple Machine Learning BlogApple engineers · AppleBlogs
  • AWS Machine Learning BlogAmazon Web Services engineers · Amazon Web ServicesBlogs
  • Machine Learning for Indexing Text from Billions of Images at DropboxDropbox engineers · DropboxCase Studies
  • Optimizing Payments with Machine Learning at DropboxDropbox engineers · DropboxCase Studies

  Show 6 moreShow fewer

  • 8 Things You Need to Know Before a System Design InterviewGainlo engineers · GainloBlogs
  • CAP Twelve Years Later: How the Rules Have ChangedEric Brewer · InfoQBlogs
  • Deep Learning for Frame Detection in Product Images at AllegroAllegro engineers · AllegroCase Studies
  • Explaining Low-Level Systems (OS, Network/Protocol, Database, Storage)Wustl engineers · WustlBlogs
  • System Design Interview (checkcheckzz)checkcheckzz · GitHubGitHub
  • Top 10 System Design Interview QuestionsHackernoon engineers · HackernoonBlogs

  Open this lesson with its quiz and exercise

• Low-Level Design Practice · 10 minATM machineA machine that behaves differently in each state: the State pattern, card and PIN checks, and cash dispensing with limits.Read hereClose

  The problem

  An ATM accepts a card, checks the PIN, then lets the customer check balance, withdraw cash or deposit, and returns the card. It talks to a bank to authorise operations and holds a limited amount of cash.

  States

  100%

  Figure SDA-LLD-06-1 · ATM states · © forcsguy.com

  In Idle, "enter PIN" makes no sense; in Authenticated, "insert card" makes no sense. The State pattern puts those rules inside state objects instead of if (state === ...) chains.

  interface AtmState {
    insertCard(atm: Atm, card: Card): void;
    enterPin(atm: Atm, pin: string): void;
    withdraw(atm: Atm, cents: number): void;
  }
  
  class Idle implements AtmState {
    insertCard(atm, card) { atm.card = card; atm.setState(new HasCard()); }
    enterPin() { throw new Error("insert a card first"); }
    withdraw() { throw new Error("insert a card first"); }
  }
  class HasCard implements AtmState {
    tries = 0;
    insertCard() { throw new Error("card already inserted"); }
    enterPin(atm, pin) {
      if (atm.bank.checkPin(atm.card!, pin)) atm.setState(new Authenticated());
      else if (++this.tries >= 3) { atm.retainOrEject(); atm.setState(new Idle()); }
    }
    withdraw() { throw new Error("enter the PIN first"); }
  }
  

  Who is responsible for what

  100%

  Figure SDA-LLD-06-2 · Withdrawal sequence · © forcsguy.com

  • Bank owns the balance and account rules. ATM owns hardware, cash on hand and the session.
  • Cash dispenser holds counts per denomination and decides which notes to give (greedy from the largest, with a fallback).
  • Every step is logged so a failure between "dispensed" and "confirmed" can be reconciled later (a mismatch report, or an automatic reversal).

  Design notes

  • Treat PINs as secrets: never store or log them; the bank verifies, the ATM only forwards an encrypted PIN block.
  • Idempotency: a retried authorise call must not debit twice; use a transaction id.
  • Timeouts: an idle session returns to Idle and ejects the card.
  • Extensions: deposits (count and hold until verified), fund transfers, receipts, and multiple card types.

  Real-world reading

  Related reading

  • Apple Machine Learning BlogApple engineers · AppleBlogs
  • AWS Machine Learning BlogAmazon Web Services engineers · Amazon Web ServicesBlogs
  • Optimizing Payments with Machine Learning at DropboxDropbox engineers · DropboxCase Studies
  • Machine Learning Platform at YelpYelp engineers · YelpCase Studies

  Show 6 moreShow fewer

  • PyKrylov: Accelerating Machine Learning Research at eBayeBay engineers · eBayCase Studies
  • Machine Learning (2 parts) at Condé NastCondé Nast engineers · Condé NastCase Studies
  • Machine Learning Platform at ZalandoZalando engineers · ZalandoCase Studies
  • Secret Detector: Preventing Secrets in Source Code at YelpYelp engineers · YelpCase Studies
  • Machine Learning at Jane StreetJane Street engineers · Jane StreetCase Studies
  • Waymo BlogWaymo engineers · WaymoBlogs

  Open this lesson with its quiz and exercise

• Low-Level Design Practice · 11 minElevator systemRequests, directions and scheduling: a state machine per car and a dispatcher that decides which car answers which call.Read hereClose

  The problem

  A building has several elevators. People press a button on a floor (a hall call with direction) and a floor button inside the car (a car request). The system moves cars, opens doors and decides which car should answer each call.

  Entities

  100%

  Figure SDA-LLD-07-1 · Elevator classes · © forcsguy.com

  One elevator as a state machine

  100%

  Figure SDA-LLD-07-2 · Elevator car states · © forcsguy.com

  A simple and effective scheduling rule is the elevator (scan) algorithm: keep two sorted sets of stops (above and below); continue in the current direction, serving stops in order; reverse only when none are left in that direction.

  class Elevator {
    floor = 0; dir: "UP" | "DOWN" | "IDLE" = "IDLE";
    up = new SortedSet<number>(); down = new SortedSet<number>();   // pending stops
  
    addStop(f: number) { (f >= this.floor ? this.up : this.down).add(f); if (this.dir === "IDLE") this.pick(); }
    private pick() { this.dir = this.up.size ? "UP" : this.down.size ? "DOWN" : "IDLE"; }
    step() {                                   // called once per tick
      if (this.dir === "UP") { this.floor++; if (this.up.deleteIfPresent(this.floor)) this.openDoors(); if (!this.up.size) this.pick(); }
      if (this.dir === "DOWN") { this.floor--; if (this.down.deleteIfPresent(this.floor)) this.openDoors(); if (!this.down.size) this.pick(); }
    }
  }
  

  Many cars: the dispatcher

  100%

  Figure SDA-LLD-07-3 · Assigning a hall call · © forcsguy.com

  Make the dispatcher a strategy so you can swap "nearest car" for "least loaded" or a smarter estimate without touching Elevator.

  Design notes

  • Concurrency: calls arrive while cars move. Use one event loop per elevator, or a queue per car, so state changes happen one at a time.
  • Safety: doors must not close on a blocked sensor; weight limits stop a car from moving.
  • Peak modes: morning (send idle cars to the lobby) and evening patterns are policies.
  • Extensions: express cars, destination dispatch (people enter their destination at the hall), emergency and maintenance modes.

  Real-world reading

  Related reading

  • Apple Machine Learning BlogApple engineers · AppleBlogs
  • AWS Machine Learning BlogAmazon Web Services engineers · Amazon Web ServicesBlogs
  • Machine Learning for Ranking Answers End-to-End at QuoraQuora engineers · QuoraCase Studies
  • Machine Learning (2 parts) at Condé NastCondé Nast engineers · Condé NastCase Studies

  Show 6 moreShow fewer

  • Google SRE Book: Distributed Periodic Scheduling with CronGoogle SRE team · GoogleBlogs
  • Please stop calling databases CP or APMartin Kleppmann · martin.kleppmann.comBlogs
  • Large-Scale Machine Learning Pipeline for Job Recommendations at IndeedIndeed engineers · IndeedCase Studies
  • Machine Learning Platform at YelpYelp engineers · YelpCase Studies
  • Engineering at QuoraQuora engineers · QuoraCase Studies
  • Distributed SchedulingUmbcPapers

  Open this lesson with its quiz and exercise