CaffeOnSpark: Distributed Deep Learning on Big Data Clusters at Yahoo

Ideas you will meet

Distributed Systems

Consensus, time, failure and the fundamentals of scale-out.

Machine Learning Systems

Recommendations, ranking, feature stores and serving models at scale.

What we already know about this

Our own lessons on these ideas. Read them here before you start, so it makes sense.

• Low-Level Design Practice · 9 minSnake gameA small real-time model: a grid, a moving snake, food and collisions, with a clean game loop that is easy to test.Read hereClose

  The problem

  A snake moves on a grid. The player steers it. Eating food makes it longer and scores a point. Hitting a wall or itself ends the game.

  Entities

  100%

  Figure SDA-LLD-13-1 · Snake classes · © forcsguy.com

  One step of the game

  100%

  Figure SDA-LLD-13-2 · One game step · © forcsguy.com

  type P = { x: number; y: number };
  const delta = { UP: { x: 0, y: -1 }, DOWN: { x: 0, y: 1 }, LEFT: { x: -1, y: 0 }, RIGHT: { x: 1, y: 0 } };
  const opposite = { UP: "DOWN", DOWN: "UP", LEFT: "RIGHT", RIGHT: "LEFT" } as const;
  
  class Game {
    body: P[] = [{ x: 5, y: 5 }];      // index 0 is the head
    dir: keyof typeof delta = "RIGHT"; food: P = { x: 8, y: 5 }; score = 0; over = false;
    constructor(readonly w = 20, readonly h = 20) {}
  
    turn(d: keyof typeof delta) { if (d !== opposite[this.dir]) this.dir = d; }
  
    step() {
      if (this.over) return;
      const head = { x: this.body[0].x + delta[this.dir].x, y: this.body[0].y + delta[this.dir].y };
      const outside = head.x < 0 || head.y < 0 || head.x >= this.w || head.y >= this.h;
      const ate = head.x === this.food.x && head.y === this.food.y;
      const hitsSelf = this.body.slice(0, ate ? undefined : -1).some(c => c.x === head.x && c.y === head.y);
      if (outside || hitsSelf) { this.over = true; return; }
      this.body.unshift(head);
      if (ate) { this.score++; this.food = this.freeCell(); } else this.body.pop();
    }
    freeCell(): P { /* pick a random cell not in body */ return { x: 0, y: 0 }; }
  }
  

  The subtle detail: when not eating, the tail moves away this step, so the head may legally enter the cell the tail is leaving. That is why the self-collision check excludes the tail unless the snake grows.

  Design notes

  • Separate model from loop and view: step() is pure game logic. An outer timer calls it; a renderer draws the state. Tests call step() directly.
  • Input buffering: store the next direction and apply it at the next step, so quick double taps do not skip.
  • Free-cell search: for a nearly full board, keep a set of free cells rather than retrying random picks.
  • Extensions: levels and speed, walls, two players, power-ups, saved high scores.

  Real-world reading

  Related reading

  • Cross-Lingual End-to-End Product Search with Deep Learning at ZalandoZalando engineers · ZalandoCase Studies
  • Moving Core Business Search to Elasticsearch at YelpYelp engineers · YelpCase Studies
  • Self-Ranking Search with Elasticsearch at WattpadWattpad engineers · WattpadCase Studies
  • Autoscaling AWS Step Functions Activities at YelpYelp engineers · YelpCase Studies

  Show 6 moreShow fewer

  • Discovering Popular Dishes with Deep Learning at YelpYelp engineers · YelpCase Studies
  • Boosting Site Speed Using Brotli Compression at LinkedInLinkedIn engineers · LinkedInCase Studies
  • The Tail at Scale (paper)Dean and Barroso · GooglePapers
  • Nautilus: Search Engine at DropboxDropbox engineers · DropboxCase Studies
  • CaffeOnSpark: Distributed Deep Learning on Big Data Clusters at YahooYahoo engineers · YahooCase Studies
  • Kafka at YelpYelp engineers · YelpCase Studies

  Open this lesson with its quiz and exercise