Infrastructure to Run User Forecasts at Spotify

Ideas you will meet

Distributed Systems

Consensus, time, failure and the fundamentals of scale-out.

Machine Learning Systems

Recommendations, ranking, feature stores and serving models at scale.

What we already know about this

Our own lessons on these ideas. Read them here before you start, so it makes sense.

• Low-Level Design Practice · 10 minChat applicationUsers, conversations and messages: delivery states, group chats and notifying clients, modelled as classes rather than infrastructure.Read hereClose

  The problem

  Users send text messages one-to-one or in groups. The app shows who is online, message status (sent, delivered, read) and notifies offline users. This lesson is about the classes; the infrastructure view (WebSockets, storage at scale) is in the chat-system lesson of the classic designs course.

  Entities

  100%

  Figure SDA-LLD-14-1 · Chat classes · © forcsguy.com

  A conversation has a type (direct or group) and participants. A message belongs to one conversation and has one receipt per recipient.

  Sending a message

  100%

  Figure SDA-LLD-14-2 · Sending a message · © forcsguy.com

  type Status = "SENT" | "DELIVERED" | "READ";
  class Conversation {
    private subscribers = new Map<string, (m: Message) => void>();
    constructor(readonly id: string, readonly members: string[]) {}
  
    subscribe(userId: string, fn: (m: Message) => void) { this.subscribers.set(userId, fn); return () => this.subscribers.delete(userId); }
  
    send(from: string, text: string): Message {
      if (!this.members.includes(from)) throw new Error("not a member");
      const m = new Message(newId(), this.id, from, text);
      m.receipts = Object.fromEntries(this.members.filter(u => u !== from).map(u => [u, "SENT" as Status]));
      store.save(m);                                         // persist first
      for (const [u, fn] of this.subscribers) if (u !== from) fn(m);   // push to online users
      return m;                                              // offline users get it when they reconnect, or by push notification
    }
  }
  

  Offline users and ordering

  • Offline users receive messages on reconnect by asking for everything after the last message id they have, and may get a push notification.
  • Order within a conversation comes from a server-assigned, increasing message id.
  • Read receipts are updates to the per-recipient receipt, then aggregated for the sender (in a group, "read by 3 of 5").

  Design notes

  • Observer decouples the conversation from how clients are connected.
  • Group size matters: a small group can notify each member directly; very large channels need a different fan-out approach.
  • Presence (online, last seen) is a separate component with its own throttled updates.
  • Privacy: block lists, muted conversations, and "delete for everyone" are policies checked before delivery.
  • Extensions: media messages (store files elsewhere, send a reference), replies, reactions, end-to-end encryption.

  Real-world reading

  Related reading

  • RFC 6455: The WebSocket ProtocolIETFDocumentation
  • Scaling Real-time Infrastructure at Alibaba for Global Shopping Holiday - Xiaowei Jiang, Senior Director at AlibabaAlibaba engineers · AlibabaCase Studies
  • The Architecture Twitter Uses to Deal with 150M Active Users, 300K QPS, a 22 MB/S Firehose, and Send Tweets in Under 5 SecondsHigh Scalability engineers · High ScalabilityCase Studies
  • Scaling Mailbox - From 0 to One Million Users in 6 Weeks and 100 Million Messages Per Day - High Scalability -High Scalability engineers · High ScalabilityCase Studies

  Show 6 moreShow fewer

  • Scaling Live Videos to a Billion Users at Facebook - Sachin Kulkarni, Director of Engineering at FacebookFacebookYouTube
  • MDN: WebSockets APIMDN Contributors · MozillaDocumentation
  • The WhatsApp Architecture Facebook Bought for $19BHigh ScalabilityCase Studies
  • Infrastructure to Run User Forecasts at SpotifySpotify engineers · SpotifyCase Studies
  • Data Infrastructure at LinkedInLinkedIn engineers · LinkedInCase Studies
  • Real-Time Data Warehouse with MemSQL on Cisco UCSCisco engineers · CiscoCase Studies

  Open this lesson with its quiz and exercise

• Low-Level Design Practice · 10 minSocial feed (Instagram-style)Users, follows, posts and likes: a feed model with the Observer pattern for notifications and a strategy for ordering.Read hereClose

  The problem

  Users follow each other, publish photo posts, like and comment, and see a home feed of posts from the accounts they follow. This lesson models the classes and rules. Scaling the feed (fan-out strategies) is covered in the news-feed lesson of the classic designs course.

  Entities

  100%

  Figure SDA-LLD-15-1 · Social feed classes · © forcsguy.com

  Publishing and the feed

  100%

  Figure SDA-LLD-15-2 · Publishing a post · © forcsguy.com

  interface FeedStrategy { build(userId: string, limit: number): Post[] }
  
  const newestFirst: FeedStrategy = {
    build(userId, limit) {
      const followees = follows.acceptedFolloweesOf(userId);
      return posts.byAuthors(followees).filter(p => canView(userId, p))
        .sort((a, b) => b.createdAt - a.createdAt).slice(0, limit);
    },
  };
  
  function canView(viewer: string, p: Post): boolean {          // one place for visibility rules
    const author = users.get(p.authorId);
    return !author.isPrivate || author.id === viewer || follows.isAccepted(viewer, author.id);
  }
  

  Putting visibility in one function means the feed, profile and search all agree about who may see what.

  Likes and comments

  • A like is unique per (user, post); enforce it with a uniqueness rule, not just a check, so double taps do not create two likes.
  • Keep a like count on the post for fast display and update it with each like or unlike, accepting that the count may briefly lag.
  • Comments are ordered by time and can have a parent id for replies.

  Design notes

  • Pagination uses a cursor (the time or id of the last item) so new posts do not shift pages.
  • Deleting a post must remove it from feeds and counts; a soft delete flag filtered at read time is the usual first step.
  • Blocking and muting are visibility rules too.
  • Extensions: stories with expiry, hashtags, tagging, search, and a ranking strategy that mixes recency with interest.

  Real-world reading

  Related reading

  • Self-Ranking Search with Elasticsearch at WattpadWattpad engineers · WattpadCase Studies
  • Monitoring Powers High Availability for LinkedIn FeedUSENIX engineers · USENIXCase Studies
  • Search Federation Architecture at LinkedIn (2018)LinkedIn engineers · LinkedInCase Studies
  • Scaling the Hadoop Distributed File System at LinkedInLinkedIn engineers · LinkedInCase Studies

  Show 6 moreShow fewer

  • Galene: Search Architecture of LinkedInLinkedIn engineers · LinkedInCase Studies
  • Infrastructure to Run User Forecasts at SpotifySpotify engineers · SpotifyCase Studies
  • Scalable User Privacy at SpotifySpotify engineers · SpotifyCase Studies
  • Moving Food Feed from Redis to Cassandra at ZomatoZomato engineers · ZomatoCase Studies
  • Clustering Similar Stories Using LDA at FlipboardFlipboard engineers · FlipboardCase Studies
  • Data Orchestration Service at SpotifySpotify engineers · SpotifyCase Studies

  Open this lesson with its quiz and exercise