Scaling Real-time Infrastructure at Alibaba for Global Shopping Holiday - Xiaowei Jiang, Senior Director at Alibaba

Ideas you will meet

WebSockets

Persistent bidirectional connections for real-time features.

What we already know about this

Our own lessons on these ideas. Read them here before you start, so it makes sense.

• Core Concepts: How Things Actually Work · 7 minPolling, long polling, SSE and WebSocketsFour ways to get live updates to a browser, and how to choose between them.Read hereClose

  The problem

  HTTP is request and response: the client asks. How does the server tell a browser something now?

  The four options

  Technique	How	Good	Weak
  Short polling	Client asks every N seconds	Trivial, works everywhere	Wasteful, delayed by N
  Long polling	Client asks; the server holds the request until there is news (or a timeout), then the client asks again	Near real time, plain HTTP	A held request per client, awkward at scale
  Server-sent events (SSE)	One long HTTP response streams events	Simple, auto-reconnect, text over HTTP	One-way only
  WebSockets	HTTP upgrades to a two-way connection	Low latency both ways	Stateful connections, more infrastructure

  100%

  Figure SDA-CORE-08-1 · The four options · © forcsguy.com

  Before: polling: keep asking

  100%

  After: WebSocket: server pushes

  100%

  What changed

  • No wasted requests when nothing has changed
  • Updates arrive the moment they happen
  • The server must hold many open connections

  Choosing

  • Updates are rare or delays are fine: polling.
  • The server pushes, the client mostly listens (feeds, scores, notifications, progress): SSE.
  • Both sides send often (chat, games, collaborative editing): WebSockets.

  Running persistent connections

  Connections are long-lived, so capacity is about how many connections a server can hold, not requests per second.

  • Put connection handling in servers that are separate from your stateless API.
  • Use heartbeats to detect dead connections, and reconnect with backoff and jitter.
  • Track which server holds which user so messages can be routed (a presence or session store), and use a pub/sub system between servers.
  • When a client reconnects, let it ask for what it missed (events after id X), because connections drop.
  • Authenticate on connect and again when tokens expire.

  More on this site

  Related reading

  • RFC 6455: The WebSocket ProtocolIETFDocumentation
  • MDN: WebSockets APIMDN Contributors · MozillaDocumentation
  • Uber's Real-Time Push PlatformUber Engineering · UberCase Studies
  • How Discord Scaled Elixir to 5,000,000 Concurrent UsersDiscordCase Studies

  Show 6 moreShow fewer

  • The WhatsApp Architecture Facebook Bought for $19BHigh ScalabilityCase Studies
  • Erlang at Facebook (slides)Eugene Letuchy · FacebookCase Studies
  • Slack EngineeringSlack engineers · SlackBlogs
  • Real-time MessagingSlack Engineering · SlackCase Studies
  • How Uber Scales Their Real-time Market Platform - High Scalability -High Scalability engineers · High ScalabilityCase Studies
  • Real-Time Predictions at DoorDashDoorDash engineers · DoorDashCase Studies

  Open this lesson with its quiz and exercise

• Low-Level Design Practice · 10 minChat applicationUsers, conversations and messages: delivery states, group chats and notifying clients, modelled as classes rather than infrastructure.Read hereClose

  The problem

  Users send text messages one-to-one or in groups. The app shows who is online, message status (sent, delivered, read) and notifies offline users. This lesson is about the classes; the infrastructure view (WebSockets, storage at scale) is in the chat-system lesson of the classic designs course.

  Entities

  100%

  Figure SDA-LLD-14-1 · Chat classes · © forcsguy.com

  A conversation has a type (direct or group) and participants. A message belongs to one conversation and has one receipt per recipient.

  Sending a message

  100%

  Figure SDA-LLD-14-2 · Sending a message · © forcsguy.com

  type Status = "SENT" | "DELIVERED" | "READ";
  class Conversation {
    private subscribers = new Map<string, (m: Message) => void>();
    constructor(readonly id: string, readonly members: string[]) {}
  
    subscribe(userId: string, fn: (m: Message) => void) { this.subscribers.set(userId, fn); return () => this.subscribers.delete(userId); }
  
    send(from: string, text: string): Message {
      if (!this.members.includes(from)) throw new Error("not a member");
      const m = new Message(newId(), this.id, from, text);
      m.receipts = Object.fromEntries(this.members.filter(u => u !== from).map(u => [u, "SENT" as Status]));
      store.save(m);                                         // persist first
      for (const [u, fn] of this.subscribers) if (u !== from) fn(m);   // push to online users
      return m;                                              // offline users get it when they reconnect, or by push notification
    }
  }
  

  Offline users and ordering

  • Offline users receive messages on reconnect by asking for everything after the last message id they have, and may get a push notification.
  • Order within a conversation comes from a server-assigned, increasing message id.
  • Read receipts are updates to the per-recipient receipt, then aggregated for the sender (in a group, "read by 3 of 5").

  Design notes

  • Observer decouples the conversation from how clients are connected.
  • Group size matters: a small group can notify each member directly; very large channels need a different fan-out approach.
  • Presence (online, last seen) is a separate component with its own throttled updates.
  • Privacy: block lists, muted conversations, and "delete for everyone" are policies checked before delivery.
  • Extensions: media messages (store files elsewhere, send a reference), replies, reactions, end-to-end encryption.

  Real-world reading

  Related reading

  • RFC 6455: The WebSocket ProtocolIETFDocumentation
  • Scaling Real-time Infrastructure at Alibaba for Global Shopping Holiday - Xiaowei Jiang, Senior Director at AlibabaAlibaba engineers · AlibabaCase Studies
  • The Architecture Twitter Uses to Deal with 150M Active Users, 300K QPS, a 22 MB/S Firehose, and Send Tweets in Under 5 SecondsHigh Scalability engineers · High ScalabilityCase Studies
  • Scaling Mailbox - From 0 to One Million Users in 6 Weeks and 100 Million Messages Per Day - High Scalability -High Scalability engineers · High ScalabilityCase Studies

  Show 6 moreShow fewer

  • Scaling Live Videos to a Billion Users at Facebook - Sachin Kulkarni, Director of Engineering at FacebookFacebookYouTube
  • MDN: WebSockets APIMDN Contributors · MozillaDocumentation
  • The WhatsApp Architecture Facebook Bought for $19BHigh ScalabilityCase Studies
  • Infrastructure to Run User Forecasts at SpotifySpotify engineers · SpotifyCase Studies
  • Data Infrastructure at LinkedInLinkedIn engineers · LinkedInCase Studies
  • Real-Time Data Warehouse with MemSQL on Cisco UCSCisco engineers · CiscoCase Studies

  Open this lesson with its quiz and exercise