How Uber Scales Their Real-time Market Platform - High Scalability -

Ideas you will meet

WebSockets

Persistent bidirectional connections for real-time features.

What we already know about this

Our own lessons on these ideas. Read them here before you start, so it makes sense.

• Core Concepts: How Things Actually Work · 7 minPolling, long polling, SSE and WebSocketsFour ways to get live updates to a browser, and how to choose between them.Read hereClose

  The problem

  HTTP is request and response: the client asks. How does the server tell a browser something now?

  The four options

  Technique	How	Good	Weak
  Short polling	Client asks every N seconds	Trivial, works everywhere	Wasteful, delayed by N
  Long polling	Client asks; the server holds the request until there is news (or a timeout), then the client asks again	Near real time, plain HTTP	A held request per client, awkward at scale
  Server-sent events (SSE)	One long HTTP response streams events	Simple, auto-reconnect, text over HTTP	One-way only
  WebSockets	HTTP upgrades to a two-way connection	Low latency both ways	Stateful connections, more infrastructure

  100%

  Figure SDA-CORE-08-1 · The four options · © forcsguy.com

  Before: polling: keep asking

  100%

  After: WebSocket: server pushes

  100%

  What changed

  • No wasted requests when nothing has changed
  • Updates arrive the moment they happen
  • The server must hold many open connections

  Choosing

  • Updates are rare or delays are fine: polling.
  • The server pushes, the client mostly listens (feeds, scores, notifications, progress): SSE.
  • Both sides send often (chat, games, collaborative editing): WebSockets.

  Running persistent connections

  Connections are long-lived, so capacity is about how many connections a server can hold, not requests per second.

  • Put connection handling in servers that are separate from your stateless API.
  • Use heartbeats to detect dead connections, and reconnect with backoff and jitter.
  • Track which server holds which user so messages can be routed (a presence or session store), and use a pub/sub system between servers.
  • When a client reconnects, let it ask for what it missed (events after id X), because connections drop.
  • Authenticate on connect and again when tokens expire.

  More on this site

  Related reading

  • RFC 6455: The WebSocket ProtocolIETFDocumentation
  • MDN: WebSockets APIMDN Contributors · MozillaDocumentation
  • Uber's Real-Time Push PlatformUber Engineering · UberCase Studies
  • How Discord Scaled Elixir to 5,000,000 Concurrent UsersDiscordCase Studies

  Show 6 moreShow fewer

  • The WhatsApp Architecture Facebook Bought for $19BHigh ScalabilityCase Studies
  • Erlang at Facebook (slides)Eugene Letuchy · FacebookCase Studies
  • Slack EngineeringSlack engineers · SlackBlogs
  • Real-time MessagingSlack Engineering · SlackCase Studies
  • How Uber Scales Their Real-time Market Platform - High Scalability -High Scalability engineers · High ScalabilityCase Studies
  • Real-Time Predictions at DoorDashDoorDash engineers · DoorDashCase Studies

  Open this lesson with its quiz and exercise