Uber's Real-Time Push Platform

Ideas you will meet

WebSockets

Persistent bidirectional connections for real-time features.

Event Driven Architecture

Loosely coupled services reacting to events.

What we already know about this

Our own lessons on these ideas. Read them here before you start, so it makes sense.

• Core Concepts: How Things Actually Work · 7 minPolling, long polling, SSE and WebSocketsFour ways to get live updates to a browser, and how to choose between them.Read hereClose

  The problem

  HTTP is request and response: the client asks. How does the server tell a browser something now?

  The four options

  Technique	How	Good	Weak
  Short polling	Client asks every N seconds	Trivial, works everywhere	Wasteful, delayed by N
  Long polling	Client asks; the server holds the request until there is news (or a timeout), then the client asks again	Near real time, plain HTTP	A held request per client, awkward at scale
  Server-sent events (SSE)	One long HTTP response streams events	Simple, auto-reconnect, text over HTTP	One-way only
  WebSockets	HTTP upgrades to a two-way connection	Low latency both ways	Stateful connections, more infrastructure

  100%

  Figure SDA-CORE-08-1 · The four options · © forcsguy.com

  Before: polling: keep asking

  100%

  After: WebSocket: server pushes

  100%

  What changed

  • No wasted requests when nothing has changed
  • Updates arrive the moment they happen
  • The server must hold many open connections

  Choosing

  • Updates are rare or delays are fine: polling.
  • The server pushes, the client mostly listens (feeds, scores, notifications, progress): SSE.
  • Both sides send often (chat, games, collaborative editing): WebSockets.

  Running persistent connections

  Connections are long-lived, so capacity is about how many connections a server can hold, not requests per second.

  • Put connection handling in servers that are separate from your stateless API.
  • Use heartbeats to detect dead connections, and reconnect with backoff and jitter.
  • Track which server holds which user so messages can be routed (a presence or session store), and use a pub/sub system between servers.
  • When a client reconnects, let it ask for what it missed (events after id X), because connections drop.
  • Authenticate on connect and again when tokens expire.

  More on this site

  Related reading

  • RFC 6455: The WebSocket ProtocolIETFDocumentation
  • MDN: WebSockets APIMDN Contributors · MozillaDocumentation
  • Uber's Real-Time Push PlatformUber Engineering · UberCase Studies
  • How Discord Scaled Elixir to 5,000,000 Concurrent UsersDiscordCase Studies

  Show 6 moreShow fewer

  • The WhatsApp Architecture Facebook Bought for $19BHigh ScalabilityCase Studies
  • Erlang at Facebook (slides)Eugene Letuchy · FacebookCase Studies
  • Slack EngineeringSlack engineers · SlackBlogs
  • Real-time MessagingSlack Engineering · SlackCase Studies
  • How Uber Scales Their Real-time Market Platform - High Scalability -High Scalability engineers · High ScalabilityCase Studies
  • Real-Time Predictions at DoorDashDoorDash engineers · DoorDashCase Studies

  Open this lesson with its quiz and exercise

• Classic System Designs · 10 minDesign a notification systemSend push, SMS and email notifications reliably at scale: queues per channel, user preferences, retries and avoiding spam.Read hereClose

  The problem

  Applications tell users things: "your order shipped", "someone replied", "your code is 482913". A notification system delivers such messages over several channels:

  • Mobile push through Apple (APNs) and Google (FCM) services.
  • SMS through a telecom provider.
  • Email through an email provider.
  • Sometimes in-app messages or webhooks.

  Requirements: soft real time (seconds, not hours), high volume with spikes, no lost notifications, no duplicate floods, user control over what they receive.

  High-level design

  100%

  Figure SDA-CSD-06-1 · Notification pipeline with a queue per channel · © forcsguy.com

  1. Other services call the notification service (an API or an event) with who and what.
  2. It validates, applies preferences, renders the message from a template and picks the channel.
  3. It places a job on a per-channel queue.
  4. Workers call the third-party provider and record the result.

  Putting a queue between each stage absorbs spikes and lets you scale the channels independently.

  100%

  Figure SDA-CSD-06-2 · Should this event become a notification? · © forcsguy.com

  Image: Océanos y dados · CC0 · Source · Wikimedia Commons: Smartphone Use

  Channels and their providers

  100%

  Figure SDA-CSD-06-3 · One provider per channel · © forcsguy.com

  Collecting contact details

  100%

  Figure SDA-CSD-06-4 · Contact information flow and tables · © forcsguy.com

  Before: one shared queue

  100%

  After: separate lanes

  100%

  What changed

  • A marketing blast no longer delays login codes
  • Each channel scales and fails on its own
  • Priorities can be set per lane

  Supporting data

  Data	Purpose
  Contact info and device tokens	where to send
  User preferences	channels allowed, topics, quiet hours, language
  Templates	consistent text, translations
  Notification log	state per notification (queued, sent, failed), for debugging and dedupe

  Reliability

  • Persist first: write the notification to the log before queueing, so a crash does not lose it.
  • At-least-once delivery from the queue, plus a dedupe check (notification_id already sent?) so retries do not double send.
  • Retries with exponential backoff and jitter; after repeated failures move to a dead-letter queue and alert.
  • Provider failures: use a second provider as a fallback for critical channels such as SMS verification codes.
  • Invalid tokens: providers tell you when a device token is dead; remove it.

  Protecting users and the system

  • Preferences and opt-out are mandatory for marketing messages (and legally required in many places).
  • Rate limit per user and per topic, and collapse bursts into a digest ("12 new comments").
  • Priority: security codes and payment alerts jump the queue ahead of marketing.
  • Scheduling: delay non-urgent notifications until a sensible local time.

  Why the design looks this way

  A first version with one notification server has three weaknesses: it is a single point of failure, everything scales together (database, cache and sending), and slow third-party calls drag down the whole thing at peak. The fixes are the ones in the design: move state out of the server, run several servers, and put a queue per channel between the API and the workers so an outage at one provider (say the SMS vendor) only backs up its own queue.

  Other points worth knowing: providers differ by region (a push service available in one market may not be in another), so keep each provider behind a plug-in interface. Only internal services or verified clients may call the send API, using an app key and secret, or the system becomes a spam relay. Notifications may be delayed or reordered but must never be silently lost, so persist each one before sending and retry from the stored record. And the single most useful health signal is the number of queued notifications: if it keeps growing, add workers.

  Life of a notification

  100%

  Figure SDA-CSD-06-5 · Notification states for tracking · © forcsguy.com

  100%

  Figure SDA-CSD-06-6 · The complete notification path · © forcsguy.com

  Observability

  Track sent, delivered, opened and failed counts per channel and per provider, plus queue depth and age. A growing queue age is the earliest sign something is wrong. Keep a trail so support can answer "did this user get the message?".

  Worked numbers: a steady stream and a blast

  Take 10 million users who each get about 5 notifications a day across push, email and in-app: 50 million a day, about 580 per second on average. That is easy. The hard case is a campaign blast: a push to all 10 million users within 10 minutes means 16,700 sends per second, and the push providers rate-limit each connection, so you need many parallel workers and connections, and you must prevent the blast from delaying a login code.

  Log volume: at about 200 bytes per delivery record, 50 million a day is 10 GB a day. Keeping 30 days of detail is about 300 GB, which is manageable; keep aggregates much longer.

  Provider failure: a push provider that fails for 10 minutes during a blast would otherwise cause 10 million retries at once. The queue absorbs it, workers back off, and delivery continues when the provider recovers.

  100%

  Figure SDA-CSD-06-7 · Separate lanes for urgent and bulk messages · © forcsguy.com

  One notification, start to finish

  1. Event: the order service publishes "order 4711 shipped" with the user id.
  2. Decide: the notification service loads the user's preferences (allowed channels, quiet hours, language), checks the per-user limit (not more than a few promotional messages a day) and the dedupe key (order-4711-shipped), and renders the template.
  3. Enqueue: a message is placed on the right queue with an idempotency key.
  4. Send: a worker calls the provider. On success it records "sent". On a temporary error it retries with backoff. On a permanent error (for example, an invalid device token) it removes the token and records the failure.
  5. Track: delivery and open receipts update the record; the data feeds dashboards and "did not receive" support tools.

  100%

  Figure SDA-CSD-06-8 · Deduplicate before sending · © forcsguy.com

  Mistakes people make

  • Sharing one queue between login codes and marketing. A blast delays the code people are waiting for. Use separate lanes.
  • No per-user rate limit. A bug that emits an event in a loop can send a user a hundred messages.
  • Ignoring preferences and legal rules. Unsubscribe, quiet hours and consent are requirements, not nice-to-haves.
  • Keeping dead device tokens. Providers tell you when a token is invalid. Remove it, or your sender reputation and efficiency suffer.
  • Retry storms. Back off with jitter and cap retries, or you will hammer a provider that is already failing.
  • Time zones. "9 am local time" needs the user's zone, and the send time must be scheduled per zone.

  Try it yourself

  Use the System Builder to see a burst absorbed by a queue, and read the rule engine lesson for how triggers, cooldowns and idempotent actions fit the "decide" step.

  More on this site

  Related reading

  • Avoiding insurmountable queue backlogsAmazon Builders' Library · Amazon Web ServicesBlogs
  • Timeouts, retries, and backoff with jitterAmazon Builders' Library · Amazon Web ServicesBlogs
  • Event SourcingMartin FowlerBlogs
  • Uber's Real-Time Push PlatformUber Engineering · UberCase Studies

  Show 6 moreShow fewer

  • NATS ServerNATS · CNCF / NATSGitHub
  • CQRSMartin FowlerBlogs
  • CQRS patternMicrosoft · Microsoft AzureDocumentation
  • Scaling Push Messaging for Millions of Devices at NetflixNetflix engineers · NetflixCase Studies
  • Reference Architectures for Big Data SystemsCmu engineers · CmuCase Studies
  • Messaging System Model at DropboxDropbox engineers · DropboxCase Studies

  Open this lesson with its quiz and exercise

• Classic System Designs · 13 minDesign a chat systemReal-time one-to-one and group messaging: persistent connections, message routing, ordering, storage and presence.Read hereClose

  The problem

  A chat system lets people exchange messages in real time, one-to-one and in groups, on several devices, with history and online status.

  Requirements

  • Low latency delivery (under a second when both are online).
  • 1-to-1 and group chat (say up to a few hundred members).
  • Online/offline presence.
  • Multiple devices per user; message history; push notifications.
  • No lost messages.

  Why not plain HTTP

  With HTTP, the client asks and the server answers. To receive a message the client would have to poll repeatedly: wasteful and slow. Chat uses a persistent connection, usually WebSocket, so the server can push a message the moment it arrives.

  Polling, long polling, WebSocket

  • Polling: the client asks every few seconds. Simple, but most answers are "nothing new" and each costs a request.
  • Long polling: the server holds the request open until a message arrives or a timeout passes, then the client asks again. Better, but the server that holds the request may not be the one that receives the message (a stateless load balancer spreads requests), the server cannot easily tell the client has gone, and a quiet user still reconnects on every timeout.
  • WebSocket: starts as an HTTP request, is upgraded, and stays open in both directions. It usually passes firewalls because it uses the web ports. It can carry sending as well as receiving, so one mechanism does both; ordinary actions (sign-up, profile) can stay on plain HTTP.

  Connections are the scarce resource, not memory: at roughly 10 KB per connection, a million concurrent users need only about 10 GB, so the real limits are the number of connections per machine and, above all, not having a single machine as a single point of failure.

  Image: The Salt Lake Herald · Public domain · Source · Wikimedia Commons: Telephone switchboard 6 Oct 1907 Salt Lake City

  100%

  Figure SDA-CSD-08-1 · Polling versus long polling · © forcsguy.com

  100%

  Figure SDA-CSD-08-2 · WebSocket upgrade · © forcsguy.com

  Before: plain HTTP polling

  100%

  After: persistent WebSocket

  100%

  What changed

  • Messages arrive instantly instead of on the next poll
  • Far fewer wasted requests
  • Servers must keep many open connections

  High-level design

  Separate two kinds of traffic:

  • Stateless request/response work (sign-in, profile, history lookup): normal API servers behind a load balancer.
  • Stateful real-time work: chat servers holding the WebSocket connections.

  100%

  Figure SDA-CSD-08-3 · Chat servers, message store and push · © forcsguy.com

  Message flow (1-to-1)

  1. A sends the message over its WebSocket to its chat server.
  2. The server assigns a message id (increasing within the conversation) and stores it.
  3. It looks up which chat server B is connected to (a session store) and forwards the message.
  4. If B is online, it is delivered over B's WebSocket. If offline, a push notification is sent and the message waits in storage.
  5. B's client acknowledges receipt; receipts flow back to A.

  100%

  Figure SDA-CSD-08-4 · Stateless and stateful parts of a chat system · © forcsguy.com

  100%

  Figure SDA-CSD-08-5 · Connecting to the right chat server · © forcsguy.com

  100%

  Figure SDA-CSD-08-6 · One-to-one message path · © forcsguy.com

  Groups

  For small groups, deliver the message to each member's inbox (or sync queue). It is the same flow repeated per member. For very large groups, switch to a pull model where members fetch new messages on open, as in the news feed lesson.

  100%

  Figure SDA-CSD-08-7 · Small group chat as inboxes · © forcsguy.com

  Storing messages

  Chat produces enormous write volume and is mostly read as "the latest messages of a conversation". That suits a key-value or wide-column store partitioned by conversation, with messages sorted by id. Relational databases struggle at this scale, though they are fine for user and group data.

  Keep recent messages hot and move old ones to cheaper storage.

  Two details on storage keys. For one-to-one chat the message id alone orders the messages (never the creation time, since two can share a timestamp). For group chat use a composite key of channel id plus message id, with the channel as the partition key because every query is within one channel. Ids only need to be ordered within a conversation, so a per-conversation counter is simpler than a global generator. Key-value stores suit this because the data has a long tail of old messages that large relational indexes handle poorly, and because well-known chat products run on wide-column stores.

  Ordering and sync

  • Device clocks cannot be trusted, so the server assigns ids.
  • Each client remembers the last id it saw per conversation and on reconnect asks for everything after it. The same mechanism syncs multiple devices.

  Presence

  Show online/offline status with heartbeats: while connected, the client sends a small ping every few seconds. If heartbeats stop for a while, mark the user offline. Do not announce every blip: wait a grace period so a flaky connection does not make a user flicker. For large friend lists, fetch presence on demand or only for visible contacts to avoid a flood of updates.

  100%

  Figure SDA-CSD-08-8 · Syncing several devices · © forcsguy.com

  100%

  Figure SDA-CSD-08-9 · Presence with heartbeats · © forcsguy.com

  100%

  Figure SDA-CSD-08-10 · Publishing status changes to friends · © forcsguy.com

  Scaling and failure

  • Connection capacity: a chat server holds many thousands of connections, so scale by adding servers; keep per-connection memory small.
  • Server failure: clients reconnect (with backoff and jitter) to another server and sync missed messages.
  • Service discovery: choose a chat server for a new connection with a load-aware or consistent-hash assignment.
  • Security: authenticate on connect, use TLS, and consider end-to-end encryption, which means the server cannot read message content and affects features like search.

  Worked numbers: connections and message rate

  Assume 50 million daily users, 10 million online at peak, each sending 40 messages a day.

  • Connections: 10 million open WebSockets. One well-tuned server can hold on the order of 100,000 mostly idle connections, limited by memory per connection (tens of KB) and file descriptors. So you need about 100 chat servers, plus spare, say 130.
  • Message rate: 50M x 40 = 2 billion messages a day, about 23,000 per second on average, 70,000 at peak.
  • Storage: at 100 bytes per message (text plus metadata) that is 200 GB a day, 73 TB a year before replication and media. That is a write-heavy, append-mostly workload suited to a store that handles large sequential writes and range reads by conversation, such as a wide-column database, rather than a single relational node.
  • Heartbeats: if every connected client sends a heartbeat every 30 seconds, that is 10M / 30 = 333,000 tiny messages per second just to know who is online. Presence is often more expensive than chat itself, which is why designs batch and sample it.

  100%

  Figure SDA-CSD-08-11 · Sizing a chat system · © forcsguy.com

  What happens when a chat server dies

  A server holding 100,000 connections crashes. What do those users experience?

  1. Their clients notice the connection dropped (immediately if the socket closes, after a heartbeat timeout otherwise).
  2. Each client reconnects with jittered, exponential backoff; without jitter, 100,000 clients reconnect in the same second and overload the next server (a reconnect storm).
  3. The load balancer routes them to healthy servers, which register the connections in the session registry.
  4. Each client sends the id of the last message it received; the server replays anything newer from the message store. No message is lost, because messages were saved before being delivered.
  5. Presence flips offline then online; debounce this so friends do not see a flicker.

  This works only because the connection servers hold no source of truth: messages live in the store, and the connection is just a pipe.

  100%

  Figure SDA-CSD-08-12 · Recovering after a chat server fails · © forcsguy.com

  Delivery guarantees you can state

  Chat promises are about at least once delivery with idempotent handling: the sender assigns each message a unique id; the server stores it before acknowledging; the recipient acknowledges receipt; if no acknowledgement arrives the message is redelivered, and the client discards duplicates by id. Ordering is guaranteed within a conversation by a sequence number per conversation (not by wall-clock time, which differs between devices). Say clearly that you do not promise global ordering across conversations, because that would not scale.

  Mistakes people make

  • Using the client's clock for ordering. Clocks differ; use a server-assigned sequence per conversation.
  • Acknowledging before saving. If the server crashes after the ack but before the write, the message is lost. Save first.
  • Large groups as many small inboxes. Writing a message into the inbox of every member of a 100,000 person group is a fan-out explosion; use a shared timeline that members read instead.
  • Stateful connection servers. Keep nothing important in a connection server's memory; make reconnection cheap.
  • Treating presence as exact. "Online" is a best-effort guess; design the UI around that.

  Try it yourself

  The System Builder challenge "A burst of writes" mirrors saving messages: a queue protects the store. For real-time connection basics, read the WebRTC lesson and compare when you would use WebSockets and when peer-to-peer. Or open a chat-shaped design on the free canvas and kill a part to see what survives.

  More on this site

  Related reading

  • Flannel: An Application-Level Edge Cache to Make Slack ScaleSlack Engineering · SlackCase Studies
  • How Discord Scaled Elixir to 5,000,000 Concurrent UsersDiscordCase Studies
  • RFC 6455: The WebSocket ProtocolIETFDocumentation
  • Uber's Real-Time Push PlatformUber Engineering · UberCase Studies

  Show 6 moreShow fewer

  • How Discord Stores Billions of MessagesDiscordCase Studies
  • Erlang at Facebook (slides)Eugene Letuchy · FacebookCase Studies
  • Slack EngineeringSlack engineers · SlackBlogs
  • Real-time MessagingSlack Engineering · SlackCase Studies
  • Storing Messages with Cassandra at DiscordDiscord engineers · DiscordCase Studies
  • Real-time Distributed Tracing at LinkedInLinkedIn engineers · LinkedInCase Studies

  Open this lesson with its quiz and exercise