Distributed counters, leaderboards and live reactions

The common difficulty

All three problems involve huge numbers of small writes to shared state that many people read at once. The pattern behind the solutions is the same: do not contend on one place. Spread the writes, combine the results, and accept that some answers can be approximate or slightly late.

1. Distributed counters

Counting likes, views or API calls looks trivial until millions of updates per second hit one row.

100%

Figure SDA-ADS-10-1 · A sharded counter · © forcsguy.com

Options, from simple to scalable:

• One atomic increment in a fast store (Redis INCR): thousands per second per key, fine for many uses.
• Sharded counter: split one logical counter into N sub-counters; write to a random one; sum on read.
• Local aggregation then merge: each server counts in memory and sends deltas periodically; a collector merges (the approach of metrics systems).
• Event log then aggregate: write events to a log, count with a stream job; exact, replayable, slower to show.
• Mergeable counters (CRDTs): counters that can be updated on several replicas and merged without conflict (see collaborative editing).
• Approximate counting: for unique counts (distinct viewers) use HyperLogLog, which counts millions of items in a few kilobytes with about one percent error (see the probabilistic structures lesson).

Decide the accuracy you need: a like count can lag and be approximate; a bill must be exact (use an append-only log and idempotent processing).

2. Leaderboards

A leaderboard needs: update a player's score, get their rank, show the top N, and show neighbours around a player.

100%

Figure SDA-ADS-10-2 · Leaderboard with a sorted set · © forcsguy.com

• Redis sorted sets (ZADD, ZRANGE, ZRANK) do this in logarithmic time and handle millions of members on one node.
• Persistence: keep scores in a database too (the sorted set is a fast view that can be rebuilt).
• Scale: split by season or time window (daily, weekly boards), by region or game mode, or by score range; compute a global top-N by merging each shard's top results.
• Approximate ranks for players far from the top are often enough ("you are in the top 12 percent"); exact rank is needed only near the top.
• Ties: break by earliest time or id so ordering is deterministic.
• Cheating and fairness: validate scores server-side.

3. Live comments and reactions

A live event with millions of viewers sends a flood of reactions that everyone expects to see "live".

100%

Figure SDA-ADS-10-3 · Reactions at massive scale · © forcsguy.com

For live comments (text), you cannot aggregate, so:

• Moderate and rank: filter spam, show a sampled or top subset to big audiences.
• Partition by chat room and cap rates per user.
• Fan-out via a pub/sub layer to connection gateways, which hold the WebSocket connections (see the chat lesson).
• Store comments in a log or wide-column store ordered by time for replay and history.

Key tricks across all of these: aggregate before you fan out, sample, throttle clients, batch writes, and accept approximate, eventually consistent displays while keeping money-related counts exact elsewhere.

Questions to ask in an interview

How exact must the number be? How fast must it be visible? What is the write rate per key (hot key)? What happens on failure (can we lose a second of counts)? Can we pre-aggregate at the edge? These questions steer you from "a table with a counter column" to a design that survives a viral moment.

Real-world reading