Rate limiting

Fixed window, sliding window, token bucket and leaky bucket against bursts.

On the roadmap:Intermediate · step 6: Protecting services

Starterabout 10 to 15 min6 experiments

Experiments

New here? A few short steps show what happens, one at a time.

Settings

Limiter

Counts requests in each whole second and starts again at zero on the boundary. Cheap, but blind to what happens across a boundary.

Limit (requests per second)10

What the traffic does

Let through

100%

Refused

Busiest second

20 (limit 10)

Average wait

0 ms

The busiest second let 20 requests through against a limit of 10. A fixed window resets at the boundary, so bursts either side of it both pass. Try a sliding window.

Fixed window over 20 seconds

Each bar is a tenth of a second. Grey is what arrived, blue is what was let through. The dashed line is the pace the limit allows, 1 requests per tenth of a second.

In plain words

A rate limiter decides how many requests a client may make in a stretch of time and turns the rest away. The simple versions have blind spots, and the way a limiter counts changes what it lets through when traffic arrives in a burst.

Why it matters

Limits protect a service from one noisy client, a bug that loops, or an attack. Here the same stream of requests meets five limiters, so you can see what each lets through in its busiest second and what it does to the requests it holds back. Times run over 20 seconds.

How real systems do it

The simulation is a simplified model. Here is what production systems add or do.

Stripe describes using token buckets for its request rate limiters, and returns HTTP 429 when a client goes over.
Amazon API Gateway throttles with a token bucket model: a steady rate and a burst allowance.
NGINX's limit_req module is described as using the leaky bucket method, with an optional burst of requests that are queued.
A rejected request should say how long to wait. HTTP has the 429 status and a Retry-After header for exactly this, and good clients back off when they see it.