Cache patterns and stampedes

Cache-aside, write-through and write-back, and a mass expiry hitting the database.

On the roadmap:Beginner · step 7: Caching basics

Starterabout 10 to 15 min7 experiments

Experiments

New here? A few short steps show what happens, one at a time.

Settings

What to look at

Pattern

The application looks in the cache, and on a miss reads the database and fills the cache itself. A write goes to the database and removes the cached copy.

Operations per second1,000

Share that are reads80%

Reads found in the cache80%

Database read time (ms)10

Database reads/s

160

Database writes/s

200

Average read

3.0 ms

A write takes

11 ms

Writes lost if cache dies

Reads average 3.0 ms, and the database takes 160 reads and 200 writes a second. A very small share of reads can be stale, from a race between a refill and a write.

The same traffic, every pattern

DB reads/s	DB writes/s	Read	Write	Lost on a crash
160	200	3.0 ms	11 ms	0
160	200	3.0 ms	11 ms	0
80	200	2.0 ms	11 ms	0
80	100	2.0 ms	1.0 ms	500

Cache-aside and read-through load the database the same way. Write-through and write-back fill the cache on writes, so more reads hit. Read time is cache time on a hit and cache plus database time on a miss.

In plain words

There are several ways for an application to use a cache. In cache-aside the application manages the cache itself. In read-through the cache loads data for it. Write-through updates cache and database together, and write-back saves to the database later. Each trades speed, freshness and the risk of losing data.

Why it matters

Adding a cache is easy, and the details decide whether it helps or hurts: who writes the loading code, how long a write takes, whether data can be lost, and what happens when popular keys expire at once. Here you can compare the four patterns on the same traffic, then trigger a stampede and try the usual protections.

How real systems do it

The simulation is a simplified model. Here is what production systems add or do.

Cache-aside, often called lazy loading, is the usual pattern with Redis and Memcached. Amazon DynamoDB Accelerator (DAX) is a write-through cache, so writes go through it to the table.
CPUs and operating systems use write-back: the Linux page cache holds writes in memory and flushes them later, which is why a power cut can lose recent writes that were not synced.
Facebook's memcache paper describes leases to stop a thundering herd of misses on one key, and to avoid a stale value being set after a delete.
Probabilistic early expiration, described by Vattani and colleagues, lets a request refresh a key shortly before it expires, with a chance that grows as expiry nears, so that popular keys rarely go missing.