Redis: persistence, replication, Sentinel and Cluster

Fast because it is in memory, durable because of logs

Redis keeps its dataset in RAM (the reason it is so fast; see the Redis and Memcached lesson). To survive a restart it offers two persistence mechanisms, which can be combined.

Snapshots (RDB)

Redis periodically writes a point-in-time snapshot of the whole dataset to disk, using a forked child process so the main process keeps serving. Snapshots are compact and quick to load, but anything written since the last snapshot is lost on a crash.

Append-only file (AOF)

Redis appends every write command to a log. On restart it replays the log. The fsync policy sets the loss window:

Policy	Behaviour	Risk
always	Flush after every command	Slowest, almost no loss
everysec	Flush once per second	At most about a second lost (common default)
no	Let the OS decide	Fastest, larger loss

The AOF grows, so Redis periodically rewrites it into a compact form. Many deployments use AOF with everysec, sometimes plus snapshots for fast backups.

100%

Figure SDA-ADS-04-1 · Redis persistence choices · © forcsguy.com

Replication and Sentinel

A Redis primary replicates to one or more replicas asynchronously. Replicas serve reads (possibly slightly stale) and can take over. Sentinel is a small set of monitor processes that:

1. watch the primary and replicas,
2. agree by quorum that the primary is down,
3. promote a replica and tell clients where the new primary is.

100%

Figure SDA-ADS-04-2 · Sentinel failover · © forcsguy.com

Because replication is asynchronous, a failover can lose the last few acknowledged writes. Redis offers settings to reduce this (for example requiring a minimum number of replicas to acknowledge), at the cost of availability.

Redis Cluster: sharding

When data or traffic outgrow one node, Redis Cluster shards the keyspace into 16,384 hash slots. A key's slot is a hash of the key modulo that number; each node owns a range of slots, and each primary has replicas. Clients learn the slot map and talk to the right node; a wrong guess gets a redirect.

• Resharding moves slots (and their keys) between nodes while the cluster runs.
• Multi-key operations only work when all keys are in the same slot. Hash tags (a {tag} part of the key) force related keys into one slot.
• Failover is built in: replicas of a failed primary are promoted by the cluster.
• This is the "fixed number of slots" cousin of consistent hashing (see the consistent-hashing lesson).

Which setup when

Need	Choice
Pure cache, loss acceptable	Single node or replicas, little or no persistence
Must survive restart	AOF (everysec) and snapshots
Must survive node loss	Replica plus Sentinel
Data larger than one node, or very high throughput	Redis Cluster

Also keep memory in mind: set a max memory and an eviction policy (see the caching lessons) so Redis behaves predictably when full, and watch for large keys and slow commands, which block a single-threaded executor.

Real-world reading