Consistency, availability and the CAP theorem

The problem

On one machine, data has one value. Spread it over several machines and new questions appear:

• If I write to machine A, when can machine B see it?
• If A and B cannot talk to each other, should they keep serving requests?
• If two people change the same item at the same moment, who wins?

Consistency models are the answers to the first question. The CAP theorem frames the second.

Consistency: what a read may return

From strongest to weakest, in plain words:

• Strong (linearizable): once a write completes, every later read anywhere sees it. It behaves like one copy. Easiest to reason about, slowest and least available.
• Read-your-writes: you always see your own updates, though others may lag.
• Eventual: if writes stop, all copies will eventually agree. Reads may be stale in between. Fast and highly available.

100%

Figure SDA-FND-07-1 · Replica lag and eventual consistency · © forcsguy.com

Stronger guarantees cost latency (replicas must coordinate) and availability (some must be reachable). Weaker ones are cheaper but push complexity to the application.

The CAP theorem

For a distributed data store, three properties are in tension:

• Consistency (C): every read sees the latest write (in the strong sense above).
• Availability (A): every request to a working node gets a response.
• Partition tolerance (P): the system keeps working even when messages between nodes are lost or delayed.

Networks do fail, so P is not optional. The real choice is what to do when a partition happens:

100%

Figure SDA-FND-07-2 · Choosing C or A during a partition · © forcsguy.com

• CP systems (for example many coordination stores and strongly consistent databases) may reject requests rather than risk a wrong answer.
• AP systems (for example many Dynamo-style stores) stay up and let replicas diverge briefly, then merge.

A more useful lens: latency too

Even without partitions, strong consistency costs time because replicas must agree. The PACELC idea adds this: if there is a Partition choose A or C, Else (normal operation) choose between Latency and Consistency. It explains why a store may feel slower when configured to be strongly consistent.

100%

Figure SDA-FND-07-3 · How many copies must confirm a write · © forcsguy.com

Resolving conflicts

If two replicas accept different writes, the system needs a rule:

• Last write wins: simple, but silently drops one update.
• Version vectors or merge logic: detect that the writes were concurrent, then merge or ask the user.
• CRDTs: data types designed so concurrent updates always merge cleanly (used for counters and collaborative editing).

Choosing per feature, not per system

You rarely need one consistency level for the whole product. Use strong consistency where wrong data is costly (payments, inventory, seat booking) and eventual consistency where speed and uptime matter more (counts, feeds, recommendations). Many databases let you pick per query.

A worked example: quorums with numbers

A key is stored on N = 3 replicas. A write must be acknowledged by W of them, and a read asks R of them and takes the newest version.

Setting	W	R	R + W > N?	What you get
Fast reads	3	1	yes (4 > 3)	Strong reads, but one dead replica blocks writes
Balanced	2	2	yes (4 > 3)	Strong reads; survives one replica down for both reads and writes
Fast writes	1	3	yes (4 > 3)	Quick writes, slow reads, reads fail if any replica is down
Very fast	1	1	no (2 < 3)	Fastest, but a read can miss the latest write: eventual consistency

The rule R + W > N guarantees the set of replicas read overlaps the set that acknowledged the last write, so at least one has the newest value. Concretely, with W = 2, R = 2: a write of version 5 reaches replicas A and B; C is still at version 4. A read that asks B and C sees versions 5 and 4 and returns 5. Whichever two replicas you ask, at least one has version 5.

100%

Figure SDA-FND-07-4 · Why read and write quorums overlap · © forcsguy.com

A story: the comment that disappeared

A user posts a comment. The write goes to the primary and the page reloads, but the read is served by a replica that is 300 ms behind. The comment is missing, the user posts it again, and now there are duplicates.

This is a read-your-writes violation, the most common user-visible consistency bug. Fixes, from cheapest to strongest:

1. For a short time after a user writes (say 5 seconds), route that user's reads to the primary.
2. Return the new item in the write response and show it on the page without re-reading.
3. Track a version or log position from the write, and have the replica serve the read only when it has caught up to it.
4. Make that data strongly consistent end to end (higher latency, fewer failure modes tolerated).

Not every read deserves strong consistency. A like counter that is a few seconds old is fine; a balance after a transfer is not.

What a network partition actually does

Imagine a three-node store split into two nodes on one side and one node on the other.

• A CP system (consistency over availability) lets only the side with a majority accept writes. The lone node refuses requests, which is unavailable but correct.
• An AP system (availability over consistency) lets both sides accept writes. Everyone gets an answer, but the two sides now hold different versions, and after the partition heals the system must merge: last-write-wins (simple, can lose data and trusts clocks), version vectors that keep both and ask the application, or data types designed to merge (CRDTs such as counters and sets).

100%

Figure SDA-FND-07-5 · Behaviour during a partition · © forcsguy.com

Mistakes people make

• Reading CAP as "pick any two". Partitions happen whether you like it or not, so the real choice is what to give up during a partition. Without partitions you can have both.
• Assuming "eventual" means "soon enough". It means "if writes stop, replicas converge", with no bound. Measure the real lag.
• Last-write-wins on unsynchronised clocks. A node with a clock a second ahead silently overwrites newer data. Prefer logical versions.
• One setting for the whole system. Choose per feature: strong for payments and inventory, eventual for counters, feeds and recommendations.
• Forgetting the latency side. Even without partitions, stronger consistency costs round trips between replicas, a trade-off often called PACELC.

Try it yourself

Use the CAP theorem playground to partition a cluster and watch CP and AP behave differently, and the replication playground to see stale reads and what a failover loses. Then try the System Builder with replicas.

More on this site