Delivery guarantees

At most once, at least once and exactly once, with a consumer that crashes.

On the roadmap:Intermediate · step 4: Messaging

Intermediateabout 10 to 15 min6 experiments

Experiments

New here? A few short steps show what happens, one at a time.

Settings

When the consumer acknowledges

Acknowledge firstat most onceAcknowledge after the workat least onceAcknowledge after, and skip repeatseffectively onceWork and acknowledgement togetherexactly once (in one system)

The consumer does the work, then acknowledges. A crash before the acknowledgement makes the broker send the message again, so work done just before the crash is done twice.

Consumer crashes while handling10%

The producer retries when the broker's acknowledgement is lost

Handled exactly once

94.7%

Lost

Handled twice or more

527

Copies in the queue

10,000

Nothing was lost, but 527 messages were handled more than once (559 extra times in all). The consumer has to cope with repeats.

The same messages, every way of acknowledging

Guarantee	Exactly once	Lost	Done twice or more
at most once	89.9%	1,008	0
at least once	94.7%	0	527
effectively once	100.0%	0	0
exactly once (in one system)	100.0%	0	0

Where a crash can fall

1. Before the work
Acknowledging first has already told the broker it is done, so the message is lost. Acknowledging after leaves it unacknowledged, and it is sent again.
2. After the work, before the acknowledgement
The work happened and the broker does not know, so it sends the message again and the work repeats, unless repeats are recognised.
3. After the acknowledgement
Nothing is lost: the work is finished and the broker knows. This is the only crash that is always safe.

In plain words

A message queue promises something about how often a message is handled: at most once, at least once, or exactly once. The promise depends on when the consumer tells the queue it is done, because a crash can fall between handling a message and saying so. Exactly once is built from at-least-once plus a way to recognise repeats.

Why it matters

Every system that moves messages has to choose what happens when a worker dies mid-task, and the choice is between losing work and doing it twice. Understanding the three guarantees explains why almost everything is at-least-once, why handlers must be idempotent, and what 'exactly once' really means.

How real systems do it

The simulation is a simplified model. Here is what production systems add or do.

Amazon SQS standard queues deliver at least once and may deliver a message twice, so handlers must cope. SQS FIFO queues add deduplication by message ID within a window of five minutes.
Kafka's consumers commit offsets, and committing after processing gives at-least-once delivery. Kafka's idempotent producer (enable.idempotence) stops retries creating duplicates, and transactions let a consumer commit its output and its offsets together, which gives exactly-once processing within Kafka.
Exactly-once delivery across two separate systems is not possible in general, because a crash can always fall between the two. What systems offer is exactly-once effect: at-least-once delivery plus idempotent handling.
Idempotent handling is often a unique constraint on the message ID, or a stored record of IDs already processed in the same database transaction as the effect.