Glossary
Idempotency
Also: Idempotent
An operation is idempotent when running the same request several times has the same result as running it once.
In distributed systems the retry is the normal case: a call runs into a timeout, the caller does not know whether it arrived, and tries again. Without idempotency that becomes a second payment or a duplicate order.
It is usually implemented with a key supplied per operation, which the receiver stores. If the same key arrives again, it returns the stored result instead of processing again.
The reason there is no way around it is a property of the network: a timeout says nothing about whether the other side processed the request. It may have been lost, it may have been processed with the response lost, or it may still be running. All three cases look identical to the caller, which is why the retry has to be safe rather than avoided.
The same situation arises in message processing. Queue services usually guarantee at-least-once delivery, not exactly once. A message whose acknowledgement is lost, or whose processing exceeds the deadline, is delivered again. A consumer without idempotency then creates duplicate bookings, and not during failures but in normal operation under load.
The key has to come from the caller and identify the business operation, not the individual attempt. An order number or a case number from the source system works, or an identifier generated on the first attempt that stays the same across all retries. A random value generated per attempt looks like idempotency and protects against nothing.
The most widespread faulty implementation is the check before processing without a lock: first look whether the key already exists, then process, then store. Two concurrent retries both pass the check before either of them stores. The correct approach is to insert the key with a uniqueness constraint in the same transaction as the business change; the second attempt then fails on the constraint rather than on a query.
The stored result is part of the full picture. If a retry only reports "already processed" without returning the original response, the caller is none the wiser. Idempotency means giving the same answer again, not rejecting the second request.
How you notice it
- Calls can be repeated because of timeouts.
- It involves payments, orders or shipping.
- Duplicate records appear sporadically, mostly under load.
- A queue service guarantees at-least-once delivery.
- A third-party system resends callbacks when no acknowledgement arrives.
Not to be confused with
- Exactly-once delivery
- A promise from the transport layer that does not exist in practice. What does exist is at-least-once delivery plus idempotent processing, and together they produce the intended effect.
- Deduplication
- Sorting out duplicates after the fact, often over a time window. Idempotency prevents the second effect instead of cleaning it up later.
- Transaction
- Ensures that a piece of processing happens completely or not at all. It says nothing about what happens on a second identical request.
When it fits
- For every operation that moves money, orders or shipments.
- For consumers of queues with at-least-once delivery.
- Everywhere callers retry automatically on a timeout.
- For callbacks from third-party systems that resend when no acknowledgement arrives.
When it does not
- For pure read queries: those are idempotent by nature.
- For operations whose repetition is intended, such as creating several identical line items.
How to approach it
- Determine the business keyWhat exactly is one operation: an order, a payment, a shipment. The key identifies that, not the transmission attempt.
- Require the key from the callerAs a field or a header, mandatory. If it is generated on the server, the retry does not know it.
- Enforce uniqueness in the databaseA unique index on the key, inserted in the same transaction as the business change. A query placed in front of it does not protect against concurrency.
- Store the result alongsideResponse status and payload. The retry gets the same answer, not an error message.
- Define a retention periodAs long as retries are realistic, usually 24 hours to seven days, with automatic clean-up.
- Force retries in the testsSend the same request twice and in parallel. If no test does that, the bug is found in production under load.
Frequently asked
How long must idempotency keys be kept?
As long as retries are realistic, usually 24 hours to seven days. Shorter reopens the gap, much longer only costs storage. What matters is that the key identifies the business operation, not the individual attempt.
Is it enough to check first whether the operation already exists?
No, that is the most common faulty implementation. Two concurrent retries both pass the check before either of them writes. Uniqueness has to be enforced by the database, through a unique index set in the same transaction as the business change.
What do you return on a retry?
The same answer as the first time, from the stored result. An error such as "already processed" forces the caller to handle a special case, and that is exactly what produces new bugs. If the same key arrives with a different payload, that is a genuine conflict and should be reported as one.
Does this apply to queues as well?
Especially there. Common services promise at-least-once delivery, not exactly once. A message whose processing exceeds the visibility timeout is delivered again while the first run is still going. Without idempotent consumers, duplicate bookings appear in normal operation, not only during failures.