♻️Idempotency

Retry safely by making repeated requests produce one effect.

📌 Key IdeaDistributed systems retry on timeouts, queue redelivery, and client errors. Idempotency keys plus durable dedupe storage ensure repeated requests produce one side effect and return the same response — preventing double charges, duplicate orders, and inconsistent state.

🧩

Problem it solves

Networks and load balancers cause ambiguous failures — client sends request, server processes, but response is lost. Client retries → duplicate side effect without idempotency.
At-least-once message delivery (SQS, Kafka, RabbitMQ) redelivers messages — consumers must not apply the same event twice.
Payment APIs, order creation, and inventory reservation are non-idempotent by default — retries can double-charge or oversell.

📖

Definition

An operation is idempotent if performing it multiple times has the same effect as performing it once. Idempotency keys (client-generated unique IDs) let the server detect replays: first request processes and stores the result; subsequent requests with the same key return the stored result without re-executing.

✨

Why It Matters

Enables safe retries — the foundation of reliable distributed systems.
Required companion to at-least-once delivery in queues and event consumers.
Stripe, PayPal, and major payment APIs mandate idempotency keys — industry standard pattern.
Without it, "exactly-once" semantics are impossible; you only get at-least-once with duplicates.

🖼

Visual

♻️ Safe retries with idempotency key

Client -> POST /payments (Idempotency-Key: K1)
Server idempotency store:
  miss  -> process + persist response (201)
  hit   -> return stored response (201) — no duplicate charge

Same key + same payload -> same outcome
Different payload + same key -> 409 Conflict

✅

When to Use

Payment/order creation and any non-repeatable financial side effect.
POST endpoints that create resources (not just GET/PUT/DELETE which are often naturally idempotent).
At-least-once event/message consumers (SQS, Kafka, RabbitMQ).
Client retries after timeout, 502, or connection reset.
Webhook handlers that may receive duplicate delivery notifications.

🚫

When NOT to Use

Pure reads (GET) — already safe to retry.
Append-only analytics counters where approximate counts are acceptable (unless dedupe required).
Operations where generating a fresh idempotency key per retry defeats the purpose (client must reuse same key on retry).

💻

Code Variations

1️⃣ API idempotency key checkjava

String key = request.header("Idempotency-Key");
if (key == null) throw new BadRequestException("missing key");

Optional<CachedResponse> cached = idemStore.get(key);
if (cached.isPresent()) return cached.get();

Response r = processPayment(cmd);
idemStore.put(key, r, Duration.ofHours(24));
return r;

Store final response + status. Return same response on replay.

2️⃣ Event consumer dedupe with unique constraintjava

@Transactional
public void handle(OrderCreatedEvent event) {
  try {
    processedRepo.insert(new ProcessedEvent(event.id()));
  } catch (DuplicateKeyException e) {
    return; // already handled
  }
  applyBusinessLogic(event);
}

Insert dedupe record BEFORE side effects, in same transaction if possible.

3️⃣ Concurrent request race — DB unique guardjava

@Transactional
public Order createOrder(CreateOrderCmd cmd, String idemKey) {
  idemRepo.insert(new IdemRecord(idemKey, "IN_PROGRESS"));
  Order order = orderRepo.save(buildOrder(cmd));
  idemRepo.updateResult(idemKey, order.id());
  return order;
}
// Unique constraint on idemKey — second concurrent insert fails

Unique constraint prevents double processing under concurrent identical requests.

4️⃣ Idempotency checklisttext

- Client generates key (UUID) BEFORE first attempt; reuses on retry
- Server stores: key, request hash, response, status, timestamp
- TTL >= client retry window (24–72h payments)
- Reject same key + different body (409 Conflict)
- Dedupe BEFORE side effect, not after
- Works with: HTTP APIs, webhooks, queue consumers

📊

Idempotency strategies

Aspect	Strategy	How it works
Idempotency key store	Client sends key; server stores response by key	HTTP APIs, payments, order creation
Natural idempotency (UPSERT)	DB unique constraint + INSERT ON CONFLICT / upsert	Entity creation with stable business ID
Event dedupe table	Store processed event IDs; skip if seen	Queue/Kafka consumers
State machine / version check	Reject transition if current state != expected	Order status workflows
Distributed lock + check	Lock resource, check if already processed	When DB constraint alone is insufficient

⚖️

Key trade-offs

Area	Benefit	Cost	Mitigation
Correctness	Safe retries under ambiguity	Extra storage + lookup on every request	Redis/DB with TTL; index on idempotency key
Key TTL	Bounded storage	Too short → duplicate after TTL expiry	TTL ≥ max client retry window (24–72h for payments)
Concurrent duplicates	Two identical requests in flight	Race if check-then-act without lock	Unique DB constraint or INSERT-first pattern
Key reuse with different body	Detect client bugs	Must reject (409) not silently ignore	Hash request body; compare on key hit

🔀

Common Confusions Cleared Up

Idempotent vs safe to retry

Safe to retry means retrying won't cause harm. Idempotent means N executions = 1 effect. All idempotent ops are safe to retry, but not all safe-to-retry ops are idempotent (e.g. read that returns different data each time is safe but not idempotent in strict sense).

HTTP method idempotency (spec vs reality)

PUT and DELETE are idempotent by HTTP spec intent. POST is not. But implementation bugs (POST that creates new row every time) violate expectations — always verify side effects, don't assume PUT/POST behavior from method alone.

Idempotency key vs correlation ID

Correlation ID traces a request across services (observability). Idempotency key prevents duplicate processing (correctness). Use both — they serve different purposes.

🧪

Variations / Implementations

1) API idempotency key (Stripe-style)

Client sends Idempotency-Key header. Server checks store: if key exists, return cached response; else process, store response + status code, return result. TTL 24h+ for payment flows.

2) Event consumer dedupe

Before processing event, INSERT INTO processed_events (event_id) VALUES (?)" — unique constraint rejects duplicate. Or Redis SETNX event_id with TTL.

3) Database upsert idempotency

INSERT INTO orders (id, ...) VALUES (?, ...) ON CONFLICT (id) DO NOTHING — same order ID always produces one row. Business ID as natural idempotency key.

⚠️

Common Mistakes / Anti-Patterns

Generating new idempotency key on each retry (defeats dedupe).
Storing idempotency record after side effect (race window).
No TTL on idempotency store — unbounded growth.
Same key accepting different request bodies silently.
Assuming POST is "fine to retry" without idempotency key.
In-memory dedupe only — lost on restart, duplicates on redeploy.

🎤

Interview Q&A

🔮

Likely Follow-up Questions

How would you design idempotency for a webhook that may arrive 5 times?
What should happen if same idempotency key is used with a different amount?
How do you test idempotency in integration tests?

🧾

Quick Cheat Sheet

Core ruleSame key → same effect + same response.

Client dutyGenerate key once; reuse on retry.

StoreDurable (DB/Redis) with TTL ≥ retry window.

OrderDedupe check BEFORE side effect.

QueuesAt-least-once delivery requires idempotent handler.

ConflictSame key + different body → 409.

🔗