Design a notification platform capable of sending emails, SMS, and push notifications. Every notification should be delivered exactly once from the business perspective.

How I frame it

Wire protocols are at-least-once. Business rule: one OrderPlaced event → at most one email + one SMS + one push per user preferences — not duplicate confirmations on any channel.

High-level architecture

• Ingest: REST POST /notifications or consume domain events from Kafka with event_id.
• Router: Resolves user preferences, quiet hours, channel priority (push first, email fallback).
• Job store: notification_jobs(event_id, channel, template, status) with UNIQUE(event_id, channel).
• Channel adapters: SendGrid, Twilio, FCM — each with retries, circuit breaker, provider idempotency keys.
• Workers per channel: Scale SMS separately from email; DLQ for poison templates.

OrderPlaced E99
  → jobs: (E99, EMAIL), (E99, PUSH)  UNIQUE per channel
  Email worker: send → SENT
  Retry E99 EMAIL: unique block → skip

Exactly-once semantics

Insert job row before send (outbox). Status machine: PENDING → PROCESSING → SENT | FAILED. Reconciliation polls FAILED with exponential backoff. Analytics on delivery latency per channel.

Walkthrough

OrderPlaced event E99 arrives twice (Kafka redelivery). Router tries INSERT (E99, EMAIL) and (E99, PUSH) — second delivery hits unique constraint, skips duplicate jobs. Email worker sends once, marks SENT. Push worker crashes after FCM accept — retry reads SENT or uses FCM collapse key — user gets one notification per channel.

Weak answer

"Deduplicate in memory per worker." Does not survive restarts or scale-out. Separate email/SMS/push services without shared event_id dedupe send duplicate confirmations on different channels inconsistently.

Follow-up questions

User opts out of SMS mid-flight?

Preference check at PROCESSING time; cancel job if opted out before send.

Priority / digest mode?

Batch low-priority into daily digest — separate job type with UNIQUE(user_id, digest_date).

Multi-region?

Job store in primary region; consumers idempotent via event_id — safe if event bus duplicates across regions.