An admin updates a product title in the database, but Elasticsearch still shows the old title for two minutes. A customer searches the new title and gets zero results. How do you design indexing and consistency for this?
How I frame it
Search is a derived read model. DB is source of truth; ES lags. Design for bounded staleness, reliable indexing, and UX that does not promise instantaneous global search after every admin save.
Indexing pipeline
- Transactional outbox on product update →
ProductChanged(id, version)to Kafka. - Indexer consumer: Load latest row by id (or embed payload),
index/updateES doc withexternal_version= DB version — ignore stale events. - CDC (Debezium) as alternative to app outbox — same consumer pattern.
- Refresh policy: Prefer near-real-time (
refresh_interval1s) for catalog; do not block admin HTTP on ES refresh unless "preview search" is required.
Admin: title "Galaxy Buds" → "Galaxy Buds Pro" (version 44) Outbox → Kafka → Indexer → ES doc version 44 Search "Buds Pro" before index: 0 hits Mitigations below
Closing the UX gap
- Admin preview: After save, query primary DB / admin API by id — not public search — to confirm edit.
- Synchronous index on critical path (optional): Admin "Publish" waits for indexer ack +
_refreshfor that id — rare, paid for correctness. - Fallback search: If ES miss on exact SKU / id, GET product by id from DB.
- Alias swap for big reindex: Blue/green index; never dual half-updated public alias mid-migration.
Weak answer
"Write to ES in the same request as DB." Dual-write drifts on crashes; no versioning. "Just lower refresh_interval to 0." Costly; still eventual if indexer is behind.
Follow-up questions
Indexer lag of 10 minutes — alert?
Yes — consumer lag + "age of oldest unindexed outbox row" SLO. Page when search SLO burn threatens launch windows.
Partial updates / nested variants?
Index denormalized product document; version whole doc. Partial ES update scripts only with care for lost updates.
Delete product?
Outbox ProductDeleted → ES delete by id; idempotent. Tombstone if consumers reorder.