A trading platform's Market Data Service receives ~500,000 price updates per second from the exchange. Order Validation, Risk Engine, Portfolio Valuation, and Order Execution all need the latest price with low latency. How would you design this?

How I frame it

Two problems: ingest 500k/sec without dropping ticks, and serve latest price to four consumers with sub-millisecond reads. I'd separate the write path (exchange → canonical price book) from the read path (services read local copy, not the exchange feed).

Core architecture

• Market Data Service (MDS): Single writer per symbol partition — maintains symbol → { price, bid, ask, timestamp, sequence } in memory (off-heap / lock-free map). Sequence number detects gaps and stale ticks.
• Partition by symbol: Shard ingest across MDS instances — hash(symbol) → partition. 500k/sec spread across shards; hot symbols (index futures) get dedicated partition.
• Fan-out to downstream: Publish coalesced updates on a low-latency bus (Kafka with compaction per key, Aeron, Redis pub/sub for smaller scale, or multicast UDP for HFT). Coalesce: if AAPL gets 200 ticks in 1ms, downstream may receive last tick + sequence.
• Local read replica per service: Each downstream keeps an in-process price cache updated via subscription — reads never hit network on order path. Risk check: priceCache.get("AAPL").sequence in nanoseconds.

Stock exchange          Market Data Service (MDS)              Downstream
     |                         |                                    |
     |-- multicast feed ------>| ingest + normalize                 |
     |   (500k updates/s)      | in-memory price book (per symbol)  |
     |                         |   AAPL: 187.42 @ seq 9_812_004    |
     |                         |                                    |
     |                         |-- fan-out (Kafka / Aeron / SHM) -->| Order Validation
     |                         |                                    | Risk Engine
     |                         |                                    | Portfolio Valuation
     |                         |                                    | Order Execution
     |                         |                                    |
     |                         |<-- GET /price/AAPL (local cache) --| (sub-ms read)

Keeping "most recent" correct

• Monotonic sequence per symbol — reject or buffer out-of-order ticks; exchange seq is source of truth.
• Last-write-wins on timestamp + seq — not wall-clock on consumer (clock skew).
• Snapshot + delta: Periodic full snapshot + incremental updates so late joiners catch up without replaying 500k/sec history.
• Staleness guard: If no tick for symbol in N ms, block new orders or flag price as stale in Risk Engine.

// In-process read (Order Validation)
PriceTick tick = localBook.get(symbol);
if (tick.sequence < minRequiredSeq) reject("stale price");
if (now() - tick.exchangeTs > 500ms) reject("market data stale");

Latency vs throughput trade-offs

• Coalescing: Portfolio Valuation may tolerate 100ms batched ticks; Order Execution needs every tick or 1ms coalesce — different consumer SLAs, same source feed.
• UDP multicast from MDS for ultra-low fan-out; TCP/Kafka for durability and replay.
• Colocation: MDS and execution in same AZ / same rack as exchange gateway — cross-region adds ms unsuitable for execution path.

Weak answer

"Each service calls MDS REST API on every order." At 500k updates/sec and thousands of order checks/sec, central HTTP becomes bottleneck and adds ms latency. Push + local cache is required.

Follow-up questions

MDS crashes — how do downstream services recover?

Failover standby MDS from exchange feed (hot standby). Downstream detects sequence gap → request snapshot API → resume delta stream. Execution halts if snapshot lag exceeds threshold.

One symbol (AAPL) gets 50% of all ticks — hot key?

Dedicated partition + dedicated thread for hot symbol. Coalesce aggressively for Portfolio Valuation; full fidelity for Execution on that symbol only.

Kafka fast enough?

Kafka works for many retail platforms with coalescing and partitioning. Sub-ms execution often uses Aeron, shared memory, or bespoke multicast — mention tiered design: Kafka for audit/replay, memory bus for hot path.

How does Risk Engine use price on a basket order?

Atomic read of all symbols in basket at validation time; check all sequences > watermark. If any stale, reject entire basket — avoids partial stale exposure.