Your SaaS API serves 2,000 merchants. One noisy merchant floods the API and slows everyone. How do you enforce per-tenant quotas and isolate failure without a single shared bottleneck?

How I frame it

This is the noisy neighbor problem. Isolation has two layers: control plane (how much traffic each tenant may send) and data / runtime plane (one tenant's query load or poison traffic cannot exhaust shared pools). A global 100k RPS limit alone is not enough — it still lets one tenant consume the budget.

Per-tenant quotas (control plane)

• Identity first: Resolve tenant_id at the gateway from API key / OAuth client before business work.
• Quota by plan: Redis counter per rate:<tenant>:<window> — Free 100 RPS, Pro 2k RPS, Enterprise custom. Return 429 + Retry-After + X-RateLimit-Remaining.
• Burst vs sustained: Token bucket so brief spikes pass; sustained flood trips.
• Cost-based limits: Heavy endpoints (export, search) cost more tokens than GET by id — one tenant can't burn the cluster with "allowed" expensive calls.

2,000 merchants → API Gateway
                      |
         +------------+------------+
         | identify tenant_id      |
         | (API key / JWT claim)   |
         +------------+------------+
                      |
         +------------v------------+
         | Per-tenant rate limit   |  Redis: rate:{tenant}:{window}
         | (quota by plan)         |  429 + Retry-After if over
         +------------+------------+
                      |
         +------------v------------+
         | Bulkhead thread pools   |  noisy tenant can't fill
         | / queue per tenant tier |  all workers for others
         +------------+------------+
                      |
         App servers → DB (tenant_id on every row; RLS or shard key)

Runtime isolation (so quotas aren't the only line)

• Bulkheads: Separate worker pools / queue partitions for tier:enterprise vs tier:free. Noisy free tier exhausts its pool, not enterprise.
• Concurrency caps: Max in-flight requests per tenant (e.g. 50) — stops slow endpoint pile-up even under limit.
• DB isolation: Every table carries tenant_id; indexes lead with it. Prefer Postgres RLS or app-enforced filters. Heavy analytics on separate read replica or warehouse — not on OLTP.
• Hot-tenant shard escape hatch: If one merchant is consistently 10%+ of load, move them to a dedicated DB schema / shard — quarantine without rewriting the product.
• Circuit per tenant: If tenant T's error rate or latency explodes, trip their circuit; other tenants unaffected.

Avoiding a single shared bottleneck

• Redis Cluster shards rate keys by tenant_id — no one Redis node holds all counters.
• Gateway horizontally scaled; tenant map cached; limit check is O(1) Lua, not a central "quota service" single box.
• Don't put all tenants on one DB primary forever — partition by tenant hash when row count or IOPS demands it.

Walkthrough

Merchant M42 scripts a 50k RPS scrape on GET /orders. Gateway hits their Pro limit (2k RPS) within 1s → 429. Remaining in-flight (concurrency 50) drains. Other 1,999 merchants see normal p99. Ops gets alert on tenant throttle rate for M42 — optional sales outreach or temporary hard ban if abuse.

Weak answer

"One global rate limiter." Quiet tenants starve when one floods. "Separate Kubernetes cluster per merchant." Unviable at 2,000 — use soft isolation first, dedicated only for whales.

Follow-up questions

Data leak — tenant A reads tenant B?

Defense in depth: query always filters tenant_id, RLS policies, integration tests that assert cross-tenant 404, no shared sequential IDs in public URLs without authz check.

Fairness within the same plan?

Same quota ceiling; optional soft priority for lower recent usage. Never let one free-tier peer monopolize a shared bulkhead beyond concurrency cap.

Rate limit Redis is down?

Fail closed for unauthenticated / free tier; fail open with local token bucket for paid critical paths — document per tier. See Redis-down failure design.

Billing on usage vs hard quota?

Soft quota with overage billing for Enterprise; hard 429 for Free — product call, not pure infra.