🧯

Spring Boot — Real-World Scenarios

Production problems and how to actually solve them · 15 questions

Your endpoint needs data from three downstream services. Calling them one after another is too slow — how do you fetch them in parallel?

Must Know📊 Mid

Calling three services sequentially means your total latency is the sum of all three response times. Since the calls are independent, kick them all off at once and combine the results once every future completes — the total latency drops to roughly the slowest single call, not the sum.

Fan-out with CompletableFuturejava

CompletableFuture<User> userFuture = CompletableFuture.supplyAsync(() -> userClient.get(id), executor);
CompletableFuture<List<Order>> ordersFuture = CompletableFuture.supplyAsync(() -> orderClient.getFor(id), executor);
CompletableFuture<Wallet> walletFuture = CompletableFuture.supplyAsync(() -> walletClient.get(id), executor);

CompletableFuture<ProfileResponse> combined = CompletableFuture.allOf(userFuture, ordersFuture, walletFuture)
    .thenApply(v -> new ProfileResponse(userFuture.join(), ordersFuture.join(), walletFuture.join()));

ProfileResponse response = combined.get(2, TimeUnit.SECONDS); // overall timeout, not per-call

Two things people forget: use a dedicated executor (not the common ForkJoinPool) so a burst of these calls can't starve unrelated async work elsewhere in the app, and set an overall timeout on the combined future so one slow downstream doesn't hang the whole request forever. If the app is fully reactive, the equivalent is Mono.zip(...) with WebClient instead of blocking futures.

ScenarioCompletableFuturePerformance

Different controllers in your codebase return wildly different error response shapes, confusing API consumers. How do you standardize error responses across the whole application?

Must Know📊 Mid

The fix is a single global exception handler and one consistent error response shape that every controller funnels through, instead of each controller handling (or not handling) exceptions its own way.

One error shape, one place it's builtjava

record ApiError(String code, String message, Instant timestamp, List<String> details) {}

@RestControllerAdvice
class GlobalExceptionHandler {
    @ExceptionHandler(ResourceNotFoundException.class)
    ResponseEntity<ApiError> handleNotFound(ResourceNotFoundException ex) {
        return ResponseEntity.status(404)
            .body(new ApiError("NOT_FOUND", ex.getMessage(), Instant.now(), List.of()));
    }

    @ExceptionHandler(MethodArgumentNotValidException.class)
    ResponseEntity<ApiError> handleValidation(MethodArgumentNotValidException ex) {
        var details = ex.getBindingResult().getFieldErrors().stream()
            .map(f -> f.getField() + ": " + f.getDefaultMessage()).toList();
        return ResponseEntity.status(400)
            .body(new ApiError("VALIDATION_FAILED", "Request validation failed", Instant.now(), details));
    }

    @ExceptionHandler(Exception.class) // catch-all -- never leak a raw stack trace to clients
    ResponseEntity<ApiError> handleUnexpected(Exception ex) {
        log.error("Unhandled exception", ex);
        return ResponseEntity.status(500)
            .body(new ApiError("INTERNAL_ERROR", "Something went wrong", Instant.now(), List.of()));
    }
}

Migration strategy for an existing large codebase: add the catch-all handler first (immediately standardizes anything currently leaking raw 500s/stack traces), then progressively replace scattered try/catch blocks in individual controllers with specific custom exceptions handled centrally.

ScenarioException HandlingAPI Design

A downstream microservice your app depends on becomes slow. Requests start piling up and your own service starts failing too. How do you prevent this cascading failure?

Must Know📊 Mid

Without protection, every caller thread ends up blocked waiting on the slow dependency; thread pools fill up, and now your own service can't serve any request — even ones that don't need the slow dependency at all. Three complementary defenses (Resilience4j is the standard library for this in Spring Boot):

Timeout — never wait indefinitely; fail the call after a bounded time so the calling thread is freed.
Circuit breaker — after enough recent failures, stop even trying to call the dependency for a cooldown period, failing fast instead.
Bulkhead — cap how many concurrent calls to this one dependency are allowed, so a flood of slow calls to it can't consume your entire thread pool, leaving capacity for unrelated requests.

Resilience4j annotationsjava

@CircuitBreaker(name = "inventoryService", fallbackMethod = "fallbackInventory")
@TimeLimiter(name = "inventoryService")
@Bulkhead(name = "inventoryService")
CompletableFuture<Stock> getStock(String sku) {
    return CompletableFuture.supplyAsync(() -> inventoryClient.getStock(sku));
}

Stock fallbackInventory(String sku, Throwable t) {
    return Stock.unknown(sku); // degrade gracefully instead of failing the whole request
}

See Circuit Breaker / Retry / Bulkhead → for the deeper pattern discussion.

ScenarioResilience4jMicroservices

A flaky mobile network causes clients to retry your POST /payments endpoint, occasionally charging customers twice. How do you make the endpoint idempotent?

Must Know📊 Mid

The client can't reliably tell "my request never arrived" from "my request succeeded but the response got lost" — so it retries, and a naive endpoint just processes the payment again. The fix: require the client to send a unique idempotency key per logical operation, and use it to detect duplicates.

Idempotency key patternjava

@PostMapping("/payments")
ResponseEntity<PaymentResult> createPayment(
        @RequestHeader("Idempotency-Key") String key,
        @RequestBody PaymentRequest request) {

    Optional<PaymentResult> existing = idempotencyStore.find(key);
    if (existing.isPresent()) {
        return ResponseEntity.ok(existing.get()); // same key seen before -- return the ORIGINAL result, don't re-charge
    }

    PaymentResult result = paymentService.charge(request);
    idempotencyStore.save(key, result); // atomic insert -- a unique DB constraint on the key prevents a race
    return ResponseEntity.ok(result);
}

The critical detail: the "has this key been seen, then store it" step must be atomic — a unique database constraint on the idempotency key column (rejecting a concurrent duplicate insert outright) is more reliable than a check-then-act read followed by a separate write, which has a race window if two retries arrive at nearly the same instant.

ScenarioIdempotencyPayments

A singleton @Service caches request counts in a plain HashMap field. Under load, counts come out wrong or the app occasionally throws exceptions. What’s going on, and how do you fix it?

Must Know📊 Mid

Spring @Service beans are singletons by default — one shared instance handles every concurrent request across every thread. A plain HashMap field on that singleton is therefore shared, mutable state accessed concurrently — exactly the setup for lost updates or, with HashMap specifically, potential internal corruption (infinite loops or dropped entries) since HashMap was never designed to be thread-safe at all.

Fix: a genuinely concurrent structure, no manual locking needed for simple countsjava

@Service
class RequestCounterService {
    private final Map<String, LongAdder> counts = new ConcurrentHashMap<>();

    void increment(String endpoint) {
        counts.computeIfAbsent(endpoint, k -> new LongAdder()).increment();
    }
}

ConcurrentHashMap handles concurrent structural access safely; LongAdder is specifically optimized for high-contention increments (faster than an AtomicLong under heavy concurrent writes, since it internally spreads updates across multiple cells to reduce contention). For anything beyond a simple counter — compound multi-step updates — reach for explicit synchronization or a database row with proper transaction isolation instead of trying to hand-roll it with more concurrent collections.

ScenarioConcurrencyThread Safety

A user notices that changing the id in GET /api/invoices/{id} lets them view another customer’s invoice. How do you find and fix this class of bug?

Must Know📊 Mid

This is IDOR (Insecure Direct Object Reference) — the endpoint correctly checks authentication (you must be logged in) but forgets object-level authorization (are you allowed to see this specific invoice?). It's one of the most common real-world API vulnerabilities precisely because authentication passing feels like "security is handled."

The bug, and the fixvulnerablejava

@GetMapping("/invoices/{id}")
Invoice getInvoice(@PathVariable Long id) {
    return invoiceRepo.findById(id).orElseThrow(); // never checks WHOSE invoice this is
}

Ownership check addedfixedjava

@GetMapping("/invoices/{id}")
Invoice getInvoice(@PathVariable Long id, @AuthenticationPrincipal UserDetails user) {
    Invoice invoice = invoiceRepo.findById(id).orElseThrow();
    if (!invoice.getOwnerId().equals(currentUserId(user))) {
        throw new AccessDeniedException("not your invoice");
    }
    return invoice;
}
// or, cleaner: scope the query itself so a mismatched id simply returns nothing
Invoice getInvoice(Long id, Long ownerId) {
    return invoiceRepo.findByIdAndOwnerId(id, ownerId).orElseThrow();
}

Finding this class of bug systematically: audit every endpoint that takes an ID from the client (path variable, request param, or even body field) and ask "does this query also scope by the authenticated user, or could any valid ID from anyone be passed here?" — a quick grep for repository calls that take only an ID with no owner/tenant check is a fast way to find candidates.

ScenarioSpring SecurityIDOR

Your Spring Boot application takes 45 seconds to start up, slowing down every deploy and local dev cycle. How do you diagnose and fix it?

Important📊 Mid

First, actually measure where the time goes — set --debug or check the auto-configuration report, and enable management.metrics/startup timing (context.getStartupInfo() or Spring Boot's built-in ApplicationStartup tracking with Java Flight Recorder events) instead of guessing.

Common real causes, roughly in order of how often they turn out to be the culprit:

Component scanning too broad a package — scanning far more classes than actually needed for beans; narrow @ComponentScan or the main class's package.
Eager, heavy bean initialization — a bean's constructor doing real work (warming a cache, pinging a remote service) at startup instead of lazily on first use. Mark genuinely optional beans @Lazy.
Too many auto-configurations evaluating conditions — excluding unused ones (@SpringBootApplication(exclude = ...)) if a starter pulled in something you don't use.
Database/schema validation or migration tools (Hibernate ddl-auto=validate, Flyway) scanning many tables — usually unavoidable but worth confirming it's not pointed at a much bigger schema than expected.
JVM/classloading overhead itself — for genuinely large apps, this is where tools like Class Data Sharing (CDS) or, longer-term, migrating hot paths to native images (GraalVM) actually move the needle.

ScenarioStartup Performance

During a rolling deployment, requests in flight get dropped when an old instance is killed. How do you achieve a graceful, zero-downtime shutdown?

Important📊 Mid

The problem: when an orchestrator (Kubernetes, an ASG) kills an old instance, it typically sends SIGTERM; if the app dies immediately, any request mid-flight gets an abrupt connection reset. Spring Boot has built-in graceful shutdown support since 2.3 — enable it and give it a sensible timeout:

application.ymltext

server:
  shutdown: graceful
spring:
  lifecycle:
    timeout-per-shutdown-phase: 30s

With this enabled, on SIGTERM the embedded server stops accepting new requests immediately but lets requests already in progress finish (up to the configured timeout) before the JVM actually exits.

That alone isn't the whole picture in a load-balanced deployment: the load balancer/service mesh also needs to stop routing new traffic to this instance before it starts shutting down (readiness probe flips to not-ready first), or traffic keeps arriving at an instance that's already draining. The typical sequence is: readiness probe fails → LB stops sending new traffic → a short delay for that to propagate → SIGTERM sent → graceful shutdown drains in-flight requests → process exits.

ScenarioDeploymentGraceful Shutdown

A database password was accidentally committed in application.yml. How should secrets and per-environment config actually be managed in a Spring Boot application?

Important📊 Mid

Secrets should never live in a file that's checked into version control, period — even a "private" repo, since history persists and access lists change over time. The standard approach layers Spring Boot's existing configuration precedence (see the property precedence question) with a secrets-specific source:

application.yml holds only non-sensitive, structural configuration — property names, not secret values.
Actual secret values come from environment variables injected at deploy time (Kubernetes Secrets, ECS task definitions), or a dedicated secrets manager (AWS Secrets Manager, HashiCorp Vault, Spring Cloud Config with encryption) fetched at startup.
Locally, developers use a personal, .gitignore'd application-local.yml or a .env file loaded outside the repo — never a committed file with real credentials, even placeholder-looking ones that later get "temporarily" filled in.

If a secret has already been committed, rotating it is mandatory — removing it from the latest commit isn't enough, since it's still recoverable from git history until the credential itself is invalidated at the source (database, provider dashboard).

ScenarioSecrets ManagementConfiguration

One API client is hammering your public REST API and degrading service for everyone else. How do you add per-client rate limiting?

Important📊 Mid

Rate limiting needs to track request counts per client (by API key or user ID, not just globally) against a rolling window, and it needs to work correctly across multiple instances of your service, not just in one JVM's memory.

Token bucket in Redis (shared across all instances)java

@Component
class RateLimiter {
    private final StringRedisTemplate redis;

    boolean allow(String apiKey, int limitPerMinute) {
        String key = "ratelimit:" + apiKey + ":" + (System.currentTimeMillis() / 60_000);
        Long count = redis.opsForValue().increment(key);
        if (count == 1) redis.expire(key, Duration.ofMinutes(2)); // auto-cleanup old windows
        return count <= limitPerMinute;
    }
}

// applied as a filter/interceptor before the request reaches the controller:
if (!rateLimiter.allow(apiKey, 100)) {
    response.setStatus(429); // Too Many Requests
    response.setHeader("Retry-After", "30");
    return;
}

A per-instance in-memory counter (e.g. Guava's RateLimiter) is simpler but gives each instance its own independent budget — with 4 instances behind a load balancer, a client could actually get 4× the intended limit depending on which instances its requests land on. Storing the counter in Redis (or an API gateway's built-in rate limiting, which many teams use instead of rolling their own) gives one shared, consistent limit across the whole fleet. See the Rate Limiting deep dive → for sliding-window vs fixed-window trade-offs.

ScenarioRate LimitingAPI Design

You run 4 instances of a service behind a load balancer, each with an identical @Scheduled job. Every night, the job runs 4 times instead of once. How do you fix it?

Advanced📊 Mid

Plain @Scheduled has no idea other instances exist — each JVM just runs its own timer independently. You need a way for instances to agree "only one of us runs this tonight."

🔒 Distributed lock around the job (ShedLock pattern)

Instance A @Scheduled fires --> tries to acquire lock "nightly-settlement" in shared store (DB/Redis)
Instance B @Scheduled fires --> tries to acquire the SAME lock, at nearly the same time
Instance C @Scheduled fires --> same

Only ONE instance's lock-acquire succeeds (atomic INSERT/SETNX) -- it runs the job
The other two see the lock is held -- they skip this run entirely

ShedLock — the standard library for exactly this problemjava

@Scheduled(cron = "0 0 2 * * *")
@SchedulerLock(name = "nightlySettlement", lockAtMostFor = "10m", lockAtLeastFor = "1m")
void runNightlySettlement() {
    // only one instance across the whole cluster actually executes this body per trigger
}

lockAtMostFor is the crash-safety net (if the holder dies mid-job, others can eventually take over); lockAtLeastFor prevents a fast job from letting another instance immediately re-run it in the same window due to clock drift between nodes.

ScenarioSchedulingDistributed Systems

Under load, requests start timing out with "Connection is not available, request timed out." How do you diagnose and fix a HikariCP connection pool exhaustion issue?

Advanced📊 Mid

This means every connection in the pool is checked out and none is being returned fast enough for new requests to get one within the wait timeout. Diagnosis before fix:

Check /actuator/metrics/hikaricp.connections.active vs hikaricp.connections.max — if active is pinned at max under load, the pool really is exhausted, not just slow.
Look for long-running or leaked transactions — a connection held open by a slow query, an accidentally missing @Transactional boundary that never commits, or code that opens a connection and forgets to close it outside try-with-resources.
Check for N+1 queries or missing indexes making each request hold its connection far longer than it should (see the N+1 question) — more requests then compete for the same limited pool.

Common fixes, roughly in order of how often each is the real causetext

1. Fix the actual slow query / N+1 / missing index -- connections held too long is usually the root cause
2. Set leakDetectionThreshold (HikariCP) to log a warning with a stack trace for connections held too long
3. Size the pool correctly -- bigger isn't always better; (core_count * 2) + effective_spindle_count is
   a reasonable starting formula, then tune from real observed data
4. Set a sane connectionTimeout so requests fail fast with a clear error instead of hanging indefinitely

Bumping maximum-pool-size as the first move often just delays the same symptom at a higher concurrency level — it's a band-aid unless the pool was genuinely undersized for legitimate concurrent load, not held-too-long connections. See Connection Pooling (HikariCP) deep dive →.

ScenarioHikariCPPerformance

Your Spring Boot service’s memory usage climbs steadily over days until it OOMs and restarts. How would you diagnose and fix the leak in production?

Advanced📊 Mid

Confirm it's a real leak, not just expected heap sawtooth behavior — check /actuator/metrics/jvm.memory.used for the Old Generation over many GC cycles: normal apps show usage rise then drop after a Full GC; a leak shows the post-GC baseline trending upward over hours/days instead of returning to a stable floor.

Enable -XX:+HeapDumpOnOutOfMemoryError so the next OOM automatically captures a heap dump, or trigger one manually via jmap/Actuator's heapdump endpoint before it crashes.
Open the dump in a memory analyzer (Eclipse MAT, VisualVM) and look at the "dominator tree" / leak suspects report — it'll show which object graph is retaining the most memory.
Trace the retention path back to what's holding the reference — classic culprits: a static collection used as an unbounded cache, listeners/callbacks registered but never unregistered, ThreadLocal values never cleared in a pooled-thread environment, or an unclosed resource accumulating native/off-heap memory.

Fix the retention path (add eviction to the cache, unregister the listener, clear the ThreadLocal in a finally), then confirm the Old Gen baseline actually stabilizes under the same sustained load afterward — don't declare victory from code review alone.

ScenarioMemory LeakActuator

A paginated listing endpoint backed by a 50-million-row table has become unbearably slow, mostly because of the COUNT(*) needed for total pages. How do you speed it up?

Advanced📊 Mid

A naive Page<T> query runs the main SELECT ... LIMIT/OFFSET plus a separate COUNT(*) over the same filtered predicate — on a huge table, that count can dominate total query time, especially once OFFSET gets large (the database still has to scan and discard all the skipped rows).

Skip the exact count when you don't truly need it — most UIs only need "is there a next page?", not "there are exactly 4,382,109 results." Spring Data's Slice<T> (instead of Page<T>) fetches one extra row to answer that without running COUNT(*) at all.
Keyset (cursor) pagination instead of offset pagination — instead of OFFSET 1000000, query WHERE id > :lastSeenId ORDER BY id LIMIT 20. This stays fast regardless of how deep into the results you page, since it's an indexed range scan, not a skip-and-discard.
Cache or approximate the total count when an exact number genuinely must be shown — a periodically refreshed approximate count is often good enough for "about 4.3M results," and avoids computing it on every single request.

ScenarioSpring Data JPAPerformance

You need to rename a column a live table depends on, without taking the application down or breaking in-flight deploys. How do you do it safely?

Advanced📊 Mid

The danger is that old application code (still running during a rolling deploy) and new application code both need to work against the database at the same time — a migration that both renames a column and deploys code expecting the new name in one step will break whichever instances haven't updated yet. The standard technique is the expand-contract (parallel change) pattern, executed as several small, independently-deployable steps (using Flyway/Liquibase-versioned migrations):

🔀 Expand → migrate → contract

1. EXPAND:   add the new column alongside the old one (both exist, nothing reads new yet)
2. DUAL-WRITE: deploy code that writes to BOTH old and new columns
3. BACKFILL: migrate existing rows' data from old column into new column
4. CUTOVER: deploy code that READS from the new column (still writes both, for safety)
5. CONTRACT: once fully rolled out and stable, stop writing the old column,
             then drop it in a LATER migration

Every individual step is backward-compatible with the previous version of the code still running elsewhere in the cluster — which is exactly what makes it safe to deploy gradually without a maintenance window.

ScenarioFlywayDatabase Migration