Performance & Optimization

Scalability & Resilience Patterns

Scaling past one JVM is an architecture problem: statelessness enables horizontal scale, backpressure prevents overload collapse, and bulkheads/circuit breakers/timeouts stop one failing dependency from taking the fleet down.

Stateless services scale by replication; push state to stores designed for it
Every remote call gets a timeout, a retry budget (with jittered backoff), and a fallback decision
Retries without idempotency duplicate effects; retries without backoff are a self-inflicted DDoS
Backpressure: bounded queues + load shedding beat unbounded buffering, always
Bulkheads isolate resource pools per dependency; circuit breakers fail fast when a dependency is down
Cache with a policy: TTL + bounded size + measured hit rate — or it's a leak with good PR

The cascade anatomy (OCNJ ch. 14, Java Secrets' fault-tolerance chapters): dependency B slows → A's threads pile up waiting on B → A's pool exhausts → A's callers queue → memory grows, GC thrashes, everything times out at once. Each defense targets a link: timeouts cap the wait, bulkheads cap the threads any one dependency can hold hostage, circuit breakers stop sending after failures cross a threshold (probing periodically for recovery), load shedding rejects work beyond capacity early, when rejection is still cheap.

The resilient call, assembled

// Resilience4j-style composition around a remote call:
CircuitBreaker breaker = CircuitBreaker.ofDefaults("inventory");
Retry retry = Retry.of("inventory", RetryConfig.custom()
        .maxAttempts(3)
        .intervalFunction(IntervalFunction.ofExponentialRandomBackoff(100, 2.0))
        .retryExceptions(TimeoutException.class)      // retry ONLY idempotent, transient failures
        .build());
Bulkhead bulkhead = Bulkhead.of("inventory", BulkheadConfig.custom()
        .maxConcurrentCalls(20).build());

Supplier<Stock> call = () -> client.stock(sku);       // client itself has a 500 ms timeout
Stock stock = Decorators.ofSupplier(call)
        .withBulkhead(bulkhead).withCircuitBreaker(breaker).withRetry(retry)
        .withFallback(List.of(CallNotPermittedException.class), e -> Stock.UNKNOWN)
        .get();

Backpressure is the system-level version of the bounded BlockingQueue rule: somewhere, a limit must exist; the only choice is whether it fails explicitly (429/503, caller-runs, shed) or implicitly (OOM, 30-second GC pause, cascading timeout storm). Unbounded thread pools, unbounded queues, and unlimited retries all "work" until the day they define your outage.