Design a resilient Circuit Breaker pattern to protect distributed microservices from cascading failures when downstream dependencies degrade or fail. The breaker must support Closed, Open, and Half-Open states with automatic recovery testing.
From patterns to decisions.
Design a resilient Circuit Breaker pattern to protect distributed microservices from cascading failures when downstream dependencies degrade or fail. The breaker must support Closed, Open, and Half-Open states with automatic recovery testing.
When Service B slows down.
When Service B slows down, Service A's worker threads block waiting for responses. Soon Service A runs out of threads and fails to serve its own clients, cascading the failure through the entire system. Circuit Breakers fail fast immediately, preventing thread pool exhaustion.
| Choice | What it buys | What it costs |
|---|---|---|
| Fail-Fast vs Persistent Retries | Retrying a struggling service creates a retry storm that worsens the outage. Circuit breaker stops calls completely to allow downstream recovery. |
State machine with Count-based or.
State machine with Count-based or Time-based sliding window metrics.
1. Closed
Calls pass to downstream. Record call outcomes (success, failure, timeout). If failureRate > threshold (e.g. 50% over last 100 calls), trip to Open.
2. Open
Reject incoming calls immediately with CircuitBreakerOpenException or execute fallback. Start waitDuration timer (e.g. 30 seconds).
3. Half-Open
When timer expires, enter Half-Open. Allow a limited number of test calls (e.g. 10 probe calls).
4. If probe failureRate < threshold
reset to Closed. If probe fails: return to Open for another waitDuration.
Alternative 1
Simple timeout + retry with exponential backoff: handles transient network blips, but does not stop cascading thread exhaustion.
Alternative 2
Rate limiting: limits request volume, but doesn't adapt dynamically to downstream health.
// 3-State Circuit Breaker (CLOSED, OPEN, HALF_OPEN)
export enum CircuitState { CLOSED, OPEN, HALF_OPEN }
export class CircuitBreaker {
private state = CircuitState.CLOSED;
private failureCount = 0;
private nextAttempt = Date.now();
constructor(
private readonly failureThreshold: number = 5,
private readonly timeoutMs: number = 10000
) {}
public async execute<T>(fn: () => Promise<T>, fallback: () => T): Promise<T> {
if (this.state === CircuitState.OPEN) {
if (Date.now() > this.nextAttempt) {
this.state = CircuitState.HALF_OPEN; // Test downstream health with probe request
} else {
return fallback(); // Fast-fail immediately without network call
}
}
try {
const result = await fn();
if (this.state === CircuitState.HALF_OPEN) {
this.state = CircuitState.CLOSED; // Downstream healed!
this.failureCount = 0;
}
return result;
} catch (err) {
this.failureCount++;
if (this.failureCount >= this.failureThreshold || this.state === CircuitState.HALF_OPEN) {
this.state = CircuitState.OPEN; // Trip circuit
this.nextAttempt = Date.now() + this.timeoutMs;
}
return fallback();
}
}
}
Trade-Off Analysis.
Key architectural decisions and failure handling for Design a Circuit Breaker Pattern.
| Choice | What it buys | What it costs |
|---|---|---|
| Count-based (last N calls) | Good for predictable call volumes, but in low-traffic periods 5 failures over 2 days could trip the circuit. | |
| Time-based (last T seconds) | Reflects current operational health accurately, but requires minimum call threshold before evaluation to avoid tripping on a single failure. |
What to Focus On
- Primary Objective: Protect distributed microservices from cascading failures, thread pool exhaustion, and latency death spirals by detecting downstream service outages, failing fast without blocking caller threads, and safely orchestrating progressive recovery.
- Key Decision Levers:
- Deployment Boundary: In-Process Circuit Breaker (Resilience4j / Polly) vs Service Mesh Sidecar (Envoy / Istio).
- Failure Metric Tracking: Sliding Time Window (e.g. last 10 seconds) vs Count-Based Window (e.g. last 100 requests).
- Recovery Verification: Rate-Limited Canary Probes in Half-Open State vs Periodic Polling.
- Degradation / Fallback Strategy: Return Stale Cached Data vs Fallback Default Value vs Immediate HTTP 503 Fail-Fast.
- What You Should Be Able to Explain:
- The Finite State Machine transitions between CLOSED, OPEN, and HALF-OPEN.
- Why caller thread starvation (exhaustion of connection pools and worker threads) is more catastrophic than immediate errors.
- Why 4xx client errors (e.g. 400 Bad Request, 404 Not Found) must NEVER trip a circuit breaker.
- How exponential backoff with jitter during the OPEN cooldown prevents the thundering herd problem when the downstream service restarts.
Structured Trade-Off Matrix
| Approach | Optimizes For | Downsides / Trade-offs | When to Choose |
|---|---|---|---|
| In-Process Circuit Breaker (Recommended) | Zero network hop latency; rich application fallback logic | Configured per service; memory overhead of tracking state | Microservice-to-microservice internal calls, rich fallbacks |
| Service Mesh Sidecar (Envoy) | Language-agnostic, centralized policy enforcement | Lacks application context; fallback options limited to status codes | Polyglot microservices, infrastructure-managed resilience |
| Centralized Distributed Breaker (Redis) | Global coordinated tripping across all 100 client pods | Adds Redis as a network SPOF on the critical failure path | Coordinated third-party API rate limit quota exhaustion |
| Count-Based vs Time-Based Window | Count-based ensures statistical sample size | In low traffic, count window takes too long to trip; time window can trip on 1 error | Use Count-Based with minimum volume threshold |
Hidden Assumptions & Common Traps
- Hidden Assumption 1 — Minimum Request Volume: If a service receives 1 request every 5 minutes and it fails, failure rate is . Without a minimum request volume threshold (e.g.
minimumNumberOfCalls = 20), the breaker will falsely trip on single fluke errors. - Hidden Assumption 2 — Timeout Precedence: A circuit breaker does NOT cancel slow requests on its own; it counts errors. A strict request timeout (e.g. 500 ms) MUST be configured alongside the circuit breaker, otherwise slow requests will saturate threads before the breaker trips.
- Common Trap — Tripping on 4xx Errors: Treating HTTP 400 (Bad Request) or 401 (Unauthorized) as circuit failures causes a rogue client sending malformed requests to knock out the service for all legitimate users.
Example State Transition Trace & Metric Telemetry
{
"circuitBreaker": "inventory-service-client",
"state": "OPEN",
"timestamp": "2026-03-01T08:00:00.100Z",
"metrics": {
"slidingWindowSize": 100,
"totalCalls": 100,
"failedCalls": 62,
"failureRatePercent": 62.0,
"failureThresholdPercent": 50.0,
"slowCallsPercent": 14.0
},
"action": {
"tripped": true,
"cooldownRemainingMs": 9950,
"fallbackStrategy": "SERVE_STALE_CACHE"
}
}
Failure Modes & Recovery.
Key architectural decisions and failure handling for Design a Circuit Breaker Pattern.
Distributed state (Redis)
High precision global state, but adds network latency to every call and creates a dependency on Redis. If Redis is down, circuit breakers fail!
Local in-memory state
Zero latency overhead, highly resilient. Downstream receives at most 500 * probe_count test calls, which is typically safe for downstream capacity.
Contextual fallback strategies.
Contextual fallback strategies.
1. Recommendations / Feeds
Return cached recommendations or default popular items. Degradation is invisible to user.
2. Search
Fall back to basic keyword search instead of vector neural search.
3. Payments / Checkout
NEVER fake a payment. Return clean error advising user to retry shortly, or queue transaction for asynchronous background processing.
Theory & Tips and Tricks
Practical heuristics & interview traps
- 01Interview anchor: Isolate ingress routing, decouple transactional state from background operations, and enforce explicit timeout and retry budgets across all service boundaries.
- 02Trade-off tip: Count-based (last N calls): Good for predictable call volumes, but in low-traffic periods 5 failures over 2 days could trip the circuit.
- 03Trade-off tip: Time-based (last T seconds): Reflects current operational health accurately, but requires minimum call threshold before evaluation to avoid tripping on a single failure.
- 04Watch out for: Distributed state (Redis): High precision global state, but adds network latency to every call and creates a dependency on Redis. If Redis is down, circuit breakers fail!
- 05Watch out for: Local in-memory state: Zero latency overhead, highly resilient. Downstream receives at most 500 * probe_count test calls, which is typically safe for downstream capacity.
Circuit Breaker Pattern — Normal → Load / Fault → Recovery
Isolate ingress routing, decouple transactional state from background operations, and enforce explicit timeout and retry budgets across all service boundaries.
Putting Design a Circuit Breaker Pattern Together
Isolate ingress routing, decouple transactional state from background operations, and enforce explicit timeout and retry budgets across all service boundaries.
Putting Design a Circuit Breaker Pattern Together
- 1. Closed: Calls pass to downstream. Record call outcomes (success, failure, timeout). If failureRate > threshold (e.g. 50% over last 100 calls), trip to Open.
- 2. Open: Reject incoming calls immediately with CircuitBreakerOpenException or execute fallback. Start waitDuration timer (e.g. 30 seconds).
- 3. Half-Open: When timer expires, enter Half-Open. Allow a limited number of test calls (e.g. 10 probe calls).
- 4. If probe failureRate < threshold: reset to Closed. If probe fails: return to Open for another waitDuration.
Interview Readiness Check
Resilient end-to-end architecture ready for production spikes.
Walk through bottlenecks, back-of-the-envelope math, and failure recovery.
Make the trade-off yourself.
Practice Check
Test your understanding
In a Circuit Breaker state machine (Closed, Open, Half-Open), what triggers the transition from OPEN back to HALF-OPEN?
Design a Circuit Breaker Pattern
Isolate ingress routing, decouple transactional state from background operations, and enforce explicit timeout and retry budgets across all service boundaries.
