Skip to main content
← PROBLEMS / DESIGN-PATTERN
An end-to-end domain learning journey

Design a Circuit
Breaker Pattern.

Design a resilient Circuit Breaker pattern to protect distributed microservices from cascading failures when downstream dependencies degrade or fail. The breaker must support Closed, Open, and Half-Open states with automatic recovery testing.
Start the journey5 stages
Design a Circuit Breaker Pattern architecture sketch diagram
FIRST PRINCIPLES. PRODUCTION SCALE.
The idea to keep with you

Design a resilient Circuit Breaker pattern to protect distributed microservices from cascading failures when downstream dependencies degrade or fail. The breaker must support Closed, Open, and Half-Open states with automatic recovery testing.

From patterns to decisions.

Design a resilient Circuit Breaker pattern to protect distributed microservices from cascading failures when downstream dependencies degrade or fail. The breaker must support Closed, Open, and Half-Open states with automatic recovery testing.

01
DESIGN-PATTERN · STAGE 01L4 · FUNDAMENTALS

When Service B slows down.

When Service B slows down, Service A's worker threads block waiting for responses. Soon Service A runs out of threads and fails to serve its own clients, cascading the failure through the entire system. Circuit Breakers fail fast immediately, preventing thread pool exhaustion.

ChoiceWhat it buysWhat it costs
Fail-Fast vs Persistent RetriesRetrying a struggling service creates a retry storm that worsens the outage. Circuit breaker stops calls completely to allow downstream recovery.
02
DESIGN-PATTERN · STAGE 02L5 · APPLIED DESIGN

State machine with Count-based or.

State machine with Count-based or Time-based sliding window metrics.

1. Closed

Calls pass to downstream. Record call outcomes (success, failure, timeout). If failureRate > threshold (e.g. 50% over last 100 calls), trip to Open.

2. Open

Reject incoming calls immediately with CircuitBreakerOpenException or execute fallback. Start waitDuration timer (e.g. 30 seconds).

3. Half-Open

When timer expires, enter Half-Open. Allow a limited number of test calls (e.g. 10 probe calls).

4. If probe failureRate < threshold

reset to Closed. If probe fails: return to Open for another waitDuration.

Alternative 1

Simple timeout + retry with exponential backoff: handles transient network blips, but does not stop cascading thread exhaustion.

Alternative 2

Rate limiting: limits request volume, but doesn't adapt dynamically to downstream health.

// 3-State Circuit Breaker (CLOSED, OPEN, HALF_OPEN)
export enum CircuitState { CLOSED, OPEN, HALF_OPEN }

export class CircuitBreaker {
  private state = CircuitState.CLOSED;
  private failureCount = 0;
  private nextAttempt = Date.now();

  constructor(
    private readonly failureThreshold: number = 5,
    private readonly timeoutMs: number = 10000
  ) {}

  public async execute<T>(fn: () => Promise<T>, fallback: () => T): Promise<T> {
    if (this.state === CircuitState.OPEN) {
      if (Date.now() > this.nextAttempt) {
        this.state = CircuitState.HALF_OPEN; // Test downstream health with probe request
      } else {
        return fallback(); // Fast-fail immediately without network call
      }
    }

    try {
      const result = await fn();
      if (this.state === CircuitState.HALF_OPEN) {
        this.state = CircuitState.CLOSED; // Downstream healed!
        this.failureCount = 0;
      }
      return result;
    } catch (err) {
      this.failureCount++;
      if (this.failureCount >= this.failureThreshold || this.state === CircuitState.HALF_OPEN) {
        this.state = CircuitState.OPEN; // Trip circuit
        this.nextAttempt = Date.now() + this.timeoutMs;
      }
      return fallback();
    }
  }
}
03
DESIGN-PATTERN · STAGE 03L6 · SCALE & TRADE-OFFS

Trade-Off Analysis.

Key architectural decisions and failure handling for Design a Circuit Breaker Pattern.

ChoiceWhat it buysWhat it costs
Count-based (last N calls)Good for predictable call volumes, but in low-traffic periods 5 failures over 2 days could trip the circuit.
Time-based (last T seconds)Reflects current operational health accurately, but requires minimum call threshold before evaluation to avoid tripping on a single failure.

What to Focus On

  • Primary Objective: Protect distributed microservices from cascading failures, thread pool exhaustion, and latency death spirals by detecting downstream service outages, failing fast without blocking caller threads, and safely orchestrating progressive recovery.
  • Key Decision Levers:
    1. Deployment Boundary: In-Process Circuit Breaker (Resilience4j / Polly) vs Service Mesh Sidecar (Envoy / Istio).
    2. Failure Metric Tracking: Sliding Time Window (e.g. last 10 seconds) vs Count-Based Window (e.g. last 100 requests).
    3. Recovery Verification: Rate-Limited Canary Probes in Half-Open State vs Periodic Polling.
    4. Degradation / Fallback Strategy: Return Stale Cached Data vs Fallback Default Value vs Immediate HTTP 503 Fail-Fast.
  • What You Should Be Able to Explain:
    • The Finite State Machine transitions between CLOSED, OPEN, and HALF-OPEN.
    • Why caller thread starvation (exhaustion of connection pools and worker threads) is more catastrophic than immediate errors.
    • Why 4xx client errors (e.g. 400 Bad Request, 404 Not Found) must NEVER trip a circuit breaker.
    • How exponential backoff with jitter during the OPEN cooldown prevents the thundering herd problem when the downstream service restarts.

Structured Trade-Off Matrix

Approach Optimizes For Downsides / Trade-offs When to Choose
In-Process Circuit Breaker (Recommended) Zero network hop latency; rich application fallback logic Configured per service; memory overhead of tracking state Microservice-to-microservice internal calls, rich fallbacks
Service Mesh Sidecar (Envoy) Language-agnostic, centralized policy enforcement Lacks application context; fallback options limited to status codes Polyglot microservices, infrastructure-managed resilience
Centralized Distributed Breaker (Redis) Global coordinated tripping across all 100 client pods Adds Redis as a network SPOF on the critical failure path Coordinated third-party API rate limit quota exhaustion
Count-Based vs Time-Based Window Count-based ensures statistical sample size In low traffic, count window takes too long to trip; time window can trip on 1 error Use Count-Based with minimum volume threshold

Hidden Assumptions & Common Traps

  • Hidden Assumption 1 — Minimum Request Volume: If a service receives 1 request every 5 minutes and it fails, failure rate is 100%100\%. Without a minimum request volume threshold (e.g. minimumNumberOfCalls = 20), the breaker will falsely trip on single fluke errors.
  • Hidden Assumption 2 — Timeout Precedence: A circuit breaker does NOT cancel slow requests on its own; it counts errors. A strict request timeout (e.g. 500 ms) MUST be configured alongside the circuit breaker, otherwise slow requests will saturate threads before the breaker trips.
  • Common Trap — Tripping on 4xx Errors: Treating HTTP 400 (Bad Request) or 401 (Unauthorized) as circuit failures causes a rogue client sending malformed requests to knock out the service for all legitimate users.

Example State Transition Trace & Metric Telemetry

{
  "circuitBreaker": "inventory-service-client",
  "state": "OPEN",
  "timestamp": "2026-03-01T08:00:00.100Z",
  "metrics": {
    "slidingWindowSize": 100,
    "totalCalls": 100,
    "failedCalls": 62,
    "failureRatePercent": 62.0,
    "failureThresholdPercent": 50.0,
    "slowCallsPercent": 14.0
  },
  "action": {
    "tripped": true,
    "cooldownRemainingMs": 9950,
    "fallbackStrategy": "SERVE_STALE_CACHE"
  }
}
04
DESIGN-PATTERN · STAGE 04L6 · SCALE & TRADE-OFFS

Failure Modes & Recovery.

Key architectural decisions and failure handling for Design a Circuit Breaker Pattern.

Distributed state (Redis)

High precision global state, but adds network latency to every call and creates a dependency on Redis. If Redis is down, circuit breakers fail!

Local in-memory state

Zero latency overhead, highly resilient. Downstream receives at most 500 * probe_count test calls, which is typically safe for downstream capacity.

05
DESIGN-PATTERN · STAGE 05L7 · STAFF / PRINCIPAL

Contextual fallback strategies.

Contextual fallback strategies.

1. Recommendations / Feeds

Return cached recommendations or default popular items. Degradation is invisible to user.

2. Search

Fall back to basic keyword search instead of vector neural search.

3. Payments / Checkout

NEVER fake a payment. Return clean error advising user to retry shortly, or queue transaction for asynchronous background processing.

Theory & Tips and Tricks

Practical heuristics & interview traps

  • 01Interview anchor: Isolate ingress routing, decouple transactional state from background operations, and enforce explicit timeout and retry budgets across all service boundaries.
  • 02Trade-off tip: Count-based (last N calls): Good for predictable call volumes, but in low-traffic periods 5 failures over 2 days could trip the circuit.
  • 03Trade-off tip: Time-based (last T seconds): Reflects current operational health accurately, but requires minimum call threshold before evaluation to avoid tripping on a single failure.
  • 04Watch out for: Distributed state (Redis): High precision global state, but adds network latency to every call and creates a dependency on Redis. If Redis is down, circuit breakers fail!
  • 05Watch out for: Local in-memory state: Zero latency overhead, highly resilient. Downstream receives at most 500 * probe_count test calls, which is typically safe for downstream capacity.
Behavioral Progression · Circuit Breaker Pattern

Circuit Breaker Pattern — Normal → Load / Fault → Recovery

Send Request
1. Normal Request Ingress
Traffic: 1x baseline · Latency (p99): 4 ms
Process State
2. Core Execution & State Lookup
Traffic: 1x baseline · Latency (p99): 9 ms
Inject 10x Spike
3. Scale Spike / Failure Injection
Traffic: 10x burst · Latency (p99): 185 ms

Isolate ingress routing, decouple transactional state from background operations, and enforce explicit timeout and retry budgets across all service boundaries.

Synthesis / One connected design

Putting Design a Circuit Breaker Pattern Together

Isolate ingress routing, decouple transactional state from background operations, and enforce explicit timeout and retry budgets across all service boundaries.

A candidate architecture, not a universal answer

Putting Design a Circuit Breaker Pattern Together

Step 1
Step 2
Step 3
Step 4
  1. 1. Closed: Calls pass to downstream. Record call outcomes (success, failure, timeout). If failureRate > threshold (e.g. 50% over last 100 calls), trip to Open.
  2. 2. Open: Reject incoming calls immediately with CircuitBreakerOpenException or execute fallback. Start waitDuration timer (e.g. 30 seconds).
  3. 3. Half-Open: When timer expires, enter Half-Open. Allow a limited number of test calls (e.g. 10 probe calls).
  4. 4. If probe failureRate < threshold: reset to Closed. If probe fails: return to Open for another waitDuration.
Resilient end-to-end architecture ready for production spikes.
Interview Readiness Check

Resilient end-to-end architecture ready for production spikes.

Walk through bottlenecks, back-of-the-envelope math, and failure recovery.

Practice / Explain the consequences

Make the trade-off yourself.

Practice Check

Test your understanding

Question 1 of 3

In a Circuit Breaker state machine (Closed, Open, Half-Open), what triggers the transition from OPEN back to HALF-OPEN?

Key takeaway

Design a Circuit Breaker Pattern

Isolate ingress routing, decouple transactional state from background operations, and enforce explicit timeout and retry budgets across all service boundaries.