Design a Circuit Breaker for Microservices
Design a resilient Circuit Breaker pattern to protect distributed microservices from cascading failures when downstream dependencies degrade or fail. The breaker must support Closed, Open, and Half-Open states with automatic recovery testing.
Requirements
- ✓Track failure rates and slow call rates over a sliding evaluation window.
- ✓Transition from Closed to Open when failure threshold is exceeded, failing fast immediately without calling downstream.
- ✓Transition from Open to Half-Open after a configured sleep window, allowing a trial percentage of probe calls.
- ✓Transition from Half-Open to Closed if probes succeed, or back to Open if probes fail.
- ✓Provide fallback responses when circuit is Open.
Constraints & Scale
- —Minimal overhead on the critical request path (< 0.1 ms latency added).
- —Thread-safe state transitions without blocking concurrent in-flight requests.
Understand
Guided mode
What is cascading failure in microservices? Why does an upstream service calling a slow downstream service exhaust thread pools?
Context & Constraints — Human Reviewed
When Service B slows down, Service A's worker threads block waiting for responses. Soon Service A runs out of threads and fails to serve its own clients, cascading the failure through the entire system. Circuit Breakers fail fast immediately, preventing thread pool exhaustion.
Trade-offs
- ⚖Fail-Fast vs Persistent Retries: Retrying a struggling service creates a retry storm that worsens the outage. Circuit breaker stops calls completely to allow downstream recovery.
Concepts used in this stage