Circuit Breaker Pattern
Circuit Breaker Pattern is the software resilience pattern that prevents cascading failures by automatically stopping requests to failing services — allowing downstream services time to recover while protecting calling services from thread pool exhaustion, timeout accumulation, and resource depletion that would otherwise propagate a single service failure into a system-wide outage across an entire microservices architecture.
What Is the Circuit Breaker Pattern?
- Definition: A design pattern that wraps calls to external services in a stateful proxy that monitors failures and automatically short-circuits requests when a failure threshold is exceeded.
- Analogy: Works like an electrical circuit breaker — when current (failures) exceeds safe limits, the breaker trips to protect the system.
- Origin: Popularized by Michael Nygard in "Release It!" and implemented in Netflix's Hystrix library.
- Core Value: Provides fail-fast behavior that preserves system resources and enables automatic recovery detection.
The Three States
- Closed (Normal Operation): Requests pass through to the downstream service normally. The breaker monitors failure rates and counts consecutive failures.
- Open (Service Failing): After failures exceed the threshold, the breaker opens. All requests fail immediately without calling the downstream service, returning a fallback response or error instantly.
- Half-Open (Testing Recovery): After a configured timeout period, the breaker allows a limited number of test requests through. If tests succeed, the breaker closes (recovery confirmed). If tests fail, it reopens.
Why Circuit Breakers Matter
- Prevent Cascading Failures: One failing service can exhaust connection pools and threads in every service that calls it, cascading across the system.
- Reduce Latency During Failures: Instead of waiting 30 seconds for a timeout, requests fail in milliseconds when the breaker is open.
- Protect Resources: Thread pools, database connections, and memory are preserved for healthy request paths.
- Enable Graceful Degradation: Open breakers trigger fallback logic that provides reduced but functional service.
- Automatic Recovery: The half-open state automatically detects when failed services recover, restoring normal operation without manual intervention.
Configuration Parameters
| Parameter | Description | Typical Value |
|---|---|---|
| Failure Threshold | Number or percentage of failures to trip breaker | 5 consecutive or 50% in window |
| Timeout Period | How long breaker stays open before testing | 30-60 seconds |
| Half-Open Limit | Number of test requests in half-open state | 1-3 requests |
| Monitoring Window | Time window for counting failures | 10-60 seconds |
| Success Threshold | Successes needed in half-open to close | 3-5 consecutive |
Implementation in ML Systems
- Model Serving: Breakers on inference endpoints prevent one slow model from blocking request threads.
- Feature Stores: Breakers on feature retrieval trigger cached or default feature fallbacks.
- External APIs: Breakers on third-party API calls (enrichment, validation) protect core prediction paths.
- Data Pipelines: Breakers on upstream data sources prevent pipeline stalls from propagating downstream.
Popular Implementations
- Resilience4j: Modern Java circuit breaker library replacing Netflix Hystrix.
- Polly (.NET): Circuit breaker and resilience library for .NET applications.
- Istio/Envoy: Service mesh-level circuit breaking without application code changes.
- Python: tenacity, pybreaker, and custom implementations with decorators.
Circuit Breaker Pattern is the essential resilience primitive for distributed systems — providing automatic failure isolation and recovery detection that prevents individual service failures from cascading into system-wide outages, making it a mandatory component of any production microservices architecture.