Ecommet
Topic 51 of 75

Circuit Breaker

A circuit breaker stops calling a dependency that keeps failing, so your service fails fast instead of waiting on timeouts, and the dependency gets room to recover. Learn the closed, open and half-open states and how to tune them.

Intermediate 5 min read 3 quiz questions

Jump to the quiz

In a nutshell

  • Closed: calls go through and failures are counted. Too many failures and the breaker opens.
  • Open: calls fail immediately (or use a fallback) without touching the dependency.
  • Half-open: after a cool-down, a few trial calls are let through; success closes it, failure opens it again.
  • Pair it with fallbacks: cached data, a default, a degraded feature, or a clear error.

Why fail fast

If the recommendations service is down and every product page waits 2 seconds for its timeout, pages become slow, threads pile up waiting, and the shop may fall over because of a non-essential feature. A circuit breaker notices the failures and, for a while, does not even try: the page renders instantly without recommendations.

Three states

Circuit breaker states

Circuit breaker states Closed → Open: too many failures Open → Half-open: cool-down over Half-open → Closed: trial succeeds Half-open → Open: trial fails Closedcalls passOpenfail fastHalf-opentrial calltoo many failurescool-down overtrial succeedstrial fails

All steps Closed counts failures; open fails fast; half-open tests whether the dependency is back.

Diagram as text
  1. While closed, calls go through. When failures pass a threshold, the breaker opens.
  2. While open, calls fail immediately. After the cool-down period it moves to half-open.
  3. In half-open, a trial call is allowed. Success closes the breaker; failure opens it again for another cool-down.

A circuit breaker (consecutive-failure threshold; time passed in for testing)

Java
import java.util.function.Supplier;

class CircuitBreaker {
    enum State { CLOSED, OPEN, HALF_OPEN }

    private final int failureThreshold;
    private final double coolDownSeconds;
    private State state = State.CLOSED;
    private int failures = 0;
    private double openedAt = 0;

    CircuitBreaker(int failureThreshold, double coolDownSeconds) {
        this.failureThreshold = failureThreshold;
        this.coolDownSeconds = coolDownSeconds;
    }

    synchronized State state(double now) {
        if (state == State.OPEN && now - openedAt >= coolDownSeconds) {
            state = State.HALF_OPEN;
        }
        return state;
    }

    // call returns null on failure; fallback is used when the call fails or is blocked.
    synchronized String call(Supplier<String> call, String fallback, double now) {
        if (state(now) == State.OPEN) {
            return fallback;                  // fail fast, dependency untouched
        }
        String result = call.get();
        if (result != null) {
            failures = 0;
            state = State.CLOSED;
            return result;
        }
        failures++;
        if (state == State.HALF_OPEN || failures >= failureThreshold) {
            state = State.OPEN;
            openedAt = now;
        }
        return fallback;
    }
}
Python
import threading

class CircuitBreaker:
    def __init__(self, failure_threshold, cool_down_seconds):
        self.threshold = failure_threshold
        self.cool_down = cool_down_seconds
        self._state = "closed"
        self.failures = 0
        self.opened_at = 0
        self.lock = threading.Lock()

    def state(self, now):
        if self._state == "open" and now - self.opened_at >= self.cool_down:
            self._state = "half-open"
        return self._state

    def call(self, fn, fallback, now):
        """fn returns None on failure; fallback is used when the call fails or is blocked."""
        with self.lock:
            if self.state(now) == "open":
                return fallback               # fail fast, dependency untouched
            result = fn()
            if result is not None:
                self.failures = 0
                self._state = "closed"
                return result
            self.failures += 1
            if self._state == "half-open" or self.failures >= self.threshold:
                self._state = "open"
                self.opened_at = now
            return fallback

Tuning and fallbacks

  • Failure rate over a window (for example 50% of the last 100 calls) is usually better than consecutive failures, and ignore windows with too few calls.
  • Count the right failures: timeouts and 5xx, not 404s or validation errors, which say nothing about the dependency's health.
  • One breaker per dependency (or per endpoint), so one bad endpoint does not block the rest.
  • Expose the state in metrics and alert when breakers open.

Common mistakes

  • Counting client errors (4xx) as failures and opening the breaker on bad input.
  • One global breaker for all dependencies.
  • Breakers with no monitoring, so nobody knows features are degraded.
  • No fallback, so an open breaker just produces errors faster.

Interview questions

Try answering out loud before you open each one.

The recommendation service is slow and product pages time out. What do you do?

Give the call a tight timeout and wrap it in a circuit breaker with a fallback of popular or cached recommendations. When failures cross a threshold, the breaker opens and pages render immediately without waiting. After a cool-down, trial calls test whether the service has recovered. Alert on the breaker state.

Check yourself

1. What happens to calls while the breaker is open?
Show the answer

Answer: They fail immediately or use a fallback. That is what protects both sides.

2. What is the half-open state for?
Show the answer

Answer: To allow trial calls and see if the dependency recovered. Success closes the breaker; failure reopens it.

3. Which errors should count as failures?
Show the answer

Answer: Timeouts and 5xx responses. They indicate an unhealthy dependency.

Further reading

Your notes

Saved in this browser as you type.

Finished this topic?

Mark it read to track your progress, then carry on with the next one.

Next topic