SD System Design In Depth
0%
From-Scratch Build

Circuit Breaker & Adaptive Load Shedder

Three-state rolling-window circuit breaker (CLOSED, OPEN, HALF_OPEN) with single-probe half-open recovery, error-rate threshold tripping, and priority-aware Little’s Law concurrency load shedding.

~35 min Intermediate Lessons: Circuit breakers and timeouts, Retries, timeouts, and idempotency, Load shedding, Tail latency, Backpressure
Interactive Build Lab 0/4 Steps Completed

Code & Verify in the Browser Workbench

Complete 4 progressive engineering steps in JavaScript, TypeScript, or C++20 (WebAssembly) against an in-browser Node.js v22, Bun v1.1 & Clang/WASI runtime (virtual filesystem, loopback TCP/HTTP, and automated unit tests) — zero local setup required.

1. Architectural Overview & Why Naive Retries Cause Cascading Meltdowns

In a distributed microservices topology, when a downstream database or payment gateway slows down from 15ms to 2,500ms, upstream worker pools quickly saturate their thread/connection limits waiting on doomed sockets. Without a Circuit Breaker and Priority-Aware Load Shedder, every slow downstream dependency turns into a fleet-wide cascading outage.

stateDiagram-v2 [*] --> CLOSED CLOSED --> OPEN : Window Calls >= MinVolume AND Failure Rate >= Threshold OPEN --> HALF_OPEN : Cooldown Timer Elapsed (Single/Bounded Trial Probe) HALF_OPEN --> CLOSED : Consecutive Probe Successes >= Threshold HALF_OPEN --> OPEN : Any Probe Failure / Timeout

Core Engineering Invariants

  1. Minimum Volume Gate (minimumCalls): Never trip a circuit on 1 failure out of 1 request during low-traffic intervals. Require a statistically meaningful sample within the sliding time window (windowSizeMs).
  2. Fast-Fail Isolation in OPEN State: While open, reject calls in $O(1)$ memory without allocating sockets or timers, returning a degraded fallback or CircuitOpenError.
  3. Bounded HALF_OPEN Probing: When cooldownMs expires, prevent a thundering herd of 10,000 concurrent requests from hammering a recovering service. Admit at most halfOpenMaxCalls trial probes; if any probe fails, immediately re-open the circuit.
  4. Priority-Aware Concurrency Shedding (Little's Law $L = \lambda W$): Before the circuit even trips, bound in-flight concurrency (maxConcurrency). Once utilization crosses shedLowPriorityAt (e.g., 75%), shed LOW priority background/analytics requests so capacity remains reserved for HIGH/NORMAL user-facing traffic.

2. State Machine & Sliding Health Window Deep Dive

Rolling Time-Window Pruning

Instead of resetting counters on arbitrary fixed clock ticks (which suffers from boundary blind spots), _pruneWindow(now) evicts call outcomes older than now - windowSizeMs. Every recordSuccess() and recordFailure() evaluates live health across the trailing window.

Graceful Degradation & Fallback Hooks

When execute(fn, { priority, fallback }) rejects due to CIRCUIT_OPEN or ELOADSHED, callers can supply a deterministic fallback(err) handler—serving stale cache entries, queued acknowledgments, or default recommendations instead of HTTP 500s.

3. Verification & Running the Lab

Run the interactive demonstration:

bash
node demo.js

Run the automated assertion suite:

bash
node test.js