System Design Advanced: Security, Rate Limiting, and Reliability

Scenario context

Three reliability tools every backend system needs: Rate Limiting prevents API spam and DDoS, Circuit Breakers stop cascading failures when downstream services degrade, and Bulkheads isolate failure blast radius. Knowing when and how to combine them separates junior from senior system design.

Observable outcome: Each pool is bounded so it cannot consume resources beyond its allocation.

Stage 2 — Advanced Concepts: Combining Patterns for Defense in Depth

• Each pattern in isolation solves one failure mode.
• Together they form defense in depth — no single layer is the only thing standing between your system and failure.
• The retry storm problem illustrates why combining patterns matters.
• Add exponential backoff retries to every service call.

Observable outcome: Add exponential backoff retries to every service call.

Stage 3 — System Flow: Request Through the Reliability Stack

• Every inbound request traverses all three layers.
• The order matters : rate limit first, then bulkhead, then circuit breaker.
• At each gate, a rejection is cheap and deterministic.
• By the time a request reaches the downstream service it has already proven: within rate limits, a thread is available, and the target is believed healthy.

flowchart LR
    Client[Client Request] --> RL[Rate Limiter
Token Bucket]
    RL -->|429 if limit hit| Reject[Reject: 429]
    RL --> BH[Bulkhead
Thread Pool]
    BH -->|503 if pool full| Full[Reject: 503]
    BH --> CB[Circuit Breaker
CLOSED / OPEN / HALF-OPEN]
    CB -->|OPEN: fail fast| FB[Fallback Response]
    CB -->|CLOSED: call through| Svc[Downstream Service]
    Svc -->|success| CB
    Svc -->|failure threshold hit| CB

Observable outcome: By the time a request reaches the downstream service it has already proven: within rate limits, a thread is available, and the target is believed healthy.

Stage 4 — Real-World Applications: Where These Patterns Run in Production

• Rate Limiting in the wild: - GitHub API enforces 5,000 requests/hour per authenticated token.
• Unauthenticated requests are limited to 60/hour to force authentication.
• • Stripe limits to 100 requests/second per secret key.
• Exceeding this returns 429 with a Retry-After header; Stripe SDKs implement automatic exponential backoff.

Observable outcome: Exceeding this returns 429 with a Retry-After header; Stripe SDKs implement automatic exponential backoff.

Stage 5 — Practical: Resilience4j Configuration Approach

• Resilience4j provides composable annotations that layer circuit breaking, rate limiting, and bulkhead isolation directly on service methods.
• Each reliability pattern is configured independently by specifying key thresholds: the circuit breaker watches a sliding window of recent calls and trips when the failure rate exceeds a percentage (commonly 50%) across…
• The rate limiter is set with a call limit per refresh period — for example, 100 requests per second per service instance.
• The bulkhead caps the maximum number of concurrent calls and defines how long a request waits for a slot before being rejected.

Observable outcome: The bulkhead caps the maximum number of concurrent calls and defines how long a request waits for a slot before being rejected.

Stage 6 — Defense Stack: Authentication, Rate Limiting, and Reliability in Layers

• A complete defense stack for a production API service combines authentication, per-client rate limiting, bulkhead isolation, and circuit breaking into a sequential request pipeline.
• Each layer is independent and can fail fast without involving the layers that follow it.
• An incoming request presents a JWT or API key, which is validated against a signing secret or public key.
• If authentication fails, the request is rejected immediately with a 401 Unauthorized response before any business logic is executed.

Observable outcome: If authentication fails, the request is rejected immediately with a 401 Unauthorized response before any business logic is executed.

End-to-end outcome

Rate limiting is enforced at the API Gateway or reverse proxy layer before requests reach your application.

Alternatives considered

Decision Guide: Which Pattern for Which Problem

Non-obvious edge cases: - Set circuit breaker recovery timeout <= your SLA budget or users wait the full timeout before recovery is even attempted.

Failure and degraded modes

Trade-offs & Failure Modes: Trade-offs and Failure Modes: Choosing the Right Pattern

• Detection: Trade-offs & Failure Modes: Trade-offs and Failure Modes: Choosing the Right Pattern
• User or operator impact: Trade-offs & Failure Modes: Trade-offs and Failure Modes: Choosing the Right Pattern

Key Lessons from Reliability Pattern Failures in Production

• Detection: - Rate limiting without a Retry-After header breaks clients — they cannot back off intelligently without knowing when to retry.
• User or operator impact: - Rate limiting without a Retry-After header breaks clients — they cannot back off intelligently without knowing when to retry.