Advanced Data Structures & Algorithms

Java Backend Modernization

Beginner's Guide to Data Structures

Agentic AI Systems

Data Engineering & Spark Mastery

Low-Level Design Interview Prep

Microservices Interview Roadmap

Design Patterns Interview Roadmap

Distributed Systems Interview Roadmap

Mastering Python Development

System Design Interview Prep

Distributed Systems Refresher

AI Engineering Launchpad

Architecture Patterns for Scalable Systems

RAG & Vector Search Mastery

Production DevOps & Reliability

Enterprise Patterns Playbook

Master Coding Interviews

Coding Techniques Interview Roadmap

Data Structures Interview Roadmap

Architecture Patterns Interview Roadmap

software-engineering-principles

Migrate your Spring Boot services from thread-pool exhaustion to JDK 21 virtual threads — the practical Spring developer guide

Adapting to Virtual Threads for Spring Developers

how-it-works-internals-explained

From WAL and MVCC to fsync and isolation levels — the internals behind every ACID guarantee in PostgreSQL, MySQL, Oracle, and SQL Server.

ACID Properties Explained: How SQL Databases Guarantee Atomicity, Consistency, Isolation, and Durability

How modern NoSQL databases achieve atomicity and isolation without a single machine

ACID Transactions in Distributed Databases: DynamoDB, Cosmos DB, and Spanner Compared

llm-engineering

Is an LLM a brain in a jar? To make it truly useful, we need to give it access to the world. This guide explains RAG and Agents.

Advanced AI: Agents, RAG, and the Future of Intelligence

Production AI needs explicit routing, memory, execution, and evaluation layers rather than one loop.

AI Architecture Patterns: Routers, Planner-Worker Loops, Memory Layers, and Evaluation Guardrails

datastructures-and-algorithms

Master shortest paths, dependency resolution, and cycle detection with robust Java solutions.

Advanced Graph Algorithms: Dijkstra, Topological Sort, and Disjoint Set Union (DSU)

An LLM can talk, but an AI Agent can *act*. We explain how Agents use the ReAct framework to brow...

AI Agents Explained: When LLMs Start Using Tools

A production-first guide to picking the right vector index for recall, latency, memory, and update patterns.

ANN Index Types Explained: When to Choose Flat, HNSW, IVF, or IVF-PQ

apache-spark-engineering

Process petabytes with Python: understand Spark's execution model, DataFrame transformations, partitioning strategy, and real-time streaming.

Apache Spark for Data Engineers: RDDs, DataFrames, and Structured Streaming

ACID guarantees consistency. BASE guarantees availability. We explain the trade-offs between SQL...

BASE Theorem Explained: How it Stands Against ACID

machine-learning-fundamentals

From the query-key-value formulation to multi-head attention — the mechanism that made transformers dominate AI

Attention Mechanism Explained: How Transformers Learn to Focus

system-design-interview-prep

They all sit in front of your servers. But do you need Nginx, HAProxy, or Kong? We clarify the roles of each component.

API Gateway vs. Load Balancer vs. Reverse Proxy: What's the Difference?

architecture-patterns-for-production-systems

Mobile apps need less data than Desktop dashboards. Why serve them the same JSON? The Backend for Frontend (BFF) pattern solves this.

Backend for Frontend (BFF): Tailoring APIs for UI

BFS explores graphs level by level using a queue — the key to shortest paths and level-order problems.

BFS — Breadth-First Search: Level-by-Level Graph Exploration

Backtracking is a brute-force algorithmic technique that solves problems by trying to build a sol...

Exploring Backtracking Techniques in Data Structures

big-data-engineering

Choose ingestion, serving, and ownership patterns deliberately when data platforms start to scale.

Big Data Architecture Patterns: Lambda, Kappa, CDC, Medallion, and Data Mesh

Why traditional databases fail at scale, what the 5 Vs really mean, and a practical map of the big data ecosystem from ingestion to insights.

Big Data 101: The 5 Vs, Ecosystem, and Why Scale Breaks Everything

Not all Binary Trees are created equal. We explain Full, Complete, Perfect, and Balanced Binary T...

Exploring Different Types of Binary Trees

Master Big O notation: measure algorithm efficiency from O(1) to O(n²) with growth diagrams, worked examples, and the interview mental model.

Big O Notation Explained: Time Complexity, Space Complexity, and Why They Matter in Interviews

Binary search has five distinct patterns — classic, leftmost, rightmost, rotated arrays, and 2D matrices.

Binary Search Patterns: Five Variants Every Senior Engineer Knows

Don't panic. System Design interviews are open-ended discussions. This framework (Requirements, API, DB, Scale) will help you structure your answer.

The Ultimate Guide to Acing the System Design Interview

A beginner-friendly way to structure requirements, estimates, diagrams, and trade-offs in your first design round.

System Design Interview Basics: A Beginner-Friendly Framework for Clear Answers

A practical framework for clarifying functional scope, non-functional targets, and trade-off boundaries in interviews.

System Design Requirements and Constraints: Ask Better Questions Before You Draw

Don't guess. Calculate. We explain how to estimate QPS, Storage, and Bandwidth for your system de...

The Role of Data in Precise Capacity Estimations for System Design

Design APIs interviewers trust by focusing on resource boundaries, request contracts, and failure-safe behavior.

System Design API Design for Interviews: Contracts, Idempotency, and Pagination

The eternal debate: SQL or NoSQL? We break down ACID vs BASE, Sharding vs Replication, and when to use MongoDB vs PostgreSQL.

System Design Databases: SQL vs NoSQL and Scaling

Cache-Aside to Refresh-Ahead, LRU to ARC — every caching pattern, eviction policy, and distributed strategy you need for system design

System Design: Complete Guide to Caching — Patterns, Eviction, and Distributed Strategies

A practical guide to shard keys, routing, hot partitions, and how to scale storage without breaking queries.

System Design Sharding Strategy: Choosing Keys, Avoiding Hot Spots, and Resharding Safely

How to choose the right uniqueness strategy — from 6-char short URLs to distributed int64 order IDs

ID Generation Strategies in System Design: Base62, UUID, Snowflake, and Beyond

The celebrity threshold, Redis sorted sets, and the hybrid model every social platform actually ships.

Write-Time vs Read-Time Fan-Out: How Social Feeds Scale

The internet's traffic control system. We explain how DNS resolves names, CDNs cache content, and Load Balancers distribute traffic.

System Design Networking: DNS, CDNs, and Load Balancers

Design asynchronous pipelines with queues, retries, and consumer scaling that survive traffic spikes.

System Design Message Queues and Event-Driven Architecture: Building Reliable Asynchronous Systems

Learn how clients find services safely with registries, heartbeats, and health-aware load balancing.

System Design Service Discovery and Health Checks: Routing Traffic to Healthy Instances

A clear guide to replicas, failover detection, replica lag, and the trade-offs behind reliable database designs.

System Design Replication and Failover: Keep Services Alive When a Primary Dies

Design telemetry, SLOs, and response playbooks that detect failure early and recover predictably.

System Design Observability, SLOs, and Incident Response: Operating Systems You Can Trust

A senior-level HLD for a high-throughput short-link platform with 100:1 read-write ratio.

System Design HLD Example: URL Shortener (TinyURL and Bitly)

Sub-10ms prefix lookup, ranking, typo tolerance, and language-partitioned indexes at Google scale.

System Design HLD Example: Search Autocomplete (Google/Amazon)

Interview HLD for a distributed cache with eviction, invalidation, and resilience trade-offs.

System Design HLD Example: Distributed Cache Platform

Interview-focused HLD for a scalable social feed with fan-out and ranking trade-offs.

System Design HLD Example: News Feed (Home Timeline)

An interview-ready HLD for payments focusing on correctness, idempotency, and recovery.

System Design HLD Example: Payment Processing Platform

## Scenario context

System Design interviews are collaborative whiteboard sessions, not trick-question coding tests. Follow the framework — Requirements → Estimations → API → Data Model → High-Level Architecture → Deep-Dive — and you turn vague product ideas into defensible, production-ready designs.

---



```architecture
{
  "version": 1,
  "title": "TLDR:** System Design interviews are collaborative whiteboard sessions, not trick-q…

This walkthrough focuses on **Why System Design Interviews Feel Chaotic (and How to Fix That)**.

Before making a design choice, clarify:

- Picture two candidates who received the same prompt: "Design a real-time activity feed for 500,000 events per second." Candidate A: "I'd store events in a database and add a cache layer in front of it." Candidate B: "I'…
- I'd put Redis sorted sets in front as a fan-out cache for the hot feed.
- I'd choose hybrid fan-out: push to regular users, pull for accounts with over a million followers to avoid write amplification.

**Concepts in play:** system-design-hld-architectures, caching, consistency-replication, event-driven-architecture

## Stage 1 — What Makes a Strong System Design Response

- Before diving into the framework, understand what separates a strong response from a weak one.
- Weak responses jump straight to a specific technology ("I'd use Kafka and Cassandra") before understanding requirements.
- They treat the problem as a trivia question with a known answer.
- Strong responses start by asking clarifying questions, establish constraints with back-of-envelope estimates, and justify every decision with explicit trade-offs.

*Applies **system-design-hld-architectures** to the decision above.*

**Observable outcome:** Strong responses start by asking clarifying questions, establish constraints with back-of-envelope estimates, and justify every decision with explicit trade-offs.

## Stage 2 — The Six-Step Interview Framework

- A repeatable structure turns chaos into confidence.
- Step 1: Clarify Requirements (~5 min) Ask before you design.
- Split into: - Functional requirements — What does the system need to do?
- ("Post a tweet", "Follow a user", "Generate a timeline") - Non-functional requirements — How should it behave?

*Applies **caching** to the decision above.*

```mermaid
flowchart TD
    subgraph Client
        C[Mobile / Web App]
    end
    C -->|HTTPS| LB[Load Balancer]
    LB --> GW[API Gateway - auth, rate-limit]
    GW --> TS[Tweet Service]
    GW --> US[User Service]
    GW --> FS[Feed Service]
    TS -->|Write| CASS[Cassandra Cluster]
    TS -->|Event| KAF[Kafka]
    KAF --> FAN[Fan-out Workers]
    FAN --> RED[Redis Feed Cache]
    US --> PG[PostgreSQL]
    FS --> RED

```

```mermaid
flowchart TD
    A[Receive Prompt]
    B[Clarify Requirements]
    C[Estimate Capacity]
    D[Define API]
    E[Choose Data Models]
    F[High-Level Architecture]
    G{Interviewer probes?}
    H[Deep-Dive: Scaling]
    I[Raise hardest problem]
    J[Summarise trade-offs]
    K[Invite feedback]
    
    A --> B
    B --> C
    C --> D
    D --> E
    E --> F
    F --> G
    G -->|Yes| H
    G -->|No| I
    H --> J
    I --> J
    J --> K
```

**Observable outcome:** ("Post a tweet", "Follow a user", "Generate a timeline") - Non-functional requirements — How should it behave?

## Stage 3 — Key Mechanics: Caching, Fan-out, and Hot Users

- Caching the feed Each user's feed is stored in Redis as a sorted set, where the tweet ID is the member and the tweet's creation timestamp is the score.
- This structure makes inserting a new tweet O(log N) and retrieving the most recent top-N tweets equally fast — both well within the sub-10 ms latency budget of a production feed service.
- The score-based ordering also enables cursor-based pagination naturally: slide the score window backward in time to fetch the next page without offset scanning.
- Fan-out strategies Hot user problem: A celebrity with 50 M followers cannot push-fan-out on write without overloading the system.

*Applies **consistency-replication** to the decision above.*

```architecture
{
  "version": 1,
  "title": "Key Mechanics: Caching, Fan-out, and Hot Users - Storage Topology",
  "description": "Multi-tier caching and replicated storage pipeline ensuring fast reads and transactional consistency.",
  "view": "data-flow",
  "pattern": "cache-aside",
  "direction": "LR",
  "groups": [
    { "id": "compute", "label": "Application Workers", "tone": "blue" },
    { "id": "cache-tier", "label": "Volatile Memory Tier", "tone": "amber" },
    { "id": "storage-tier", "label": "Persistent Disk Tier", "tone": "green" }
  ],
  "nodes": [
    { "id": "worker", "component": "workflow-service", "label": "Worker Process", "group": "compute" },
    { "id": "mem", "component": "redis", "label": "In-Memory Cache", "group": "cache-tier", "badge": "Hot Keys" },
    { "id": "primary", "component": "postgres", "label": "Primary Database", "group": "storage-tier", "badge": "Write Master" }
  ],
  "edges": [
    { "from": "worker", "to": "mem", "kind": "read", "label": "Fast Read" },
    { "from": "worker", "to": "primary", "kind": "write", "label": "Transactional Commit" }
  ],
  "takeaway": {
    "title": "Storage Access Pattern",
    "body": "In-memory caching handles high-frequency read spikes, reserving disk transactions strictly for state modifications."
  }
}
```

**Observable outcome:** Fan-out strategies Hot user problem: A celebrity with 50 M followers cannot push-fan-out on write without overloading the system.

## Stage 4 — Deep Dive: How Interviewers Score System Design Answers

- Most interviewers use a rubric with four dimensions: requirements clarity (did you ask the right questions?), estimation accuracy (are numbers in the right order of magnitude?), architectural completeness (does the desi…

*Applies **event-driven-architecture** to the decision above.*

**Observable outcome:** Most interviewers use a rubric with four dimensions: requirements clarity (did you ask the right questions?), estimation accuracy (are numbers in the right order of magnitude?), architectural completeness (does the desi…

## Stage 5 — Framework Flow: From Prompt to Architecture

- This flowchart shows the recommended decision sequence for a 45-minute system design session.
- The loop at step G is critical: interviewers probe whatever they find most interesting or whatever you mentioned confidently.
- Prepare to defend every component you sketch.

*Applies **system-design-hld-architectures** to the decision above.*

```architecture
{
  "version": 1,
  "title": "Framework Flow: From Prompt to Architecture - Subsystem Overview",
  "description": "Decoupled system architecture isolating ingress, stateless logic, and persistent datastores.",
  "view": "high-level",
  "pattern": "microservices",
  "direction": "LR",
  "groups": [
    { "id": "edge", "label": "Edge Traffic Ingress", "tone": "slate" },
    { "id": "services", "label": "Stateless Microservices", "tone": "blue" },
    { "id": "data", "label": "Datastore Infrastructure", "tone": "green" }
  ],
  "nodes": [
    { "id": "gw", "component": "api-gateway", "label": "API Gateway", "group": "edge", "badge": "TLS & Auth" },
    { "id": "app", "component": "workflow-service", "label": "Domain Service", "group": "services", "badge": "Stateless" },
    { "id": "cache", "component": "redis", "label": "Redis Cache", "group": "data", "badge": "Sub-ms Reads" },
    { "id": "db", "component": "postgres", "label": "Transactional DB", "group": "data", "badge": "ACID Log" }
  ],
  "edges": [
    { "from": "gw", "to": "app", "kind": "request", "label": "gRPC Request" },
    { "from": "app", "to": "cache", "kind": "read", "label": "Cache Lookups" },
    { "from": "app", "to": "db", "kind": "write", "label": "Durable Write" }
  ],
  "takeaway": {
    "title": "Architecture Boundary Invariant",
    "body": "Stateless compute nodes scale horizontally while shielding underlying database clusters behind fast cache layers."
  }
}
```

```mermaid
flowchart TD
    A[Interview Prompt Received]
    B[Clarify Requirements]
    C[Estimate Capacity]
    D[Define API]
    E[Choose Data Models]
    F[Sketch Architecture]
    G{Interviewer probes?}
    H[Deep-Dive: Scaling]
    I[Raise hardest problem]
    J[Summarise trade-offs]
    
    A --> B
    B --> C
    C --> D
    D --> E
    E --> F
    F --> G
    G -->|Yes| H
    G -->|No| I
    H --> J
    I --> J
```

**Observable outcome:** Prepare to defend every component you sketch.

## Stage 6 — Real-World Applications: What Interviewers Are Actually Evaluating

- You are not graded on getting a "correct" architecture.
- (Not skipping this is a differentiator) 2.
- Did you estimate before you designed?
- (Shows engineering maturity) 3.

*Applies **caching** to the decision above.*

**Observable outcome:** (Shows engineering maturity) 3.

## End-to-end outcome

Picture two candidates who received the same prompt: "Design a real-time activity feed for 500,000 events per second." Candidate A: "I'd store events in a database and add a cache layer in front of it." Candidate B: "I'd use Cassandra with a write-heavy schema partitioned by user id and event timestamp — because this is append-only click data at 500K events/sec, and Cassandra's LSM-tree storage handles that write pattern without contention.

## Alternatives considered

### Decision Guide: Common Interview Scenarios

Decision Guide: Common Interview Scenarios


## Failure and degraded modes

### Trade-offs & Failure Modes: Common Failure Modes and How to Discuss Them

- **Detection:** Trade-offs & Failure Modes: Common Failure Modes and How to Discuss Them
- **User or operator impact:** Trade-offs & Failure Modes: Common Failure Modes and How to Discuss Them



## Scenario context

System design interviews are not about inventing a perfect architecture on the spot. They are about showing a calm, repeatable process: clarify requirements, estimate scale, sketch a simple design, explain trade-offs, and improve it when constraints change.

---



```architecture
{
  "version": 1,
  "title": "TLDR:** System design interviews are not about inventing a perfect architecture on the spot. They are about…

This walkthrough focuses on **Why System Design Interviews Feel Hard Before They Feel Logical**.

Before making a design choice, clarify:

- Most beginners think a system design interview is a test of memory.
- They assume they must instantly recall the "right" architecture for a URL shortener, chat app, or news feed.
- That is the wrong mental model.

**Concepts in play:** system-design-hld-architectures, caching, api-gateway, consistency-replication

## Stage 1 — The Three Beginner Moves Before You Draw Anything

- Before any diagram, API, or storage choice, make three moves.
- Ask what the user can do and what the system does not need to do.
- If the prompt is "Design Instagram," you do not need to design every feature on Earth.
- You might narrow the scope to posting photos, following users, and loading a home feed.

*Applies **system-design-hld-architectures** to the decision above.*

**Observable outcome:** You might narrow the scope to posting photos, following users, and loading a home feed.

## Stage 2 — The 30-Minute Whiteboard Flow That Keeps You Organized

- Once you have the scope, use a simple flow for nearly every interview.
- Estimate rough scale with quick capacity estimation math.
- Identify the core entities and APIs with explicit API contract design and data modeling choices.
- Sketch a high-level architecture.

*Applies **caching** to the decision above.*

**Observable outcome:** Sketch a high-level architecture.

## Stage 3 — Real-World Application: Designing a Tiny URL Shortener in an Interview

- A URL shortener is one of the best beginner interview exercises because the problem is small enough to explain clearly but still exposes real design trade-offs.
- Start with the simplest product definition: - A user submits a long URL.
- - The system returns a short code.
- - Anyone with that short code can redirect to the original URL.

*Applies **api-gateway** to the decision above.*

**Observable outcome:** - Anyone with that short code can redirect to the original URL.

## Stage 4 — Practical Example: How to Answer "Design a Rate Limiter"

- If the interviewer asks you to design a rate limiter, do not start by naming Redis and token buckets immediately.
- Start with the shape of the problem.
- Is the limit per user, per IP, per API key, or per endpoint?
- Is it 100 requests per minute or 1,000 per second?

*Applies **consistency-replication** to the decision above.*

**Observable outcome:** Is it 100 requests per minute or 1,000 per second?

## Stage 5 — From Interview Diagram to Running System: Architecture in Practice

- Spring Boot is the standard Java framework for building production-ready microservices with embedded servers and auto-configuration.
- Docker Compose is the local orchestration tool that lets you wire a multi-service architecture — app server, database, cache — into a single configuration file, turning your whiteboard diagram into a runnable environmen…
- For the URL shortener example, the architecture described in the interview maps directly onto three services: the application server, a relational database for durable URL mappings, and a Redis cache for fast redirect r…
- The application uses the cache-aside pattern for reads — checking Redis first and falling back to the database on a miss, then populating Redis so the next request is served from cache.

*Applies **system-design-hld-architectures** to the decision above.*

```architecture
{
  "version": 1,
  "title": "From Interview Diagram to Running System: Architecture in Practice - Subsystem Overview",
  "description": "Decoupled system architecture isolating ingress, stateless logic, and persistent datastores.",
  "view": "high-level",
  "pattern": "microservices",
  "direction": "LR",
  "groups": [
    { "id": "edge", "label": "Edge Traffic Ingress", "tone": "slate" },
    { "id": "services", "label": "Stateless Microservices", "tone": "blue" },
    { "id": "data", "label": "Datastore Infrastructure", "tone": "green" }
  ],
  "nodes": [
    { "id": "gw", "component": "api-gateway", "label": "API Gateway", "group": "edge", "badge": "TLS & Auth" },
    { "id": "app", "component": "workflow-service", "label": "Domain Service", "group": "services", "badge": "Stateless" },
    { "id": "cache", "component": "redis", "label": "Redis Cache", "group": "data", "badge": "Sub-ms Reads" },
    { "id": "db", "component": "postgres", "label": "Transactional DB", "group": "data", "badge": "ACID Log" }
  ],
  "edges": [
    { "from": "gw", "to": "app", "kind": "request", "label": "gRPC Request" },
    { "from": "app", "to": "cache", "kind": "read", "label": "Cache Lookups" },
    { "from": "app", "to": "db", "kind": "write", "label": "Durable Write" }
  ],
  "takeaway": {
    "title": "Architecture Boundary Invariant",
    "body": "Stateless compute nodes scale horizontally while shielding underlying database clusters behind fast cache layers."
  }
}
```

**Observable outcome:** The application uses the cache-aside pattern for reads — checking Redis first and falling back to the database on a miss, then populating Redis so the next request is served from cache.

## End-to-end outcome

After working through the stages, you should be able to explain:

- - The best first answer is usually a simple answer with clear assumptions.
- - Requirements and scale estimates matter more than naming fashionable tools.

## Alternatives considered

### Decision Guide: When to Go Simple and When to Add Scale

You do not get extra points for complexity by default.


## Failure and degraded modes

### Trade-offs & Failure Modes: Where Simple Answers Break

- **Detection:** A beginner answer becomes stronger the moment you acknowledge where it breaks.
- **User or operator impact:** A beginner answer becomes stronger the moment you acknowledge where it breaks.



## Scenario context

In system design interviews, weak answers fail early because requirements are fuzzy. Strong answers start by turning vague prompts into explicit functional scope, measurable non-functional targets, and clear trade-off boundaries before any architecture diagram appears.

> **TLDR**: If you clarify requirements well, the architecture almost chooses itself.



```architecture
{
  "version": 1,
  "title": "TLDR:** In sy…

This walkthrough focuses on **Why Requirement Clarity Is the Real Beginning of System Design**.

Before making a design choice, clarify:

- Slack assumed users were on reliable corporate networks.
- When mobile users on 3G hit the app in 2015, 40% quit within 30 seconds.
- The non-functional requirement "must load in under 3 seconds on 3G" was never written down.

**Concepts in play:** system-design-hld-architectures, caching, consistency-replication, stack-and-queue-data-structures

## Stage 1 — The Requirement Stack: Functional, Non-Functional, and Business Constraints

- A reliable way to avoid chaos is to classify requirements into layers.
- Functional requirements answer "What should the system do?" Examples: - Users can create short links.
- - Users can view a personalized feed.
- - Drivers can request rides and track status.

*Applies **system-design-hld-architectures** to the decision above.*

**Observable outcome:** - Drivers can request rides and track status.

## Stage 2 — A Practical Requirement Interview Script You Can Reuse

- Candidates often ask: "What exactly should I ask first?" Use a short script in this order: 1.
- Define the primary user journey.
- Define strict consistency boundaries.
- Here is a reusable checklist table: This script works because it does not require perfect numbers.

*Applies **caching** to the decision above.*

**Observable outcome:** Here is a reusable checklist table: This script works because it does not require perfect numbers.

## Stage 3 — Deep Dive: Translating Requirements Into Enforceable Design Decisions

- Requirement gathering is useful only if it drives specific architecture decisions.
- The translation step is where many interviews are won.
- The Internals: Requirement-to-Component Mapping Every clarified constraint should map to one or more design mechanisms.
- - Low read latency target - cache layer, denormalized read model, or edge routing.

*Applies **consistency-replication** to the decision above.*

**Observable outcome:** - Low read latency target - cache layer, denormalized read model, or edge routing.

## Stage 4 — Requirement Funnel: From Vague Prompt to Defensible Architecture

- architecture { "version": 1, "title": "Requirement Funnel: From Vague Prompt to Defensible Architecture - Subsystem Overview", "description": "Decoupled system architecture isolating ingress, stateless logic, and persis…
- If the interview starts drifting, return to the funnel and show what changed in assumptions.
- 📊 Requirements Classification Tree This classification tree shows how any system requirement maps to one of three categories.
- Functional requirements define what the system does — user actions, core operations, and API boundaries; non-functional requirements define how well it must do it — latency targets, availability SLOs, and consistency le…

*Applies **stack-and-queue-data-structures** to the decision above.*

```mermaid

```

```

flowchart TD
    A[Vague interview prompt] --> B[Clarify functional scope]
    B --> C[Capture non-functional targets]
    C --> D[Set constraints and assumptions]
    D --> E[Define out-of-scope boundaries]
    E --> F[Map constraints to components]
    F --> G[Present architecture with trade-offs]

```

**Observable outcome:** Functional requirements define what the system does — user actions, core operations, and API boundaries; non-functional requirements define how well it must do it — latency targets, availability SLOs, and consistency le…

## Stage 5 — Real-World Applications: Notification, Feed, and Checkout Systems

- The same requirement framework applies across very different domains.
- Notification platform: - Functional: send notification, view delivery status.
- - Non-functional: near-real-time delivery for push, eventual for email.
- - Constraints: provider rate limits, regional SMS regulations.

*Applies **system-design-hld-architectures** to the decision above.*

**Observable outcome:** - Constraints: provider rate limits, regional SMS regulations.

## Stage 6 — Practical Example: Requirement Breakdown for "Design a Chat System"

- Suppose the interviewer says: "Design WhatsApp." A structured response starts with narrowing: - Phase 1: one-to-one messaging only.
- - Exclude group chat, media compression, and end-to-end encryption details from MVP.
- Then define measurable assumptions: Now architecture decisions follow naturally: 1.
- Per-conversation ordering requirement - partition messages by conversation ID.

*Applies **caching** to the decision above.*

**Observable outcome:** Per-conversation ordering requirement - partition messages by conversation ID.

## End-to-end outcome

After working through the stages, you should be able to explain:

- - Requirements are architecture inputs, not interview formalities.
- - Functional, non-functional, and business constraints should be separated explicitly.

## Alternatives considered

### Decision Guide: Which Requirement Style Fits the Interview Prompt?

This decision table helps you adapt your questioning style without sounding scripted.


## Failure and degraded modes

### Trade-offs & Failure Modes: What Goes Wrong When Requirements Are Weak

- **Detection:** A strong candidate explicitly narrates these failure modes and shows how requirement discipline prevents them.
- **User or operator impact:** A strong candidate explicitly narrates these failure modes and shows how requirement discipline prevents them.



## Scenario context

Capacity estimation is the skill of back-of-the-envelope math that tells you whether your system design will survive its traffic before you write a line of code. Four numbers do most of the work: **DAU, QPS, Storage/day, and Bandwidth/day**.






```mermaid
flowchart LR
    DAU["Daily Active Users (e.g., 10M)"] --> QPS["Convert to QPS (requests/sec)"]
    QPS --> Storage["Storage/day (data written)"]
    QPS --> Ba…

This walkthrough focuses on **Reference Numbers to Memorize**.

Before making a design choice, clarify:

- 

**Concepts in play:** system-design-hld-architectures, caching, jvm-internals-and-garbage-collection, consistency-replication

## Stage 1 — Estimation Decision Flow

- Use this decision tree to structure any capacity estimation from scratch.
- Start with DAU and follow the branches to determine what infrastructure tier your system actually needs — single server, horizontally scaled, or fully distributed.
- The key decision point is the scale tier .
- A system handling 100 RPS and one handling 100,000 RPS require fundamentally different architectures — not just bigger servers, but different data stores, caching strategies, and deployment models.

*Applies **system-design-hld-architectures** to the decision above.*

```mermaid
flowchart TD
    Start[Start Estimation] --> DAU[Estimate DAU]
    DAU --> RW[Estimate Read/Write Ratio]
    RW --> QPS[Calculate QPS DAU  actions / 86400]
    QPS --> Peak[Multiply by 2-3x for peak]
    Peak --> Storage[Calculate Storage/day write QPS  record size  86400]
    Storage --> BW[Calculate Bandwidth read QPS  response size]
    BW --> Scale{Scale tier?}
    Scale -->|< 1000 RPS| Small[Single server PostgreSQL]
    Scale -->|1k-100k RPS| Med[App servers + cache + sharding]
    Scale -->|> 100k RPS| Large[Distributed system CDN + microservices]
```

**Observable outcome:** A system handling 100 RPS and one handling 100,000 RPS require fundamentally different architectures — not just bigger servers, but different data stores, caching strategies, and deployment models.

## Stage 2 — Real-World Application: Capacity Estimation Across Scale Tiers

- The best way to internalize the four-step pipeline is to run it on systems you already use every day.
- Here are three scenarios at very different scales that demonstrate how the numbers drive architecture decisions.
- Twitter/X-Scale Feed - DAU: 400M users.
- Each user reads ~10 timeline requests/day; each user writes ~0.1 tweets/day.

*Applies **caching** to the decision above.*

**Observable outcome:** Each user reads ~10 timeline requests/day; each user writes ~0.1 tweets/day.

## Stage 3 — Deep Dive: Worked Example — Design a Pastebin

- - Write QPS = (1M × 1 paste/day) / 86400 ≈ 12 writes/sec - Read QPS = 12 × 10 = 120 reads/sec - Storage: 12 writes/sec × 10 KB × 86400 = ~10 GB/day → 10 TB over 3 years.
- - Read bandwidth: 120 reads/sec × 10 KB = 1.2 MB/sec → no CDN needed at this scale.
- What this tells you: - A single PostgreSQL can comfortably handle sub-1000 writes/sec.
- - Storage backend should be durable object storage to handle 10 TB over 3 years.

*Applies **jvm-internals-and-garbage-collection** to the decision above.*

**Observable outcome:** - Storage backend should be durable object storage to handle 10 TB over 3 years.

## Stage 4 — Practical: Interview Estimation Template

- In a system design interview, the interviewer is watching whether you have a structured process — not whether you arrive at the exact right number.
- Here is a reusable five-step template you can follow verbatim: Quick Reference Decision Rules Two threshold rules that short-circuit most infrastructure architecture debates: - If storage 1 TB/day → use object storage (…
- A relational database is not designed for multi-terabyte raw ingest at sustained write rates.
- - If read bandwidth 1 GB/s → put a CDN in front.

*Applies **consistency-replication** to the decision above.*

```
Step 1 — State assumptions:
  - DAU: X million
  - Read:Write ratio: X:1
  - Average object size: X KB/MB

Step 2 — Calculate QPS:
  - Write QPS = DAU × writes_per_user / 86400
  - Read QPS = Write QPS × read_write_ratio
  - Peak QPS = Average QPS × 3

Step 3 — Calculate Storage:
  - Storage/day = write QPS × record_size × 86400
  - 5-year storage = storage/day × 365 × 5
  - Account for replication: × 3

Step 4 — Calculate Bandwidth:
  - Read bandwidth = read QPS × response_size
  - CDN threshold: > 1 GB/s read bandwidth

Step 5 — Determine scale tier:
  - < 1k RPS: single server
  - 1k-100k RPS: horizontal scaling + cache
  - > 100k RPS: distributed + CDN
```

**Observable outcome:** - If read bandwidth 1 GB/s → put a CDN in front.

## Stage 5 — Spring Boot + JMeter: From Capacity Estimates to Measured Throughput

- Spring Boot is the standard Java framework for building the HTTP services whose QPS, storage, and latency you estimate.
- Apache JMeter is the open-source load testing tool that validates those estimates against a running service — generating synthetic traffic at the calculated peak load and measuring actual throughput and p95 latency.
- The workflow: estimate peak QPS on paper → implement the endpoint → run a JMeter test plan at estimated load → compare results against your SLA targets.
- Instrument the endpoint to compare live p95/p99 against your paper estimate: JMeter test plan — simulating estimated peak load: Capacity estimate vs.

*Applies **system-design-hld-architectures** to the decision above.*

```java
// Close the estimation feedback loop: @Timed captures actual latency from live traffic
// so you can compare measured p95 against the "p95 < 50ms at 11,600 RPS" estimate.

@GetMapping("/api/users/{id}")
@Timed(value = "user.lookup.latency",
       percentiles = {0.50, 0.95, 0.99},    // exported to Prometheus/Grafana/Actuator
       description = "Cache-aside: Redis ~0.5ms hit, DB ~5ms miss")
public ResponseEntity<UserDto> getUser(@PathVariable String id) {
    // L1 — Redis: serves ~90%+ of requests at sub-millisecond cost
    UserDto cached = cache.opsForValue().get("user:" + id);
    if (cached != null) return ResponseEntity.ok(cached);

    // L2 — PostgreSQL: cache miss, fills cache for next request (5-minute TTL)
    return userRepository.findById(id)
        .map(u -> { cache.opsForValue().set("user:" + id, u, 5, MINUTES); return ResponseEntity.ok(u); })
        .orElse(ResponseEntity.notFound().build());
}
// If Actuator shows p99 > 100ms → add a read replica or extend Redis TTL
```

```xml


<ThreadGroup>
  <numThreads>1000</numThreads>         
  <rampTime>60</rampTime>               
  <duration>300</duration>              

  <HTTPSamplerProxy>
    <domain>localhost</domain>
    <port>8080</port>
    <path>/api/users/${userId}</path>   
    <method>GET</method>
  </HTTPSamplerProxy>

  
</ThreadGroup>
```

**Observable outcome:** Instrument the endpoint to compare live p95/p99 against your paper estimate: JMeter test plan — simulating estimated peak load: Capacity estimate vs.

## Stage 6 — Key Lessons from Back-of-Envelope Estimation

- Five lessons that separate engineers who nail capacity estimation from those who overcomplicate it: 1.
- The goal is the scale tier, not the exact number.
- Whether your estimate lands at 80,000 RPS or 120,000 RPS doesn't change the architecture decision: both require distributed infrastructure with a load balancer and horizontal scaling.
- A 10× error is usually tolerable; a 100× error is a design problem.

*Applies **caching** to the decision above.*

```mermaid
flowchart TD
    Start[Start: State Assumptions] --> DAU[DAU  actions/user]
    DAU --> QPS[Avg Write QPS = DAU  writes / 86400]
    QPS --> Peak[Peak QPS = Avg  2-3x]
    Peak --> RQPS[Read QPS = Write QPS  read: write ratio]
    RQPS --> Storage[Storage/day = write QPS  record size  86400]
    RQPS --> BW[Bandwidth/day = read QPS  response size]
    Storage --> Scale{Scale tier?}
    BW --> CDN{BW > 1 GB/s?}
    CDN -->|Yes| AddCDN[Add CDN layer]
    CDN -->|No| NoCDN[Direct serving OK]
    Scale -->|under 1k RPS| Small[Single server]
    Scale -->|1k-100k RPS| Med[Horizontal + cache]
    Scale -->|over 100k RPS| Large[Distributed + CDN]
```

**Observable outcome:** A 10× error is usually tolerable; a 100× error is a design problem.

## End-to-end outcome

The Role of Data in Precise Capacity Estimations for System Design

## Alternatives considered

The source focuses on one primary design. A second architecture is not compared, so no alternative is invented here.

## Failure and degraded modes

### Trade-offs & Failure Modes: Trade-offs, Failure Modes & Decision Guide: Estimation Mistakes

- **Detection:** Trade-offs & Failure Modes: Trade-offs, Failure Modes & Decision Guide: Estimation Mistakes
- **User or operator impact:** Trade-offs & Failure Modes: Trade-offs, Failure Modes & Decision Guide: Estimation Mistakes

## Scenario context

In system design interviews, API design is not a list of HTTP verbs. It is a contract strategy: clear resource boundaries, stable request and response shapes, pagination, idempotency, error semantics, and versioning decisions that survive scale and failures.

> **TLDR**: Good API design reduces ambiguity for clients and prevents operational incidents when traffic grows.

In 2015, Stripe renamed the `amount` field to…

This walkthrough focuses on **Why API Design Is an Architecture Decision, Not a Syntax Exercise**.

Before making a design choice, clarify:

- In 2015, Stripe renamed the amount field to amount cents in their charge response object — a clarification that made semantic sense internally.
- They had an active changelog and sent developer notifications.
- It still silently broke integrations for hundreds of third-party apps and payment platforms that parsed the response without strict field validation.

**Concepts in play:** system-design-hld-architectures, caching, git-internals-and-versioning, api-gateway

## Stage 1 — The API Contract Checklist You Should Apply in Every Interview

- You can use a reusable checklist to keep API design systematic.
- Define the resource model and identifiers.
- Define the core operations per resource.
- Define request and response fields with explicit constraints.

*Applies **system-design-hld-architectures** to the decision above.*

**Observable outcome:** Define request and response fields with explicit constraints.

## Stage 2 — Deep Dive: Translating Product Behavior Into Stable API Contracts

- API contracts are where product semantics become system boundaries.
- The Internals: Validation, Idempotency Store, and Backward Compatibility At runtime, robust API services usually implement these internal mechanisms: - Request validation layer for schema and semantic rules.
- - Idempotency key store for safe retried writes.
- - Serialization logic with explicit field defaults.

*Applies **caching** to the decision above.*

**Observable outcome:** - Serialization logic with explicit field defaults.

## Stage 3 — API Lifecycle Flow From Client Request to Stable Response

- This flowchart captures the API contract philosophy: consistent validation, safe retries, and predictable responses.
- 📊 REST API Request Lifecycle This sequence diagram traces the full lifecycle of a single REST API request from the client through the gateway and into the order service, highlighting the idempotency key check and the d…
- The flow shows that the API gateway handles cross-cutting concerns (auth, rate limiting) while the service layer owns business validation and deduplication.
- Take away: every hop in this chain is an opportunity for failure, which is why idempotency at the service boundary prevents duplicate side effects from network retries.

*Applies **git-internals-and-versioning** to the decision above.*

```mermaid
flowchart TD
    A[Client request] --> B[Schema and semantic validation]
    B --> C{Idempotency key present?}
    C -->|Yes| D[Check idempotency store]
    D --> E{Seen before?}
    E -->|Yes| F[Return previous response]
    E -->|No| G[Execute business transaction]
    C -->|No| G
    G --> H[Persist result and emit event]
    H --> I[Return structured response]

```

```mermaid
sequenceDiagram
    participant C as Client
    participant G as API Gateway
    participant S as Order Service
    participant DB as Database
    C->>G: POST /v1/orders
    G->>G: Auth and rate limit
    G->>S: Forward request
    S->>S: Validate schema
    S->>DB: Check idempotency key
    DB-->>S: Key not found
    S->>DB: Write order record
    DB-->>S: Commit OK
    S-->>G: 201 Created
    G-->>C: 201 + order_id

```

**Observable outcome:** Take away: every hop in this chain is an opportunity for failure, which is why idempotency at the service boundary prevents duplicate side effects from network retries.

## Stage 4 — Real-World Applications: Payments, Feeds, and Internal Microservices

- Payments API: idempotency is non-negotiable because duplicate charges are business-critical incidents.
- Timeline/feed API: pagination and filtering matter most, because reads dominate and data size grows continuously.
- Internal microservice APIs: strict schemas, backward compatibility, and explicit error contracts reduce coordination cost between teams.
- Different domains stress different contract dimensions, but the checklist remains stable.

*Applies **api-gateway** to the decision above.*

**Observable outcome:** Different domains stress different contract dimensions, but the checklist remains stable.

## Stage 5 — Practical Example: Design a Create-and-List Orders API

- Suppose your interview prompt includes order creation and order history.
- You can propose: - POST /v1/orders with idempotency key.
- - GET /v1/orders/{order id} for direct lookup.
- - GET /v1/orders?customer id=...&cursor=...&limit=...

*Applies **system-design-hld-architectures** to the decision above.*

**Observable outcome:** - GET /v1/orders?customer id=...&cursor=...&limit=...

## Stage 6 — Self-Documenting API Contracts: Design Approach

- A production REST API benefits from machine-readable contract documentation that stays in sync with the implementation.
- The key principle is that the contract specification should be derived from the code rather than maintained as a separate manual artifact — ensuring that the documented API always matches what the service actually does.
- In practice, this means annotating each endpoint with its intended summary, description, request parameters, and expected response codes at the point of definition.
- A create-order endpoint would declare that it accepts an idempotency key header, requires a valid request body, and returns 201 on success, 409 on duplicate key conflict, 429 on rate limit, and 422 on malformed input.

*Applies **caching** to the decision above.*

**Observable outcome:** A create-order endpoint would declare that it accepts an idempotency key header, requires a valid request body, and returns 201 on success, 409 on duplicate key conflict, 429 on rate limit, and 422 on malformed input.

## End-to-end outcome

After working through the stages, you should be able to explain:

- - API design is about long-lived contracts, not endpoint naming alone.
- - Idempotency and pagination should be first-class concerns in write and list APIs.

## Alternatives considered

### Decision Guide: REST, RPC, and Contract Complexity

If you need protocol-level trade-offs, pair this post with System Design Protocols: REST, RPC, and TCP/UDP.


## Failure and degraded modes

### Pattern 1: Resource-first endpoint design

- **Detection:** Instead of action-heavy endpoints like /createOrder , design around resources: - POST /orders - GET /orders/{order id} - GET /orders?customer id=...
- **User or operator impact:** Instead of action-heavy endpoints like /createOrder , design around resources: - POST /orders - GET /orders/{order id} - GET /orders?customer id=...

### Pattern 2: Idempotent writes for retry safety

- **Detection:** Client retries are inevitable during network failures.
- **User or operator impact:** Client retries are inevitable during network failures.

### Pattern 3: Cursor-based pagination

- **Detection:** Offset pagination ( page=1000 ) becomes slow and unstable at scale.
- **User or operator impact:** Offset pagination ( page=1000 ) becomes slow and unstable at scale.

### Pattern 4: Structured errors

- **Detection:** Avoid free-form strings as your main failure contract.
- **User or operator impact:** Avoid free-form strings as your main failure contract.

### Trade-offs & Failure Modes: How API Contracts Break at Scale

- **Detection:** The best interview answer names at least one failure mode and one mitigation tied to API contract design.
- **User or operator impact:** The best interview answer names at least one failure mode and one mitigation tied to API contract design.



Learn the interview frame, requirements, APIs, and capacity estimation.

HLD Interview Foundation

## Scenario context

SQL gives you ACID guarantees and powerful relational queries; NoSQL gives you horizontal scale and flexible schemas. The real decision is not "which is better" — it is "which trade-offs align with your workload." Understanding replication, sharding, and the CAP theorem helps you pick correctly every time.

---



```architecture
{
  "version": 1,
  "title": "TLDR:** SQL gives you ACID guarantees and powerful relati…

This walkthrough focuses on **SQL and NoSQL Engines: What Is Actually Inside**.

Before making a design choice, clarify:

- SQL internals SQL engines parse → plan → optimize → execute.
- - B-Tree indexes for range queries.
- - Hash indexes for equality lookups.

**Concepts in play:** consistency-replication, database-indexing, btree-vs-lsmtree, caching

## Stage 1 — Key Properties That Define Each Category

- Before choosing a database, understand four foundational properties that set SQL and NoSQL apart.
- Schema enforcement : SQL requires every row to match the table schema at write time.
- NoSQL is schema-on-read — the collection stores any document shape, and your application code interprets the structure at query time.
- This is powerful for rapid iteration but requires data quality enforcement at the application layer.

*Applies **consistency-replication** to the decision above.*

**Observable outcome:** This is powerful for rapid iteration but requires data quality enforcement at the application layer.

## Stage 2 — Scaling Strategies: Vertical, Horizontal, Replication, Sharding

- architecture { "version": 1, "title": "Scaling Strategies: Vertical, Horizontal, Replication, Sharding - Storage Topology", "description": "Multi-tier caching and replicated storage pipeline ensuring fast reads and tran…
- A write enters the master's Write-Ahead Log, is applied to the in-memory MemTable, flushed to disk as an SSTable, and then propagated via a replication stream to each replica node, which applies the same changes in orde…
- The key insight is that replicas always trail the primary by some replication lag — reads from a replica may return slightly stale data, which is acceptable for social feeds but unacceptable for financial transactions.
- Replication lag is the time difference between master and replica: $$\Delta = t ext{master} - t ext{replica}$$ A small $\Delta$ is acceptable for social feeds; it is catastrophic for financial ledgers.

*Applies **database-indexing** to the decision above.*

```mermaid

```

```

graph TD
    subgraph Master
        A[Write Request] --> B[Write-Ahead Log]
        B --> C[Apply to MemTable]
        C --> D[Flush to SSTable / Commit]
    end
    subgraph Replicas
        D --> E[Replication Stream]
        E --> F[Apply to Replica Log]
        F --> G[Update Replica State]
    end
    G --> H[Read Request from Replica]

```

**Observable outcome:** Replication lag is the time difference between master and replica: $$\Delta = t ext{master} - t ext{replica}$$ A small $\Delta$ is acceptable for social feeds; it is catastrophic for financial ledgers.

## Stage 3 — Deep Dive: How Write-Ahead Logs and LSM-Trees Protect Data

- Both SQL and NoSQL engines guard against data loss using append-only logs before modifying data files.
- SQL engines use a Write-Ahead Log (WAL) : every change is written sequentially to a log before being applied to B-Tree pages, enabling crash recovery.
- NoSQL LSM-tree engines write first to an in-memory MemTable , flush to immutable SSTables on disk, and periodically compact them.

*Applies **btree-vs-lsmtree** to the decision above.*

**Observable outcome:** NoSQL LSM-tree engines write first to an in-memory MemTable , flush to immutable SSTables on disk, and periodically compact them.

## Stage 4 — Decision Flow: Selecting the Right Database

- Use this flowchart as a first-pass guide when choosing a database for a new service.
- Most production systems combine multiple databases — for example, PostgreSQL for transactional data alongside Redis for caching.
- Start here and refine with your specific read/write ratio and consistency requirements.
- 📊 Read-Your-Writes Consistency in Replicated DB This sequence diagram illustrates the read-your-writes consistency hazard in a replicated database.

*Applies **caching** to the decision above.*

```architecture
{
  "version": 1,
  "title": "Decision Flow: Selecting the Right Database - Storage Topology",
  "description": "Multi-tier caching and replicated storage pipeline ensuring fast reads and transactional consistency.",
  "view": "data-flow",
  "pattern": "cache-aside",
  "direction": "LR",
  "groups": [
    { "id": "compute", "label": "Application Workers", "tone": "blue" },
    { "id": "cache-tier", "label": "Volatile Memory Tier", "tone": "amber" },
    { "id": "storage-tier", "label": "Persistent Disk Tier", "tone": "green" }
  ],
  "nodes": [
    { "id": "worker", "component": "workflow-service", "label": "Worker Process", "group": "compute" },
    { "id": "mem", "component": "redis", "label": "In-Memory Cache", "group": "cache-tier", "badge": "Hot Keys" },
    { "id": "primary", "component": "postgres", "label": "Primary Database", "group": "storage-tier", "badge": "Write Master" }
  ],
  "edges": [
    { "from": "worker", "to": "mem", "kind": "read", "label": "Fast Read" },
    { "from": "worker", "to": "primary", "kind": "write", "label": "Transactional Commit" }
  ],
  "takeaway": {
    "title": "Storage Access Pattern",
    "body": "In-memory caching handles high-frequency read spikes, reserving disk transactions strictly for state modifications."
  }
}
```

```mermaid
flowchart TD
    A[Define your workload] --> B{Need ACID transactions?}
    B -->|Yes| C[SQL: PostgreSQL / MySQL]
    B -->|No| D{Write-heavy at massive scale?}
    D -->|Yes| E[Wide-column NoSQL: Cassandra]
    D -->|No| F{Schema changes frequently?}
    F -->|Yes| G[Document NoSQL: MongoDB]
    F -->|No| H{Ultra-fast key lookups needed?}
    H -->|Yes| I[Key-Value Store: Redis / DynamoDB]
    H -->|No| C

```

**Observable outcome:** 📊 Read-Your-Writes Consistency in Replicated DB This sequence diagram illustrates the read-your-writes consistency hazard in a replicated database.

## Stage 5 — Choosing Your Database: A Worked Example

- This example models an orders entity in an e-commerce system side-by-side in PostgreSQL and MongoDB to make the SQL-vs-NoSQL trade-off concrete and visible in real code.
- The orders entity is the canonical choice because it sits at the boundary: it demands multi-table joins and ACID guarantees for accurate reporting (SQL's strengths) yet benefits from document embedding when every read n…
- As you read, focus on how the SQL version normalises items into a separate table — enabling flexible queries at the cost of joins — while the MongoDB version embeds items directly in the order document, enabling a singl…
- SQL representation (PostgreSQL): In the normalized SQL model, customer names and order totals live in separate tables linked by a foreign key on user id .

*Applies **consistency-replication** to the decision above.*

```architecture
{
  "version": 1,
  "title": "Choosing Your Database: A Worked Example - Storage Topology",
  "description": "Multi-tier caching and replicated storage pipeline ensuring fast reads and transactional consistency.",
  "view": "data-flow",
  "pattern": "cache-aside",
  "direction": "LR",
  "groups": [
    { "id": "compute", "label": "Application Workers", "tone": "blue" },
    { "id": "cache-tier", "label": "Volatile Memory Tier", "tone": "amber" },
    { "id": "storage-tier", "label": "Persistent Disk Tier", "tone": "green" }
  ],
  "nodes": [
    { "id": "worker", "component": "workflow-service", "label": "Worker Process", "group": "compute" },
    { "id": "mem", "component": "redis", "label": "In-Memory Cache", "group": "cache-tier", "badge": "Hot Keys" },
    { "id": "primary", "component": "postgres", "label": "Primary Database", "group": "storage-tier", "badge": "Write Master" }
  ],
  "edges": [
    { "from": "worker", "to": "mem", "kind": "read", "label": "Fast Read" },
    { "from": "worker", "to": "primary", "kind": "write", "label": "Transactional Commit" }
  ],
  "takeaway": {
    "title": "Storage Access Pattern",
    "body": "In-memory caching handles high-frequency read spikes, reserving disk transactions strictly for state modifications."
  }
}
```

**Observable outcome:** SQL representation (PostgreSQL): In the normalized SQL model, customer names and order totals live in separate tables linked by a foreign key on user id .

## Stage 6 — Real-World Applications by Workload Type

- 

*Applies **database-indexing** to the decision above.*

**Observable outcome:** You can explain how “Real-World Applications by Workload Type” changes the architecture or implementation path.

## End-to-end outcome

SQL internals SQL engines parse → plan → optimize → execute.

## Alternatives considered

### The Library vs. the Drop Box: SQL and NoSQL Intuition

By 2013, they had 10 billion pins, 100 million users, and a 4-second page load time .

### Decision Guide: SQL vs. NoSQL at a Glance

Decision Guide: SQL vs. NoSQL at a Glance

### Schema Evolution: SQL vs. NoSQL in Practice

One of the most tangible day-to-day differences between SQL and NoSQL is how they handle schema changes as product requirements evolve.


## Failure and degraded modes

### Trade-offs & Failure Modes: Trade-offs and Failure Modes

- **Detection:** Split-brain scenario (master-master setup): 1.
- **User or operator impact:** Split-brain scenario (master-master setup): 1.



## Scenario context

Caching is the single highest-leverage performance tool in distributed systems. This guide covers every read/write pattern (Cache-Aside through Refresh-Ahead), every eviction policy (LRU through ARC), cache invalidation pitfalls, thundering herd mitigations, multi-level topologies, and how Twitter, Netflix, and Facebook run caches at planetary scale. Read once; reference forever.

---



```architecture
{
  "version…

This walkthrough focuses on **Why Your Database Is Begging for a Cache**.

Before making a design choice, clarify:

- Every relational database has the same story: it starts fast, traffic grows, queries get slower, you add indexes, you tune joins, and eventually you hit the wall — the database is doing too much I/O for too many concurr…
- The standard answer is a cache.
- A cache is a key-value store that lives in RAM.

**Concepts in play:** caching, consistency-replication, system-design-hld-architectures, database-indexing

## Stage 1 — How Caches Fit Into the Read and Write Path

- Before diving into patterns, ground yourself in what a cache is not: it is not a database.
- Caches are: - Volatile — data survives only as long as the cache process is running (unless persistence is explicitly configured, as in Redis RDB/AOF modes).
- - Bounded — caches have a fixed memory budget and cannot grow infinitely.
- When the budget fills, the eviction policy decides what to drop.

*Applies **caching** to the decision above.*

**Observable outcome:** When the budget fills, the eviction policy decides what to drop.

## Stage 2 — Every Cache Pattern Explained: From Cache-Aside to Refresh-Ahead

- There are six canonical caching patterns.
- Each solves a different problem and carries different consistency trade-offs.
- A typical production system mixes them — Cache-Aside for general reads, Write-Through for high-value writes, and Refresh-Ahead for peak-traffic hot keys.
- Pattern 1: Cache-Aside (Lazy Loading) The most common pattern.

*Applies **consistency-replication** to the decision above.*

```mermaid
flowchart TD
    Request[Incoming Request] --> CheckCache{Check Cache}
    CheckCache -->|Hit| ReturnCached[Return cached data instantly]
    CheckCache -->|Miss| QueryDB[Query Database]
    QueryDB --> PopulateCache[Write result to cache with TTL]
    PopulateCache --> ReturnFresh[Return fresh data to caller]

```

```mermaid
flowchart LR
    subgraph WT[Write-Through]
        WT_App[Application] -->|Write| WT_Cache[Cache]
        WT_Cache -->|Sync write| WT_DB[Database]
        WT_DB -->|Acknowledge| WT_App
    end
    subgraph WB[Write-Back]
        WB_App[Application] -->|Write| WB_Cache[Cache]
        WB_Cache -->|Acknowledge immediately| WB_App
        WB_Cache -.->|Async flush later| WB_DB[Database]
    end

```

**Observable outcome:** Pattern 1: Cache-Aside (Lazy Loading) The most common pattern.

## Stage 3 — Deep Dive: Eviction Policies, Invalidation, and the Thundering Herd

- The Internals: LRU, LFU, ARC, SLRU, and How They Actually Work When the cache is full and a new key must be inserted, the eviction policy selects the victim to remove.
- Choosing the wrong policy wastes memory, degrades hit rates, and in pathological cases (scan pollution), can evict your entire working set.
- LRU — Least Recently Used The most widely deployed policy.
- Internally implemented as a doubly linked list + hash map : the hash map provides O(1) key lookup; the linked list maintains access recency from most-recent (head) to least-recent (tail).

*Applies **system-design-hld-architectures** to the decision above.*

```mermaid
flowchart LR
    Head[HEAD - Most Recent] --> N1[Key C] --> N2[Key A] --> N3[Key F] --> Tail[TAIL - Evict First]

```

**Observable outcome:** Internally implemented as a doubly linked list + hash map : the hash map provides O(1) key lookup; the linked list maintains access recency from most-recent (head) to least-recent (tail).

## Stage 4 — Cache Topologies: Local, Distributed, CDN, and Multi-Level Hierarchies

- Caches exist at multiple levels in a real system.
- Understanding the latency and consistency characteristics of each level is essential for designing the right topology for your workload.
- The four standard layers, ordered from fastest to slowest, are shown below.
- Each hop in the diagram represents an order-of-magnitude latency jump: L1 local cache (nanoseconds), L2 distributed cache (1–2 ms), database (10–50 ms), CDN edge (20–80 ms to nearest PoP vs.

*Applies **database-indexing** to the decision above.*

```mermaid
flowchart LR
    Client[Client Request] --> CDN[CDN Edge Cache]
    CDN -->|Cache miss| LB[Load Balancer]
    LB --> App[App Server]
    App --> L1[L1 Local Cache]
    L1 -->|Miss| L2[L2 Distributed Cache]
    L2 -->|Miss| DB[Database]
```

**Observable outcome:** Each hop in the diagram represents an order-of-magnitude latency jump: L1 local cache (nanoseconds), L2 distributed cache (1–2 ms), database (10–50 ms), CDN edge (20–80 ms to nearest PoP vs.

## Stage 5 — How Twitter, Netflix, Facebook, and Cloudflare Cache at Scale

- Twitter — Redis Sorted Sets for Timeline Assembly Twitter's home timeline is one of the most complex caching problems at scale.
- Assembling a feed from tens of thousands of followed accounts in real time would require expensive fan-in joins across hundreds of database shards on every page load.
- Instead, Twitter pre-computes each user's timeline and caches it in Redis as a sorted set (scores = tweet timestamps).
- Opening the app reads at most 800 timeline entries from Redis — no database joins, purely cache reads answering in sub-millisecond time.

*Applies **caching** to the decision above.*

**Observable outcome:** Opening the app reads at most 800 timeline entries from Redis — no database joins, purely cache reads answering in sub-millisecond time.

## Stage 6 — Sizing and Tuning Your Cache Before It Breaks in Production

- Cache sizing is not optional — a cache that runs out of memory at peak load starts evicting aggressively, hit rate collapses, and the database absorbs a sudden 10× read spike exactly when you can least afford it.
- Step 1 — Measure the working set size.
- The working set is the set of keys that receive at least one access per TTL window.
- Start by logging distinct cache keys accessed over a 5-minute window at peak traffic.

*Applies **consistency-replication** to the decision above.*

**Observable outcome:** Start by logging distinct cache keys accessed over a 5-minute window at peak traffic.

## End-to-end outcome

After working through the stages, you should be able to explain:

- Fix the slow query before you cache it.
- A 500 ms query cached for 5 minutes still hammers the database every 5 minutes — and with thundering herd potential every time the TTL fires.

## Alternatives considered

### Selecting a Cache Pattern by Scenario

Selecting a Cache Pattern by Scenario

### Selecting an Eviction Policy by Workload

Selecting an Eviction Policy by Workload

### Selecting a Cache Topology by Need

Selecting a Cache Topology by Need


## Failure and degraded modes

### Consistency vs. Performance: The Core Caching Trade-off

- **Detection:** Every caching decision sits on the consistency–performance axis.
- **User or operator impact:** Every caching decision sits on the consistency–performance axis.



## Scenario context

Sharding means splitting one logical dataset across multiple physical databases so no single node carries all the data and traffic. The hard part is not adding more nodes. The hard part is choosing a shard key that keeps data balanced and queries practical.

> **TLDR**: If your database is too big for one node, sharding is the next scaling step. The wrong shard key turns that scaling step into a permanent operationa…

This walkthrough focuses on **Why Sharding Appears Right After the Simple Database Stops Being Enough**.

Before making a design choice, clarify:

- In an interview, "we can shard later" sounds reasonable only if you understand what that sentence really means.
- It means the current system may eventually outgrow a single storage node in one of three ways: - The dataset no longer fits comfortably on one machine.
- - Write throughput overloads the primary.

**Concepts in play:** database-sharding, caching, consistency-replication, jvm-internals-and-garbage-collection

## Stage 1 — The Three Common Sharding Patterns and What They Optimize

- There are three common beginner-friendly sharding patterns.
- Range sharding: rows are split by value ranges such as user IDs 1-1M on shard A, 1M-2M on shard B.
- It is intuitive but can create hot spots if new writes cluster in one range.
- Hash sharding: a hash of the shard key distributes data more evenly.

*Applies **database-sharding** to the decision above.*

```architecture
{
  "version": 1,
  "title": "The Three Common Sharding Patterns and What They Optimize - Storage Topology",
  "description": "Multi-tier caching and replicated storage pipeline ensuring fast reads and transactional consistency.",
  "view": "data-flow",
  "pattern": "cache-aside",
  "direction": "LR",
  "groups": [
    { "id": "compute", "label": "Application Workers", "tone": "blue" },
    { "id": "cache-tier", "label": "Volatile Memory Tier", "tone": "amber" },
    { "id": "storage-tier", "label": "Persistent Disk Tier", "tone": "green" }
  ],
  "nodes": [
    { "id": "worker", "component": "workflow-service", "label": "Worker Process", "group": "compute" },
    { "id": "mem", "component": "redis", "label": "In-Memory Cache", "group": "cache-tier", "badge": "Hot Keys" },
    { "id": "primary", "component": "postgres", "label": "Primary Database", "group": "storage-tier", "badge": "Write Master" }
  ],
  "edges": [
    { "from": "worker", "to": "mem", "kind": "read", "label": "Fast Read" },
    { "from": "worker", "to": "primary", "kind": "write", "label": "Transactional Commit" }
  ],
  "takeaway": {
    "title": "Storage Access Pattern",
    "body": "In-memory caching handles high-frequency read spikes, reserving disk transactions strictly for state modifications."
  }
}
```

**Observable outcome:** Hash sharding: a hash of the shard key distributes data more evenly.

## Stage 2 — How a Shard Key Determines Where Every Record Lives

- The shard key is the field or composite of fields used to decide placement.
- In practice, the shard key is often more important than the number of shards.
- Good shard keys usually have three qualities: 1.
- High cardinality so records spread out naturally.

*Applies **caching** to the decision above.*

**Observable outcome:** High cardinality so records spread out naturally.

## Stage 3 — The Data Path: Router to Shard to Rebalancing

- This is the architecture you should keep in your head during an interview.
- The router is only fast when the request includes a routing key.
- Otherwise the system begins to behave like a distributed scatter-gather query engine.
- 📊 Hash Sharding Key Distribution This diagram shows how customer id values are distributed across four database nodes using a modulo hash function.

*Applies **consistency-replication** to the decision above.*

```mermaid
flowchart TD
    C[Client Request] --> R[Shard Router]
    R --> K{Shard key present?}
    K -->|Yes| S1[Route to single shard]
    K -->|No| F[Fan out to many shards]
    S1 --> DB1[(Shard A)]
    F --> DB1
    F --> DB2[(Shard B)]
    F --> DB3[(Shard C)]
    DB1 --> M[Merge results]
    DB2 --> M
    DB3 --> M

```

```mermaid
flowchart LR
    K[customer_id] --> H[HASH_MOD 4]
    H --> S0["Shard 0 (hash=0)"]
    H --> S1["Shard 1 (hash=1)"]
    H --> S2["Shard 2 (hash=2)"]
    H --> S3["Shard 3 (hash=3)"]
    S0 --> D0[(DB Node 0)]
    S1 --> D1[(DB Node 1)]
    S2 --> D2[(DB Node 2)]
    S3 --> D3[(DB Node 3)]

```

**Observable outcome:** 📊 Hash Sharding Key Distribution This diagram shows how customer id values are distributed across four database nodes using a modulo hash function.

## Stage 4 — Real-World Applications: Tenants, Event Streams, and Social Products

- Discord (message sharding at billions of messages): Discord shards message history by channel id across Cassandra clusters.
- Their rule of thumb: 1 logical shard cluster per ~1,000 guilds on average.
- The edge case that breaks this rule: a single guild with 500,000 concurrent users (a major game publisher's community server, for example) creates a hot partition that no even-distribution key can prevent.
- Discord's solution for "celebrity guilds" is explicit manual overrides — routing oversized guilds to dedicated Cassandra nodes rather than placing them on the standard consistent-hash ring.

*Applies **jvm-internals-and-garbage-collection** to the decision above.*

**Observable outcome:** Discord's solution for "celebrity guilds" is explicit manual overrides — routing oversized guilds to dedicated Cassandra nodes rather than placing them on the standard consistent-hash ring.

## Stage 5 — Practical Example: Sharding an Orders Table Without Breaking Queries

- Imagine an e-commerce platform whose orders table has become too large for one primary database.
- You have two obvious shard-key choices: - order id - customer id If most queries are "show this customer's order history," then customer id is a better choice because each user's history stays local to one shard.
- If most queries are point lookups by order number, order id may be simpler.
- Now consider operations: - Customer support wants order history by customer.

*Applies **database-sharding** to the decision above.*

```architecture
{
  "version": 1,
  "title": "Practical Example: Sharding an Orders Table Without Breaking Queries - Storage Topology",
  "description": "Multi-tier caching and replicated storage pipeline ensuring fast reads and transactional consistency.",
  "view": "data-flow",
  "pattern": "cache-aside",
  "direction": "LR",
  "groups": [
    { "id": "compute", "label": "Application Workers", "tone": "blue" },
    { "id": "cache-tier", "label": "Volatile Memory Tier", "tone": "amber" },
    { "id": "storage-tier", "label": "Persistent Disk Tier", "tone": "green" }
  ],
  "nodes": [
    { "id": "worker", "component": "workflow-service", "label": "Worker Process", "group": "compute" },
    { "id": "mem", "component": "redis", "label": "In-Memory Cache", "group": "cache-tier", "badge": "Hot Keys" },
    { "id": "primary", "component": "postgres", "label": "Primary Database", "group": "storage-tier", "badge": "Write Master" }
  ],
  "edges": [
    { "from": "worker", "to": "mem", "kind": "read", "label": "Fast Read" },
    { "from": "worker", "to": "primary", "kind": "write", "label": "Transactional Commit" }
  ],
  "takeaway": {
    "title": "Storage Access Pattern",
    "body": "In-memory caching handles high-frequency read spikes, reserving disk transactions strictly for state modifications."
  }
}
```

**Observable outcome:** Now consider operations: - Customer support wants order history by customer.

## Stage 6 — Apache ShardingSphere and Spring: Custom Shard Routing in Java

- Apache ShardingSphere is an open-source data sharding and routing middleware that integrates with Spring Boot as a transparent JDBC or proxy layer, intercepting SQL queries and routing them to the correct physical shard…
- How it solves the problem: ShardingSphere removes the need to write a custom shard router for every query.
- You declare sharding rules in configuration — the shard key, the hash algorithm, and the physical database mappings — and ShardingSphere rewrites SQL at runtime, transparently routing each query to the correct physical…
- A query that includes customer id in its WHERE clause routes to exactly one shard; a query without a routing key fans out to all shards and ShardingSphere merges the partial results before returning them to the caller.

*Applies **caching** to the decision above.*

```architecture
{
  "version": 1,
  "title": "Apache ShardingSphere and Spring: Custom Shard Routing in Java - Storage Topology",
  "description": "Multi-tier caching and replicated storage pipeline ensuring fast reads and transactional consistency.",
  "view": "data-flow",
  "pattern": "cache-aside",
  "direction": "LR",
  "groups": [
    { "id": "compute", "label": "Application Workers", "tone": "blue" },
    { "id": "cache-tier", "label": "Volatile Memory Tier", "tone": "amber" },
    { "id": "storage-tier", "label": "Persistent Disk Tier", "tone": "green" }
  ],
  "nodes": [
    { "id": "worker", "component": "workflow-service", "label": "Worker Process", "group": "compute" },
    { "id": "mem", "component": "redis", "label": "In-Memory Cache", "group": "cache-tier", "badge": "Hot Keys" },
    { "id": "primary", "component": "postgres", "label": "Primary Database", "group": "storage-tier", "badge": "Write Master" }
  ],
  "edges": [
    { "from": "worker", "to": "mem", "kind": "read", "label": "Fast Read" },
    { "from": "worker", "to": "primary", "kind": "write", "label": "Transactional Commit" }
  ],
  "takeaway": {
    "title": "Storage Access Pattern",
    "body": "In-memory caching handles high-frequency read spikes, reserving disk transactions strictly for state modifications."
  }
}
```

**Observable outcome:** A query that includes customer id in its WHERE clause routes to exactly one shard; a query without a routing key fans out to all shards and ShardingSphere merges the partial results before returning them to the caller.

## End-to-end outcome

After working through the stages, you should be able to explain:

- - Sharding is a data-placement decision before it is a scaling decision.
- - The shard key is the heart of the design.

## Alternatives considered

### Decision Guide: When Should You Actually Shard?

In many interviews, read replicas, caching, or async pipelines are the better first answer.


## Failure and degraded modes

### Trade-offs & Failure Modes: Where Sharding Gets Painful

- **Detection:** This is why the best interview answer often delays sharding until a clear signal appears.
- **User or operator impact:** This is why the best interview answer often delays sharding until a clear signal appears.



## Scenario context

Short shareable IDs need Base62 (URL shorteners). Database primary keys at scale need time-ordered IDs (Snowflake, UUID v7). Security tokens need random IDs (UUID v4, NanoID). Picking the wrong strategy either causes B-tree fragmentation at 50M+ rows, production collisions, or IDs that leak your business metrics to competitors.



```architecture
{
  "version": 1,
  "title": "TLDR:** Short shareable IDs need Base62…

This walkthrough focuses on **The Uniqueness Problem Hiding Inside Every Production System**.

Before making a design choice, clarify:

- You have 500 million shortened URLs.
- Every ID collision is a production incident — the wrong page opens for someone clicking a shared link.
- How do you guarantee uniqueness at scale while keeping IDs short enough for humans to share?

**Concepts in play:** api-gateway, caching, hash-tables-and-bloom-filters, database-indexing

## Stage 1 — What "Base" Actually Means — Encoding Numbers Into Shorter Strings

- "Base encoding" sounds cryptographic, but the concept is simple: you have a number, and you want to represent it using a given set of characters.
- The larger your character set, the fewer characters you need for the same numeric value.
- Think of it like counting in different number systems.
- Decimal (base-10) needs more digits than hexadecimal (base-16) to represent large numbers.

*Applies **api-gateway** to the decision above.*

**Observable outcome:** Decimal (base-10) needs more digits than hexadecimal (base-16) to represent large numbers.

## Stage 2 — How Each ID Strategy Generates Uniqueness at Scale

- Auto-Increment + Base Encoding The database sequence generates a monotonically increasing integer.
- The application encodes it in Base62 (or Base58) to produce a short, human-friendly string.
- Used by bit.ly and early YouTube.
- 6 chars of Base62 gives 56 billion unique IDs from a single counter.

*Applies **caching** to the decision above.*

```mermaid
flowchart TD
    Driver["Driver Node"]
    Planner["Query Planner"]
    Optimizer["Catalyst Optimizer"]
    Executor["Executor Task"]
    
    Driver -->|"Process Flow"| Planner
    Planner --> Optimizer
    Optimizer -->|"Optimized Plan"| Executor
    Executor -->|"Result"| Driver
```

**Observable outcome:** 6 chars of Base62 gives 56 billion unique IDs from a single counter.

## Stage 3 — Visualizing the URL Shortener Write Path with Bloom Filter Guard

- When a user submits a long URL, the system must generate a short ID, verify it has not been used before, and atomically store the mapping.
- A Bloom filter acts as a probabilistic pre-check that avoids a database roundtrip for the vast majority of new IDs — if the filter says "definitely not seen before," skip the DB check.
- If it says "possibly seen," fall back to the database.
- The diagram below shows the write path for a Base62 counter-based URL shortener.

*Applies **hash-tables-and-bloom-filters** to the decision above.*

```mermaid
flowchart TD
    A[User submits long URL] --> B[App Server]
    B --> C{Bloom Filter:  ID seen before?}
    C -->|Definitely not seen| D[Encode counter  Base62 short ID]
    C -->|Possibly seen  collision risk| E[Increment counter, get next ID]
    D --> F{DB unique constraint:  ID already exists?}
    F -->|ID is unique| G[Write mapping to DB]
    F -->|Collision confirmed| E
    G --> H[Add new ID to Bloom Filter]
    G --> I[Return short URL to user]
    E --> D

```

**Observable outcome:** The diagram below shows the write path for a Base62 counter-based URL shortener.

## Stage 4 — The URL Shortener Deep Dive — Making 500M Short Links Work Without a Single Collision

- URL shorteners are the canonical ID strategy interview problem because every constraint is concrete and measurable.
- Let us walk through the decision in order.
- Requirement baseline: 500M URLs total.
- At 6-char Base62: 62^6 = 56.8 billion possible IDs.

*Applies **database-indexing** to the decision above.*

**Observable outcome:** At 6-char Base62: 62^6 = 56.8 billion possible IDs.

## Stage 5 — Choosing Your ID Strategy: Decision Flowchart and Use-Case Matrix

- The right strategy is determined by your constraints.
- Start from length, then sortability, then coordination model.
- This flowchart covers 95% of real-world decisions.
- When two strategies appear equal on the flowchart, prefer the one your database or ORM natively supports — operational familiarity reduces incident blast radius.

*Applies **api-gateway** to the decision above.*

```mermaid
flowchart TD
    A[What are your constraints?] --> B{Need short human-shareable IDs? 8 chars}
    B -->|Yes| C{Sortability required?}
    C -->|Yes| D[ULID or KSUID 2627 chars sortable]
    C -->|No| E[Base62 counter + Bloom filter]
    B -->|No  internal or longer OK| F{Need time-sorted IDs for DB performance?}
    F -->|Yes| G{Multi-node distributed?}
    G -->|Yes| H[Snowflake ID or UUID v7]
    G -->|No  single DB| I[UUID v7 or auto-increment]
    F -->|No sort needed| J{Content deduplication?}
    J -->|Yes| K[Truncated SHA-256  Base62]
    J -->|No| L{Security token? Must be unguessable}
    L -->|Yes| M[NanoID or UUID v4]
    L -->|No| N[UUID v4 is sufficient]
```

**Observable outcome:** When two strategies appear equal on the flowchart, prefer the one your database or ORM natively supports — operational familiarity reduces incident blast radius.

## Stage 6 — Sizing Your Namespace — The Birthday Paradox Applied to ID Collision Risk

- The birthday paradox reveals a counterintuitive truth: you do not need to fill a namespace before collisions become likely.
- You need roughly √(namespace size) records before the collision probability exceeds 50%.
- The approximation formula is: P(collision) ≈ n² / (2 × namespace size) Where n is the total number of IDs generated.
- Example: Base62 Namespace Collision Probability by Record Count The 6-char Base62 row reveals why the URL shortener must use 7 characters.

*Applies **caching** to the decision above.*

**Observable outcome:** Example: Base62 Namespace Collision Probability by Record Count The 6-char Base62 row reveals why the URL shortener must use 7 characters.

## End-to-end outcome

You have 500 million shortened URLs.

## Alternatives considered

The source focuses on one primary design. A second architecture is not compared, so no alternative is invented here.

## Failure and degraded modes

### Random vs Time-Ordered: The Tradeoff That Breaks B-Tree Indexes in Production

- **Detection:** The most common production ID mistake is using UUID v4 as a primary key and discovering the performance cliff at 50–100M rows — by which point a schema migration is painful.
- **User or operator impact:** The most common production ID mistake is using UUID v4 as a primary key and discovering the performance cliff at 50–100M rows — by which point a schema migration is painful.



## Scenario context

Fan-out is the act of distributing one post to many followers' feeds. Write-time fan-out (push) pre-computes feeds at post time — fast reads but catastrophic write amplification for celebrities. Read-time fan-out (pull) computes feeds on demand — no amplification but slow reads at scale. Production systems like Twitter/X, Instagram, and Facebook use a **hybrid**: write-time fan-out for users with ≤ 10K followers, re…

This walkthrough focuses on **The Social Feed Problem: One Post, Ten Million Inboxes**.

Before making a design choice, clarify:

- When Justin Bieber tweets, roughly 300 million accounts are eligible to see it.
- Your system has milliseconds to decide: do you immediately push that tweet into every follower's pre-built feed cache, or do you wait and assemble each follower's feed the moment they open the app?
- This is the fan-out problem — the act of distributing a single write event to many downstream consumers — and it is the central architectural decision in every social feed, notification system, and activity stream.

**Concepts in play:** caching, system-design-hld-architectures, database-indexing, event-driven-architecture

## Stage 1 — Deep Dive: Inside the Redis Feed Cache and the Async Fan-Out Pipeline

- The Internals: Redis Sorted Sets, Kafka Topics, and Fan-Out Workers The canonical feed cache data structure is a Redis sorted set ( ZSET ).
- Each follower's feed is stored as a key like feed:{userId} where each member is a postId and the score is the post's Unix timestamp in milliseconds.
- This enables: - ZADD feed:42 1711720800000 postId:9999 — insert a post reference - ZREVRANGE feed:42 0 49 WITHSCORES — fetch the 50 most recent posts - ZREMRANGEBYRANK feed:42 0 -501 — evict entries beyond a 500-post ca…
- Post content is fetched separately in a second read from a post cache or DB, allowing the feed cache to remain compact and fast.

*Applies **caching** to the decision above.*

```
Kafka Topic: post.created
  Partition key: authorId (ensures ordered delivery per author)
  Consumer Group 1: fan-out-workers (writes to Redis feed caches)
  Consumer Group 2: notification-workers (sends push notifications)
  Consumer Group 3: analytics-ingest (feeds the engagement pipeline)
```

**Observable outcome:** Post content is fetched separately in a second read from a post cache or DB, allowing the feed cache to remain compact and fast.

## Stage 2 — The Hybrid Architecture: Routing Posts by Follower Count

- The hybrid model routes the fan-out decision at post time, not at read time.
- The Fan-out Router — consuming from the post.created Kafka topic — checks the author's follower count against a configurable threshold (typically 10,000).
- Normal users fan out immediately; celebrity posts skip the write fan-out entirely and are stored in a dedicated Celebrity Post Store instead.
- The partition key for post.created should be authorId .

*Applies **system-design-hld-architectures** to the decision above.*

```java
@Configuration
public class FanOutKafkaConfig {

    @Bean
    public ConcurrentKafkaListenerContainerFactory<String, PostCreatedEvent> fanOutContainerFactory(
            ConsumerFactory<String, PostCreatedEvent> consumerFactory) {

        ConcurrentKafkaListenerContainerFactory<String, PostCreatedEvent> factory =
            new ConcurrentKafkaListenerContainerFactory<>();

        factory.setConsumerFactory(consumerFactory);
        factory.setConcurrency(8);             // 8 threads per pod, one per Kafka partition
        factory.getContainerProperties()
               .setAckMode(ContainerProperties.AckMode.MANUAL_IMMEDIATE); // Commit only after Redis write succeeds

        return factory;
    }
}

@Component
public class FanOutConsumer {

    @KafkaListener(
        topics = "post.created",
        groupId = "fan-out-workers",
        containerFactory = "fanOutContainerFactory"
    )
    public void consume(PostCreatedEvent event, Acknowledgment ack) {
        fanOutService.handlePostCreated(event);
        ack.acknowledge(); // Only commit offset once fan-out is complete
    }
}
```

**Observable outcome:** The partition key for post.created should be authorId .

## End-to-end outcome

After working through the stages, you should be able to explain:

- Forgetting to handle the celebrity path during cold-start backfill.
- When a user's feed cache expires and is rebuilt from the post DB, it is easy to only restore write-time fan-out posts.
- - Write-time fan-out (push) pre-computes follower feeds at post time using Redis sorted sets.
- Reads are O(1) but writes are O(N followers).

## Alternatives considered

### Fan-Out Fundamentals: The Push vs. Pull Decision

Fan-out refers to the branching factor of a write event: when one user posts, the event must "fan out" to all followers who should see it.

### Write-Time Fan-Out: Paying the Cost at Post Time

When a user with 500 followers posts, a fan-out worker writes 500 entries into Redis — one per follower's sorted set feed cache — with the score set to the post's Unix timestamp.

### Read-Time Fan-Out: Paying the Cost at Read Time

Read-time fan-out inverts the model entirely.


## Failure and degraded modes

The source does not isolate a separate failure mode. Limits that matter for the scenario are called out in the stage decisions above.


Practice database, caching, sharding, IDs, and fan-out choices.

Storage and Scale Patterns

## Scenario context

When you hit a URL, **DNS** translates the name to an IP, **CDNs** serve static assets from the edge nearest to you, and **Load Balancers** spread traffic across many servers so no single machine becomes a bottleneck. These three layers are the traffic control system of the modern internet.

---

In 2021, GitHub went dark for nearly two hours. The cause was a misconfigured BGP route that blackholed traffic to GitHub…

This walkthrough focuses on **Three Layers That Stand Between a User and Your Server**.

Before making a design choice, clarify:

- In 2021, GitHub went dark for nearly two hours.
- The cause was a misconfigured BGP route that blackholed traffic to GitHub's DNS servers — meaning no request could resolve github.com to an IP address at all, let alone reach an application server.
- The outage was not a code bug or a database crash; it was a failure in the three infrastructure layers that execute before your application code ever runs.

**Concepts in play:** caching, system-design-hld-architectures, load-balancing, jvm-internals-and-garbage-collection

## Stage 1 — The Building Blocks: What Each Component Does

- Every web request to your application routes through three core network layers before reaching your code.
- DNS (Domain Name System) is the phone book of the internet: it translates human-readable hostnames like api.example.com into machine-readable IP addresses.
- Without DNS, users would need to memorize IP addresses directly to reach your service.
- CDN (Content Delivery Network) is a distributed cache of static assets.

*Applies **caching** to the decision above.*

**Observable outcome:** CDN (Content Delivery Network) is a distributed cache of static assets.

## Stage 2 — DNS: The Internet's Phone Book

- DNS maps example.com → 192.0.2.1 .
- The resolution chain has four hops: 1.
- Recursive resolver (usually your ISP or 8.8.8.8) receives the query.
- Root server directs to the TLD server.

*Applies **system-design-hld-architectures** to the decision above.*

```mermaid
sequenceDiagram
    participant B as Browser
    participant OS as OS Cache
    participant RR as Recursive Resolver
    participant Root as Root NS (.)
    participant TLD as TLD NS (.com)
    participant Auth as Authoritative NS

    B->>OS: Resolve api.example.com
    OS-->>B: Cache miss
    B->>RR: Query api.example.com
    RR->>Root: Where is .com?
    Root-->>RR: TLD NS address for .com
    RR->>TLD: Where is example.com?
    TLD-->>RR: Auth NS for example.com
    RR->>Auth: A record for api.example.com?
    Auth-->>RR: 203.0.113.10 (TTL 300s)
    RR-->>B: 203.0.113.10 (cached per TTL)

```

**Observable outcome:** Root server directs to the TLD server.

## Stage 3 — CDNs: Bringing Content Closer to Users

- A CDN is a globally distributed cache.
- When a user requests /static/logo.png , the CDN serves it from an edge PoP (Point of Presence) in their city rather than your origin server in a distant datacenter.
- CDN hit ratio is the fraction of requests served from cache: $$ ext{Avg Latency} = r \cdot L ext{edge} + (1 - r) \cdot L ext{origin}$$ where $r$ is the hit ratio.
- Even improving $r$ from 0.80 to 0.95 dramatically reduces effective latency.

*Applies **load-balancing** to the decision above.*

**Observable outcome:** Even improving $r$ from 0.80 to 0.95 dramatically reduces effective latency.

## Stage 4 — Load Balancers: Distributing Traffic Intelligently

- A load balancer sits in front of your server pool and routes incoming requests: Layer 4 vs Layer 7: Routing algorithms: Health checks probe each backend (HTTP /health , TCP ping) on a configurable interval.
- Unhealthy servers are removed from rotation without human intervention.
- Load imbalance metric: $$ ext{Imbalance} = rac{\max( ext{server QPS})}{ ext{avg}( ext{server QPS})}$$ A ratio near 1.0 means excellent distribution; above 2.0 is a warning sign.
- 📊 CDN Cache Miss and Hit Flow This sequence diagram contrasts a CDN cache miss on the first request with a cache hit on the second request for the same static asset.

*Applies **jvm-internals-and-garbage-collection** to the decision above.*

```mermaid
flowchart TD
    A[User Browser] -->|DNS lookup returns LB IP| B[Load Balancer L7]
    B -->|Route /api| C[App Server 1]
    B -->|Route /api| D[App Server 2]
    B -->|Route /static| E[CDN Edge Node]
    C --> F[Origin DB]
    D --> F

```

```mermaid
sequenceDiagram
    participant C as Client
    participant E as CDN Edge PoP
    participant O as Origin Server

    C->>E: GET /static/logo.png
    Note over E: Cache MISS (first request)
    E->>O: Forward to origin
    O-->>E: 200 OK + logo.png payload
    E-->>C: 200 OK (served + cached at edge)
    Note over E: Cached for max-age=86400s

    C->>E: GET /static/logo.png
    Note over E: Cache HIT
    E-->>C: 200 OK (from edge, ~5ms)
    Note over O: Origin not contacted on hit

```

**Observable outcome:** 📊 CDN Cache Miss and Hit Flow This sequence diagram contrasts a CDN cache miss on the first request with a cache hit on the second request for the same static asset.

## Stage 5 — Deep Dive: Inside a DNS Resolution and Cache Lifecycle

- When you type a URL, the OS checks its local DNS cache first.
- On a miss, the query travels to a recursive resolver, which fans out to root → TLD → authoritative nameserver.
- Each hop adds a few milliseconds — but the result is cached per TTL, so repeat visits skip all hops.
- This is why TTL tuning matters: a 24-hour TTL means stale IPs persist for 24 hours after a server change.

*Applies **caching** to the decision above.*

**Observable outcome:** This is why TTL tuning matters: a 24-hour TTL means stale IPs persist for 24 hours after a server change.

## Stage 6 — The Full Request Journey: DNS → CDN → Load Balancer → App

- This diagram shows how a single user interaction splits into two parallel paths: the API request travels through DNS resolution to the load balancer and then to an app server, while the static asset request is intercept…
- Notice that the CDN request never touches the load balancer or app server — this is the core value proposition of a CDN, removing an entire class of requests from the origin.
- The combined effect of DNS caching, edge caching, and load-balanced app servers is what allows a modest backend cluster to serve millions of users globally.

*Applies **system-design-hld-architectures** to the decision above.*

```mermaid
sequenceDiagram
    participant U as User Browser
    participant R as Recursive Resolver
    participant A as Authoritative NS
    participant LB as Load Balancer
    participant CDN as CDN Edge
    participant App as App Server

    U->>R: Resolve example.com
    R->>A: Query A record
    A-->>R: Returns LB IP
    R-->>U: LB IP (cached per TTL)
    U->>LB: GET /api/data
    LB->>App: Forward request
    App-->>LB: JSON response
    LB-->>U: JSON response
    U->>CDN: GET /static/logo.png
    CDN-->>U: Serve from edge cache
```

**Observable outcome:** The combined effect of DNS caching, edge caching, and load-balanced app servers is what allows a modest backend cluster to serve millions of users globally.

## End-to-end outcome

In 2021, GitHub went dark for nearly two hours.

## Alternatives considered

### Decision Guide: When to Use What

Decision Guide: When to Use What


## Failure and degraded modes

### Trade-offs & Failure Modes

- **Detection:** Trade-offs & Failure Modes
- **User or operator impact:** Trade-offs & Failure Modes



## Scenario context

Message queues and event-driven architecture let services communicate asynchronously, absorb bursty traffic, and isolate failures. The core design challenge is not adding a queue — it is defining delivery semantics, retry behavior, and idempotent consumers.






Queues are not a default replacement for APIs. They are a pressure-relief boundary when synchronous chains become fragile under burst traffic or partial ou…

This walkthrough focuses on **When to Use Event-Driven Queues and When Not To**.

Before making a design choice, clarify:

- When to use - Work is naturally asynchronous: notifications, enrichment, indexing, billing reconciliation.
- - Producer traffic is bursty, but outcomes can be delayed by seconds or minutes.
- - You need durable handoff between independently scaled services.

**Concepts in play:** message-queues, stack-and-queue-data-structures, event-driven-architecture, consistency-replication

## Stage 1 — How the Pattern Works: Producer, Broker, Consumer, Recovery

- The practical flow is simple to explain and strict to implement.
- Producer publishes an event with stable schema and idempotency key.
- Broker persists and routes events by topic/partition.
- Consumer processes event side effects.

*Applies **message-queues** to the decision above.*

**Observable outcome:** Consumer processes event side effects.

## Stage 2 — Deep Dive: Internals That Make or Break Async Reliability

- Internals: Ordering, Acknowledgment, and Schema Evolution Ordering is partition-scoped, not global.
- If one business entity needs strict order, all events for that entity must land on the same partition key.
- Acknowledgment strategy determines correctness: 1.
- If you ack before step 3, crashes can lose work.

*Applies **stack-and-queue-data-structures** to the decision above.*

**Observable outcome:** If you ack before step 3, crashes can lose work.

## Stage 3 — Event Pipeline Flow: Publish, Process, Retry, and Recover

- This is the minimum viable reliability loop.
- If any node is missing, incident load shifts to manual cleanup.
- Every path ends in a safe ack or DLQ route — there is no silent discard.
- 📊 Pub-Sub: Publisher to Subscribers This pub-sub sequence diagram shows how a single OrderCreated event published to a topic fans out simultaneously to two independent subscriber services, each of which processes the e…

*Applies **event-driven-architecture** to the decision above.*

```mermaid



flowchart TD
    A[Producer validates and publishes event] --> B[Broker topic partition]
    B --> C[Consumer reads event]
    C --> D{Idempotency key already processed?}
    D -->|Yes| E[Ack and skip duplicate]
    D -->|No| F[Execute side effect]
    F --> G{Success?}
    G -->|Yes| H[Record dedupe key then ack]
    G -->|No| I[Retry with backoff]
    I --> J{Retry limit reached?}
    J -->|No| C
    J -->|Yes| K[Route to DLQ and alert owner]

```

```mermaid
sequenceDiagram
    participant P as Publisher
    participant T as Topic
    participant S1 as EmailService
    participant S2 as InventoryService
    P->>T: Publish OrderCreated event
    T->>S1: Deliver to EmailService
    T->>S2: Deliver to InventoryService
    S1-->>T: Ack
    S2-->>T: Ack

```

**Observable outcome:** 📊 Pub-Sub: Publisher to Subscribers This pub-sub sequence diagram shows how a single OrderCreated event published to a topic fans out simultaneously to two independent subscriber services, each of which processes the e…

## Stage 4 — Real-World Applications: Flash-Sale Checkout Under Hard Constraints

- Scenario constraints: - 70k checkouts/minute during a 10-minute flash sale.
- - Payment provider has 2% transient timeout rate.
- - Inventory updates must be per-item ordered.
- - Duplicate charge rate must remain under 0.01%.

*Applies **consistency-replication** to the decision above.*

**Observable outcome:** - Duplicate charge rate must remain under 0.01%.

## Stage 5 — Lessons From Running Async Systems in Production

- - Queue adoption is only successful when correctness rules are explicit before the first message is sent.
- - Idempotent consumers are mandatory for at-least-once delivery — not optional.
- - Partition strategy is a business decision, not only a scaling decision.
- Order guarantees map to business entities.

*Applies **message-queues** to the decision above.*

**Observable outcome:** Order guarantees map to business entities.

## End-to-end outcome

When to use - Work is naturally asynchronous: notifications, enrichment, indexing, billing reconciliation.

## Alternatives considered

### Decision Guide: Queues vs Synchronous Calls in Architecture Reviews

The retry and dead-letter semantics in AMQP are expressed through the acknowledgment itself.


## Failure and degraded modes

### When to use

- **Detection:** - Work is naturally asynchronous: notifications, enrichment, indexing, billing reconciliation.
- **User or operator impact:** - Work is naturally asynchronous: notifications, enrichment, indexing, billing reconciliation.

### When not to use

- **Detection:** - A user-facing action needs immediate, deterministic completion.
- **User or operator impact:** - A user-facing action needs immediate, deterministic completion.

### Trade-offs & Failure Modes: Trade-offs and Failure Modes: Queue-Centric Design Risks

- **Detection:** Trade-offs & Failure Modes: Trade-offs and Failure Modes: Queue-Centric Design Risks
- **User or operator impact:** Trade-offs & Failure Modes: Trade-offs and Failure Modes: Queue-Centric Design Risks



## Scenario context

Service discovery is how clients find the right service instance at runtime, and health checks are how systems decide whether an instance should receive traffic. Together, they turn dynamic infrastructure from guesswork into deterministic routing.

> **TLDR**: If you scale beyond static IPs, discovery plus health-aware routing becomes a core reliability primitive.

In small systems, service communication can start w…

This walkthrough focuses on **Why Service Discovery Is the Invisible Backbone of Modern Systems**.

Before making a design choice, clarify:

- In small systems, service communication can start with fixed hostnames and static configuration.
- That model breaks quickly once autoscaling, rolling deploys, and multi-zone failover enter the picture.
- In production, service instances come and go all day: - New instances launch during traffic spikes.

**Concepts in play:** api-gateway, database-indexing, caching, canary-deployments

## Stage 1 — The Two Discovery Models You Must Distinguish in Interviews

- Service discovery usually appears in one of two patterns.
- Client-side discovery: the client queries a service registry and chooses a backend instance directly.
- This is common in microservice SDKs where clients include load-balancing logic.
- Server-side discovery: the client calls a stable endpoint (for example, a load balancer or API gateway), and that component resolves healthy backends.

*Applies **api-gateway** to the decision above.*

**Observable outcome:** Server-side discovery: the client calls a stable endpoint (for example, a load balancer or API gateway), and that component resolves healthy backends.

## Stage 2 — How Discovery and Health Checks Work End-to-End

- A robust discovery path is usually a loop, not a one-time lookup.
- Service instance starts and registers itself with metadata.
- Registry stores endpoint, zone, version, and status.
- Clients or proxies query for candidate instances.

*Applies **database-indexing** to the decision above.*

**Observable outcome:** Clients or proxies query for candidate instances.

## Stage 3 — Discovery Flow: Registration to Health-Aware Routing

- This model captures the key principle: discovery and health checks are continuous control loops, not setup-time configuration.
- 📊 Service Registration and Client Discovery This sequence diagram traces the full lifecycle of service registration and client-driven discovery.
- OrderService registers with the registry and sends heartbeats every 5 seconds to maintain its healthy status; when a Client queries for available endpoints, the registry returns only healthy instances, and the Client ro…
- Discovery is not a one-time lookup — it is a continuous health-maintained contract that ensures clients never route to stale or unhealthy backends.

*Applies **caching** to the decision above.*

```mermaid



flowchart TD
    A[Instance boots] --> B[Register with service registry]
    B --> C[Heartbeat and metadata updates]
    C --> D{Healthy and ready?}
    D -->|Yes| E[Add to routing pool]
    D -->|No| F[Exclude from routing pool]
    E --> G[Client or proxy resolves target]
    G --> H[Request served]
    F --> I[Recovery or restart]
    I --> C

```

```mermaid
sequenceDiagram
    participant S as OrderService
    participant R as Service Registry
    participant C as Client
    S->>R: Register: host port tags
    R-->>S: ACK registration
    S->>R: Heartbeat every 5s
    C->>R: Discover order-service
    R-->>C: Return healthy endpoints
    C->>S: Route request
    S-->>C: Response

```

**Observable outcome:** Discovery is not a one-time lookup — it is a continuous health-maintained contract that ensures clients never route to stale or unhealthy backends.

## Stage 4 — Real-World Applications: API Gateways, Payments, and Internal Platforms

- HashiCorp Consul at scale: Consul's gossip protocol propagates health changes across a cluster in ~200ms on typical LAN deployments.
- The deregister critical service after field automatically removes services that remain unhealthy beyond a configurable window — preventing stale endpoints from silently accumulating in the registry.
- When a service registers with Consul, it provides its name, port, and metadata tags alongside a health check definition.
- The check specifies the HTTP path to poll, the polling interval, the per-check timeout, and — critically — a window after which Consul automatically deregisters a service that remains in a critical state.

*Applies **canary-deployments** to the decision above.*

**Observable outcome:** The check specifies the HTTP path to poll, the polling interval, the per-check timeout, and — critically — a window after which Consul automatically deregisters a service that remains in a critical state.

## Stage 5 — Practical Example: Evolving a Checkout Service Beyond Static Backends

- Imagine a checkout service initially routed via hardcoded backend IPs.
- Problems appear during traffic spikes: - New app instances launch but receive no traffic.
- - One bad instance still receives requests for minutes.
- - Rolling deploys create intermittent errors from stale endpoint lists.

*Applies **api-gateway** to the decision above.*

**Observable outcome:** - Rolling deploys create intermittent errors from stale endpoint lists.

## Stage 6 — Dynamic Registration and Health-Aware Discovery in Production

- Spring Cloud Netflix Eureka is a client-side service registry built into the Spring Cloud ecosystem; Spring Cloud Consul provides the same programming model backed by HashiCorp Consul's gossip-based registry.
- Both integrate with a discovery client abstraction and a health indicator framework to expose liveness and readiness semantics to the control plane automatically.
- How it works in practice: A microservice opts into dynamic discovery by declaring itself as a discovery client at startup.
- From that point, registration and heartbeat renewal happen automatically — the developer does not manage registry calls manually.

*Applies **database-indexing** to the decision above.*

**Observable outcome:** From that point, registration and heartbeat renewal happen automatically — the developer does not manage registry calls manually.

## End-to-end outcome

After working through the stages, you should be able to explain:

- - Service discovery is a control-plane capability, not just a DNS trick.
- - Health checks must distinguish process liveness from real request readiness.

## Alternatives considered

### Decision Guide: Choosing a Discovery Strategy

When unsure in interviews, start with server-side discovery for simpler client behavior, then discuss where client-side control may be worth the complexity.


## Failure and degraded modes

### The Internals: Registries, Heartbeats, TTLs, and Routing Metadata

- **Detection:** Most systems maintain a control plane with these pieces: - Registry store for service instances and metadata.
- **User or operator impact:** Most systems maintain a control plane with these pieces: - Registry store for service instances and metadata.

### Performance Analysis: Lookup Latency, Convergence Time, and Flapping

- **Detection:** Discovery systems are often judged by three metrics.
- **User or operator impact:** Discovery systems are often judged by three metrics.

### Trade-offs & Failure Modes: Where Discovery Can Go Wrong

- **Detection:** The interview-quality answer always includes one sentence like: "I would define clear health semantics and failure thresholds before tuning load-balancer algorithms."
- **User or operator impact:** The interview-quality answer always includes one sentence like: "I would define clear health semantics and failure thresholds before tuning load-balancer algorithms."



## Scenario context

Replication means keeping multiple copies of your data so the system can survive machine, process, or availability-zone failures. Failover is the coordinated act of promoting a healthy replica, rerouting traffic, and recovering without corrupting data.

> **TLDR**: If one database going down takes your product down, replication is the next design step. Failover is how you make that redundancy usable.






**How it…

This walkthrough focuses on **Lessons Learned**.

Before making a design choice, clarify:

- - Replication and failover are separate ideas: one copies data, the other restores authority.
- - Read replicas help scale reads, but they do not guarantee fresh data.
- - Asynchronous replication reduces write latency but can lose the newest acknowledged data during a crash.

**Concepts in play:** consistency-replication, system-design-hld-architectures, transaction-isolation-distributed-acid

## End-to-end outcome

After working through the stages, you should be able to explain:

- - Replication and failover are separate ideas: one copies data, the other restores authority.
- - Read replicas help scale reads, but they do not guarantee fresh data.

## Alternatives considered

The source focuses on one primary design. A second architecture is not compared, so no alternative is invented here.

## Failure and degraded modes

The source does not isolate a separate failure mode. Limits that matter for the scenario are called out in the stage decisions above.


## Scenario context

Observability is how you understand system behavior from telemetry, SLOs are explicit reliability targets, and incident response is the execution model when those targets are at risk. Together, they convert operational chaos into measurable, repeatable decision-making.

> **TLDR**: If your architecture has no observability and no SLOs, you do not have reliability engineering, only hopeful monitoring.

Many system de…

This walkthrough focuses on **Why Reliability Conversations Fail Without Observability and SLOs**.

Before making a design choice, clarify:

- Many system design answers stop at infrastructure choices: load balancers, replicas, caches, queues.
- Those components matter, but they do not tell you when users are actually suffering.
- Reliability is fundamentally an outcomes problem: - Are requests succeeding for users?

**Concepts in play:** api-gateway, system-design-hld-architectures, database-indexing, tree-data-structures

## Stage 1 — The Observability Pillars and SLO Vocabulary You Should Use Precisely

- A practical observability model includes: - Metrics for trend and alert thresholds.
- - Logs for event context and forensic detail.
- - Traces for request path latency and dependency attribution.
- SLO language adds decision clarity: - SLI (Service Level Indicator): measured behavior (for example, request success rate).

*Applies **api-gateway** to the decision above.*

**Observable outcome:** SLO language adds decision clarity: - SLI (Service Level Indicator): measured behavior (for example, request success rate).

## Stage 2 — How Telemetry and SLOs Drive Incident Prioritization

- A healthy reliability loop often looks like this: 1.
- Instrument critical user journeys with metrics and traces.
- Define SLOs on user-impacting paths.
- Alert on burn-rate of error budget, not raw noise.

*Applies **system-design-hld-architectures** to the decision above.*

**Observable outcome:** Alert on burn-rate of error budget, not raw noise.

## Stage 3 — Deep Dive: The Mechanics of Incident-Ready Reliability Engineering

- The Internals: Telemetry Pipelines, Correlation IDs, and Ownership Observability architecture usually has these layers: - Instrumented applications emitting metrics, logs, and traces.
- - Collection agents and pipelines.
- - Storage/index systems with retention policies.
- - Query and dashboard surfaces.

*Applies **database-indexing** to the decision above.*

**Observable outcome:** - Query and dashboard surfaces.

## Stage 4 — Reliability Loop: Measure, Detect, Respond, Improve

- This loop reflects mature operations: reliability is iterative and continuously measured, not fixed once at deploy time.
- 📊 Alert Lifecycle: Metric Spike to Incident The diagram traces the full alert lifecycle from a metrics anomaly to resolution.
- An error rate spike triggers the Alert Engine to evaluate the burn-rate rule; when it fires a FastBurn alert, PagerDuty pages the on-call engineer, who opens an incident channel, inspects traces and logs, and mitigates…
- Each handoff has a clearly defined owner, turning a noisy signal into a resolved incident with structured, predictable escalation rather than reactive firefighting.

*Applies **tree-data-structures** to the decision above.*

```mermaid
flowchart TD
    A[Instrument services] --> B[Collect metrics logs traces]
    B --> C[Evaluate SLI and SLO windows]
    C --> D{Error budget burn high?}
    D -->|No| E[Continue monitoring]
    D -->|Yes| F[Trigger incident response]
    F --> G[Mitigate and restore service]
    G --> H[Post-incident review and action items]
    H --> A

```

```mermaid
sequenceDiagram
    participant M as Metrics
    participant A as Alert Engine
    participant P as PagerDuty
    participant OC as On-Call Engineer
    participant INC as Incident Channel
    M->>A: Error rate spike
    A->>A: Check burn-rate rule
    A->>P: Fire: FastBurn alert
    P->>OC: Page on-call
    OC->>INC: Open incident
    OC->>M: Inspect traces and logs
    OC->>INC: Mitigate and resolve
    INC->>OC: Post-incident review

```

**Observable outcome:** Each handoff has a clearly defined owner, turning a noisy signal into a resolved incident with structured, predictable escalation rather than reactive firefighting.

## Stage 5 — Real-World Applications: Checkout APIs, Search, and Platform Services

- Google SRE (the burn-rate algebra): Google's SRE book formalized error budgets with concrete math.
- A 99.9% monthly SLO allows 43.8 minutes of downtime per month .
- At a 14× normal error rate , one hour of weekly budget burns in just 5 minutes — the threshold that justifies waking someone up at 3 AM.
- At a 1× steady-state rate , the monthly budget exhausts in 30 days — a ticket, not a page.

*Applies **api-gateway** to the decision above.*

**Observable outcome:** At a 1× steady-state rate , the monthly budget exhausts in 30 days — a ticket, not a page.

## Stage 6 — Practical Example: Designing SLOs for an Orders API

- Suppose you run an orders API with these user-critical outcomes: - Place order successfully.
- - Retrieve order status quickly.
- A practical first SLO set: Incident policy example: 1.
- If fast burn-rate exceeds threshold, page primary on-call.

*Applies **system-design-hld-architectures** to the decision above.*

**Observable outcome:** If fast burn-rate exceeds threshold, page primary on-call.

## End-to-end outcome

After working through the stages, you should be able to explain:

- - Observability is useful only when tied to user-impacting objectives.
- - SLOs convert reliability debates into measurable trade-offs.

## Alternatives considered

### Decision Guide: How Much Observability Is Enough?

In interview settings, prioritize user-impacting SLIs first.


## Failure and degraded modes

### Trade-offs & Failure Modes: Common Observability Mistakes

- **Detection:** Strong interview answers include both the technical and organizational side of incident response.
- **User or operator impact:** Strong interview answers include both the technical and organizational side of incident response.



Add networking, queues, service discovery, failover, and observability.

Communication and Reliability

## Scenario context

A URL shortener is a read-heavy system (100:1 ratio) that maps long URLs to short, unique aliases. The core scaling challenge is generating unique IDs without database contention—solved using a **Range-Based ID Generator** or a **Distributed Counter** with Base62 encoding. A 301/302 redirect strategy combined with aggressive Redis caching ensures sub-10ms response times.


```architecture
{
  "version": 1,
  "title"…

This walkthrough focuses on **Lessons Learned from Production URL Shortener Systems**.

Before making a design choice, clarify:

- In production deployments, the top 1% of links receive over 95% of redirect traffic.
- Standard LRU caching handles this well under steady state, but viral link events — a celebrity post, a product launch — can cause sudden cache misses on previously cold links.
- Production teams implement traffic spike detection that proactively pre-warms cache entries for links crossing a sudden request-rate threshold, preventing the cold-start penalty during viral moments.

**Concepts in play:** caching, system-design-hld-architectures, distributed-locking, event-driven-architecture

## Stage 1 — Key Takeaways: URL Shortener System Design

- - A URL shortener is a read-heavy system (100:1 read/write ratio).
- Optimize every design decision for the redirect read path first; write performance is secondary.
- - Base62 encoding with 6 characters covers 56 billion unique URLs — sufficient for any real-world deployment over the next 47 years.
- - Range-Based Allocation solves the distributed counter bottleneck: each server claims a block of IDs from ZooKeeper and increments an in-memory counter locally, with zero per-request network coordination.

*Applies **caching** to the decision above.*

**Observable outcome:** - Range-Based Allocation solves the distributed counter bottleneck: each server claims a block of IDs from ZooKeeper and increments an in-memory counter locally, with zero per-request network coordination.

## End-to-end outcome

After working through the stages, you should be able to explain:

- In production deployments, the top 1% of links receive over 95% of redirect traffic.
- Standard LRU caching handles this well under steady state, but viral link events — a celebrity post, a product launch — can cause sudden cache misses on previously cold links.
- - A URL shortener is a read-heavy system (100:1 read/write ratio).
- Optimize every design decision for the redirect read path first; write performance is secondary.

## Alternatives considered

The source focuses on one primary design. A second architecture is not compared, so no alternative is invented here.

## Failure and degraded modes

The source does not isolate a separate failure mode. Limits that matter for the scenario are called out in the stage decisions above.


## Scenario context

Search autocomplete must respond in sub-10ms to feel "instant." The core trade-off is **Latency vs. Data Freshness**: we use an offline pipeline (Spark) to pre-calculate prefix-to-suggestion mappings and store them in **Redis Sorted Sets** (or a specialized Trie) for near-instant retrieval, while sacrificing real-time updates for most queries.

Imagine you are typing in a search bar: "best res..."
You expect the sys…

This walkthrough focuses on **The "Keystroke" Performance Gap**.

Before making a design choice, clarify:

- Imagine you are typing in a search bar: "best res..." You expect the system to suggest "best restaurants" before you even finish the word "restaurants." An average person types at about 60-80 words per minute, which mea…
- In a poorly designed system, every keystroke triggers a SQL query: SELECT query FROM search logs WHERE query LIKE 'best res%' ORDER BY count DESC LIMIT 5 .
- Under the weight of 100,000 concurrent users, this LIKE query will crush your database.

**Concepts in play:** caching, tree-data-structures, database-indexing, system-design-hld-architectures

## Stage 1 — Search Autocomplete: Use Cases & Requirements

- Actors - Search User: Expects instant, relevant suggestions as they type.
- - Data Scientist / Analyst: Fine-tunes ranking algorithms based on click-through rates.
- - Admin: Manages blacklists for inappropriate or harmful suggestions.
- Functional Requirements - Prefix Matching: Given a prefix (e.g., "ap"), return the top 5-10 most popular suggestions (e.g., "apple", "amazon").

*Applies **caching** to the decision above.*

**Observable outcome:** Functional Requirements - Prefix Matching: Given a prefix (e.g., "ap"), return the top 5-10 most popular suggestions (e.g., "apple", "amazon").

## Stage 2 — Basics: The Trie Data Structure

- At the heart of most autocomplete systems is a Trie (Prefix Tree).
- A Trie is a tree-like data structure where each node represents a character.
- - To find suggestions for "app", you traverse the tree: a - p - p .
- - Every node can store the top 10 most popular queries that pass through it.

*Applies **tree-data-structures** to the decision above.*

**Observable outcome:** - Every node can store the top 10 most popular queries that pass through it.

## Stage 3 — Mechanics: Distribution & Frequency Aggregation

- We don't compute suggestions on-the-fly.
- Instead, we use a Data Ingestion Pipeline : 1.
- Log Collection: Every time a user submits a full search, we log it to a message queue (Kafka).
- Aggregation: A background job (Spark/Flink) aggregates these logs hourly/daily to calculate the "Frequency" of each query.

*Applies **database-indexing** to the decision above.*

**Observable outcome:** Aggregation: A background job (Spark/Flink) aggregates these logs hourly/daily to calculate the "Frequency" of each query.

## Stage 4 — Estimations & Design Goals

- The Math of Search - Daily Active Users (DAU): 100 Million.
- - Total Autocomplete Requests: 100M 10 20 = 20 Billion per day .
- - Average QPS: 20B / 86,400 $\approx 230,000$ requests per second.
- Design Goal: The system must be stateless and horizontally scalable .

*Applies **system-design-hld-architectures** to the decision above.*

**Observable outcome:** Design Goal: The system must be stateless and horizontally scalable .

## Stage 5 — High-Level Design: The Two-Path Architecture

- The system is split into the "Fast Path" (Serving) and the "Slow Path" (Analytics).
- The diagram shows the two-path architecture clearly.
- The Fast Path (top) serves user keystrokes directly from the Redis Sorted Set cache with a single ZREVRANGEBYSCORE call — the entire backend lookup completes in under 2ms.
- The Slow Path (bottom) records completed searches to Kafka asynchronously, aggregates frequency counts in Spark on an hourly or daily schedule, and rebuilds the suggestion index in Redis.

*Applies **caching** to the decision above.*

```architecture
{
  "version": 1,
  "title": "High-Level Design: The Two-Path Architecture - Subsystem Overview",
  "description": "Decoupled system architecture isolating ingress, stateless logic, and persistent datastores.",
  "view": "high-level",
  "pattern": "microservices",
  "direction": "LR",
  "groups": [
    { "id": "edge", "label": "Edge Traffic Ingress", "tone": "slate" },
    { "id": "services", "label": "Stateless Microservices", "tone": "blue" },
    { "id": "data", "label": "Datastore Infrastructure", "tone": "green" }
  ],
  "nodes": [
    { "id": "gw", "component": "api-gateway", "label": "API Gateway", "group": "edge", "badge": "TLS & Auth" },
    { "id": "app", "component": "workflow-service", "label": "Domain Service", "group": "services", "badge": "Stateless" },
    { "id": "cache", "component": "redis", "label": "Redis Cache", "group": "data", "badge": "Sub-ms Reads" },
    { "id": "db", "component": "postgres", "label": "Transactional DB", "group": "data", "badge": "ACID Log" }
  ],
  "edges": [
    { "from": "gw", "to": "app", "kind": "request", "label": "gRPC Request" },
    { "from": "app", "to": "cache", "kind": "read", "label": "Cache Lookups" },
    { "from": "app", "to": "db", "kind": "write", "label": "Durable Write" }
  ],
  "takeaway": {
    "title": "Architecture Boundary Invariant",
    "body": "Stateless compute nodes scale horizontally while shielding underlying database clusters behind fast cache layers."
  }
}
```

```mermaid
graph TD
    User((User)) --> LB[Load Balancer]
    LB --> AS[Autocomplete Service]
    
    subgraph Serving_Fast_Path
        AS --> RC[(Suggestion Cache: Redis)]
    end
    
    subgraph Ingestion_Slow_Path
        AS --> MQ[Kafka]
        MQ --> Agg[Aggregation Service: Spark]
        Agg --> PDB[(Search History: Postgres)]
        Agg --> IB[Index Builder]
        IB --> RC
    end
```

**Observable outcome:** The Slow Path (bottom) records completed searches to Kafka asynchronously, aggregates frequency counts in Spark on an hourly or daily schedule, and rebuilds the suggestion index in Redis.

## Stage 6 — Real-World Autocomplete Systems: Google, Amazon, and Elasticsearch

- Google Suggest operates at 230,000 QPS with suggestions updated within 15 minutes of trending events.
- Google's implementation uses a distributed Trie sharded by geographic region and query language.
- Each Trie shard serves a subset of the prefix space (e.g., all prefixes starting with characters A–M in English for the US West Coast region).
- This limits Trie size to under 10GB per shard while achieving sub-1ms traversal latency within the data center.

*Applies **tree-data-structures** to the decision above.*

**Observable outcome:** This limits Trie size to under 10GB per shard while achieving sub-1ms traversal latency within the data center.

## End-to-end outcome

Imagine you are typing in a search bar: "best res..." You expect the system to suggest "best restaurants" before you even finish the word "restaurants." An average person types at about 60-80 words per minute, which means there are only about 100-150 milliseconds between keystrokes.

## Alternatives considered

### The three hardest engineering problems in search autocomplete are choosing the right data structure for sub-10ms prefix lookup, handling typos without a full-text index, and personalizing results without violating the latency budget.

The three hardest engineering problems in search autocomplete are choosing the right data structure for sub-10ms prefix lookup, handling typos without a full-text index, and personalizing results without violating the latency budget.

### Internals: Trie vs. Redis Sorted Set — When Each Wins

A Trie (prefix tree) stores suggestions in a tree where each node represents a character.

### Performance Analysis: Offline Aggregation Pipeline and Index Freshness Budget

The serving latency of under 2ms is achievable only because the suggestion index is pre-computed offline .

### Decision Guide: Serving Strategy by System Scale

Typo tolerance should never be in the critical serving path.


## Failure and degraded modes

### Trade-offs and Failure Modes in Search Autocomplete Design

- **Detection:** Zero-Result Prefix: When a user types a prefix that has no matching entry in the suggestion index (either because it is genuinely rare or because the index hasn't been updated yet), the system must g…
- **User or operator impact:** Zero-Result Prefix: When a user types a prefix that has no matching entry in the suggestion index (either because it is genuinely rare or because the index hasn't been updated yet), the system must g…

### Lessons Learned from Autocomplete Failures at Scale

- **Detection:** A Single Hot Key Took Down the Suggestion Service During a Product Launch.
- **User or operator impact:** A Single Hot Key Took Down the Suggestion Service During a Product Launch.



## Scenario context

Distributed caches trade strict consistency for sub-millisecond read latency, using consistent hashing to scale horizontally without causing database-shattering "cache stampedes" during cluster rebalancing.

Instagram's primary database once served user profile reads at 28,000 QPS until a single viral post triggered a "cache miss storm." This storm drove the database to 95% CPU utilization in seconds, threatening to…

This walkthrough focuses on **Distributed Cache: Use Cases & Requirements**.

Before making a design choice, clarify:

- A distributed cache is an in-memory, key-value store shared across multiple servers.
- Functional Requirements - Standard Operations : Support GET(key) , SET(key, value, TTL) , and DELETE(key) .
- - TTL (Time-To-Live) : Support per-key expiration to ensure data doesn't stay stale forever.

**Concepts in play:** caching, system-design-hld-architectures, consistency-replication, hash-tables-and-bloom-filters

## Stage 1 — Basics: Baseline Architecture

- In a distributed environment, the "Cache" is not one server, but a cluster of servers.
- Cache Client : A library integrated into the application that handles the logic of where to find a key.
- Cache Nodes : Independent servers (like Redis or Memcached instances) that store data in RAM.
- Consistency Mechanism : Logic to ensure that when a database record is updated, the corresponding cache entry is invalidated or updated.

*Applies **caching** to the decision above.*

```architecture
{
  "version": 1,
  "title": "Basics: Baseline Architecture - Subsystem Overview",
  "description": "Decoupled system architecture isolating ingress, stateless logic, and persistent datastores.",
  "view": "high-level",
  "pattern": "microservices",
  "direction": "LR",
  "groups": [
    { "id": "edge", "label": "Edge Traffic Ingress", "tone": "slate" },
    { "id": "services", "label": "Stateless Microservices", "tone": "blue" },
    { "id": "data", "label": "Datastore Infrastructure", "tone": "green" }
  ],
  "nodes": [
    { "id": "gw", "component": "api-gateway", "label": "API Gateway", "group": "edge", "badge": "TLS & Auth" },
    { "id": "app", "component": "workflow-service", "label": "Domain Service", "group": "services", "badge": "Stateless" },
    { "id": "cache", "component": "redis", "label": "Redis Cache", "group": "data", "badge": "Sub-ms Reads" },
    { "id": "db", "component": "postgres", "label": "Transactional DB", "group": "data", "badge": "ACID Log" }
  ],
  "edges": [
    { "from": "gw", "to": "app", "kind": "request", "label": "gRPC Request" },
    { "from": "app", "to": "cache", "kind": "read", "label": "Cache Lookups" },
    { "from": "app", "to": "db", "kind": "write", "label": "Durable Write" }
  ],
  "takeaway": {
    "title": "Architecture Boundary Invariant",
    "body": "Stateless compute nodes scale horizontally while shielding underlying database clusters behind fast cache layers."
  }
}
```

**Observable outcome:** Consistency Mechanism : Logic to ensure that when a database record is updated, the corresponding cache entry is invalidated or updated.

## Stage 2 — Mechanics: Key Logic

- Consistent Hashing The core challenge is: Which node stores key "user:123"?
- - Naive approach : hash(key) % N .
- If you add a node (N becomes N+1), almost every key's location changes.
- This causes a 100% cache miss rate — a "Cache Stampede." - Consistent Hashing : Keys and nodes are mapped onto a logical circle (the "Ring").

*Applies **system-design-hld-architectures** to the decision above.*

**Observable outcome:** This causes a 100% cache miss rate — a "Cache Stampede." - Consistent Hashing : Keys and nodes are mapped onto a logical circle (the "Ring").

## Stage 3 — Estimations & Design Goals

- - QPS : 10M reads/sec, 1M writes/sec.
- - Storage : 100 nodes 128GB RAM = 12.8TB total capacity.
- - Object Size : Average 2KB per object.
- A 1% drop in hit rate at 10M QPS adds 100,000 queries per second to the database.

*Applies **consistency-replication** to the decision above.*

**Observable outcome:** A 1% drop in hit rate at 10M QPS adds 100,000 queries per second to the database.

## Stage 4 — High-Level Design

- architecture { "version": 1, "title": "High-Level Design - Subsystem Overview", "description": "Decoupled system architecture isolating ingress, stateless logic, and persistent datastores.", "view": "high-level", "patte…
- The application checks the Cache Client Library first; the library hashes the key and consults the Consistent Hash Ring to determine which physical node owns that key.
- On a miss (step 6), the application queries the database directly, and the result is written back to the owning cache node (step 8).
- This backfill ensures that subsequent reads for the same key are served from cache without touching the database.

*Applies **hash-tables-and-bloom-filters** to the decision above.*

```mermaid

```

```

graph LR
    App[Application Service] -->|1. Get Key| Client[Cache Client Library]
    Client -->|2. Hash Key| Ring((Consistent Hash Ring))
    Ring -->|3. Route| NodeA[Node A]
    Ring -->|4. Route| NodeB[Node B]
    Ring -->|5. Route| NodeC[Node C]
    
    App -->|6. Miss| DB[(Database)]
    DB -->|7. Backfill| Client
    Client -->|8. Set Key| NodeA
```

**Observable outcome:** This backfill ensures that subsequent reads for the same key are served from cache without touching the database.

## Stage 5 — Deep Dive: Consistent Hashing Ring Internals and LRU Eviction Performance Under Scale

- Two mechanisms dominate the engineering complexity of a distributed cache: determining which node owns a key without causing re-hashing storms when topology changes, and deciding which keys to evict when RAM is full whi…
- Internals: The Consistent Hash Ring and Virtual Nodes Naive modulo sharding — hash(key) % N — has a fatal flaw: when the node count N changes by even 1, almost every key's owning node changes.
- At 10M QPS this produces a complete cache miss storm for tens of seconds, hammering the database to destruction.
- Consistent hashing places both keys and nodes on a conceptual ring using a hash function.

*Applies **caching** to the decision above.*

**Observable outcome:** Consistent hashing places both keys and nodes on a conceptual ring using a hash function.

## Stage 6 — Real-World Cache Strategies: How Twitter, Facebook, and Netflix Use Distributed Caches

- Twitter's Twemproxy is a lightweight cache proxy that sits between application servers and Redis/Memcached instances.
- Twemproxy handles consistent hashing, connection multiplexing, and automatic failover transparently from the application.
- The key insight Twitter solved was connection fan-out: 10,000 application server threads each opening direct connections to 100 cache nodes creates 1,000,000 concurrent connections.
- Twemproxy reduces this to 100 connections from the proxy pool.

*Applies **system-design-hld-architectures** to the decision above.*

**Observable outcome:** Twemproxy reduces this to 100 connections from the proxy pool.

## End-to-end outcome

A distributed cache is an in-memory, key-value store shared across multiple servers.

## Alternatives considered

### Decision Guide: Redis vs. Memcached and Choosing the Right Cache Pattern

The most common interview mistake is proposing Write-Through caching for all workloads.


## Failure and degraded modes

### Trade-offs and Failure Modes in Distributed Cache Design

- **Detection:** Cache Stampede (Thundering Herd): When a popular key expires, dozens or thousands of simultaneous requests all miss the cache and hit the database concurrently.
- **User or operator impact:** Cache Stampede (Thundering Herd): When a popular key expires, dozens or thousands of simultaneous requests all miss the cache and hit the database concurrently.

### Lessons Learned from Cache Failures at Scale

- **Detection:** The Stampede That Took Down Instagram's Database.
- **User or operator impact:** The Stampede That Took Down Instagram's Database.



## Scenario context

A news feed system builds personalized timelines by combining content publishing, graph relationships, and ranking. The scalability crux is the fan-out amplified write path: a single celebrity post can trigger 100M writes. A hybrid fan-out strategy—push for normal users, pull for celebrities—is the industry standard for 99.99% availability.

Imagine it’s 2013. Twitter is growing at a breakneck pace. Most users have…

This walkthrough focuses on **The Katy Perry Problem**.

Before making a design choice, clarify:

- Twitter is growing at a breakneck pace.
- Most users have a few hundred followers, and the system handles them easily by "pushing" new tweets into their home timelines at write-time.
- She has 100 million followers.

**Concepts in play:** system-design-hld-architectures, caching, graph-traversal-algorithms, event-driven-architecture

## Stage 1 — News Feed: Use Cases, Actors, and Scale Requirements

- Actors - Publisher / Author: Creates and posts content (text, media).
- - Reader / Follower: Consumes a ranked home feed of followed users' posts.
- - System: Handles fan-out, ranking, and timeline materialization.
- Functional Requirements - Post Creation: Users can POST content with text/media.

*Applies **system-design-hld-architectures** to the decision above.*

**Observable outcome:** Functional Requirements - Post Creation: Users can POST content with text/media.

## Stage 2 — Basics of News Feed Architecture

- At its heart, a news feed is a many-to-many relationship pipeline .
- Unlike a simple blog where one post is read by everyone, a feed is a personalized "inbox" for every user.
- Apache Kafka is the decoupling mechanism between post creation and fan-out.
- The posts.created topic is partitioned by author id , ensuring that all posts from one author are processed by the same fan-out worker partition and arrive in creation order.

*Applies **caching** to the decision above.*

**Observable outcome:** The posts.created topic is partitioned by author id , ensuring that all posts from one author are processed by the same fan-out worker partition and arrive in creation order.

## End-to-end outcome

After working through the stages, you should be able to explain:

- The Celebrity Threshold Requires Continuous Calibration.
- Setting the celebrity threshold too low pushes too many accounts into pull fan-out, degrading read performance for mid-tier influencers whose follower lists still take seconds to query.
- - A news feed is a read-heavy, write-amplified system.
- The core design tension is between write-time cost (fan-out to millions of followers) and read-time cost (merging posts from all followed authors on every page load).

## Alternatives considered

The source focuses on one primary design. A second architecture is not compared, so no alternative is invented here.

## Failure and degraded modes

The source does not isolate a separate failure mode. Limits that matter for the scenario are called out in the stage decisions above.


## Scenario context

Payment systems optimize for correctness first, then throughput. This guide covers idempotency, double-entry ledgers, and reconciliation.

Stripe processes over 250 million API requests per day, and every single payment must be idempotent: a user clicking "Pay" twice must never produce two charges, even when the first HTTP call timed out and no confirmation reached the client. The Knight Capital incident of 2012 — w…

This walkthrough focuses on **Payment Processing: Use Cases & Requirements**.

Before making a design choice, clarify:

- A robust payment platform must handle several distinct user journeys and operational requirements.
- In an interview, defining these boundaries early is critical.
- Functional Requirements - Idempotent Authorization : Ensure that a payment request is processed exactly once, regardless of retries.

**Concepts in play:** system-design-hld-architectures, caching, consistency-replication, jvm-internals-and-garbage-collection

## Stage 1 — Basics: Baseline Architecture

- At its core, a payment system is a coordinator between three entities: the User (Client) , the Internal Ledger (Source of Truth) , and the External Payment Service Provider (PSP) .
- The baseline architecture consists of: 1.
- Payment API : The entry point that receives requests and orchestrates the flow.
- Idempotency Store : A fast, key-value store (like Redis) to track request keys.

*Applies **system-design-hld-architectures** to the decision above.*

```architecture
{
  "version": 1,
  "title": "Basics: Baseline Architecture - Subsystem Overview",
  "description": "Decoupled system architecture isolating ingress, stateless logic, and persistent datastores.",
  "view": "high-level",
  "pattern": "microservices",
  "direction": "LR",
  "groups": [
    { "id": "edge", "label": "Edge Traffic Ingress", "tone": "slate" },
    { "id": "services", "label": "Stateless Microservices", "tone": "blue" },
    { "id": "data", "label": "Datastore Infrastructure", "tone": "green" }
  ],
  "nodes": [
    { "id": "gw", "component": "api-gateway", "label": "API Gateway", "group": "edge", "badge": "TLS & Auth" },
    { "id": "app", "component": "workflow-service", "label": "Domain Service", "group": "services", "badge": "Stateless" },
    { "id": "cache", "component": "redis", "label": "Redis Cache", "group": "data", "badge": "Sub-ms Reads" },
    { "id": "db", "component": "postgres", "label": "Transactional DB", "group": "data", "badge": "ACID Log" }
  ],
  "edges": [
    { "from": "gw", "to": "app", "kind": "request", "label": "gRPC Request" },
    { "from": "app", "to": "cache", "kind": "read", "label": "Cache Lookups" },
    { "from": "app", "to": "db", "kind": "write", "label": "Durable Write" }
  ],
  "takeaway": {
    "title": "Architecture Boundary Invariant",
    "body": "Stateless compute nodes scale horizontally while shielding underlying database clusters behind fast cache layers."
  }
}
```

**Observable outcome:** Idempotency Store : A fast, key-value store (like Redis) to track request keys.

## Stage 2 — Mechanics: Key Logic

- The most critical logic in a payment system is the Persist-Before-Call pattern combined with Double-Entry Bookkeeping .
- The Persist-Before-Call Pattern To prevent lost payments, you must record the intent to pay in your database before calling the external PSP.
- This ensures that if the call to the PSP times out or the server crashes, you have a record to reconcile against later.
- Double-Entry Bookkeeping Money is never simply "updated" in a row.

*Applies **caching** to the decision above.*

```mermaid
sequenceDiagram
    participant U as User
    participant API as Payment API
    participant Redis as Idempotency Store
    participant DB as Ledger DB
    participant PSP as Payment Provider
    U->>API: POST /payments with Idempotency-Key
    API->>Redis: Check existing key
    Redis-->>API: Not found — proceed
    API->>DB: Write PENDING intent
    DB-->>API: Intent persisted
    API->>PSP: Authorize charge with intent_id
    PSP-->>API: Authorization success
    API->>DB: Update status to AUTHORIZED
    API->>Redis: Cache response with 24h TTL
    API-->>U: Payment confirmed
```

**Observable outcome:** Double-Entry Bookkeeping Money is never simply "updated" in a row.

## Stage 3 — Estimations & Design Goals

- Before diving into the high-level design, let's establish the scale.
- - Throughput : 1,000 Transactions Per Second (TPS) average, 5,000 TPS peak (e.g., Black Friday).
- - Storage : Each transaction record is ~2KB.
- At 1,000 TPS, that's 2MB/sec, or ~172GB/day.

*Applies **consistency-replication** to the decision above.*

**Observable outcome:** At 1,000 TPS, that's 2MB/sec, or ~172GB/day.

## Stage 4 — High-Level Design

- The following diagram illustrates the orchestrated flow of a payment transaction.
- The flow enforces the Persist-Before-Call pattern: the Payment API writes a PENDING intent record to the PostgreSQL ledger (step 2) before calling the external PSP (step 3).
- This ensures that even if the network drops between steps 3 and 4, the system has a record to reconcile against.
- The idempotency check in Redis (step 1) prevents a duplicate upstream call if the client retries with the same key.

*Applies **jvm-internals-and-garbage-collection** to the decision above.*

```architecture
{
  "version": 1,
  "title": "High-Level Design - Subsystem Overview",
  "description": "Decoupled system architecture isolating ingress, stateless logic, and persistent datastores.",
  "view": "high-level",
  "pattern": "microservices",
  "direction": "LR",
  "groups": [
    { "id": "edge", "label": "Edge Traffic Ingress", "tone": "slate" },
    { "id": "services", "label": "Stateless Microservices", "tone": "blue" },
    { "id": "data", "label": "Datastore Infrastructure", "tone": "green" }
  ],
  "nodes": [
    { "id": "gw", "component": "api-gateway", "label": "API Gateway", "group": "edge", "badge": "TLS & Auth" },
    { "id": "app", "component": "workflow-service", "label": "Domain Service", "group": "services", "badge": "Stateless" },
    { "id": "cache", "component": "redis", "label": "Redis Cache", "group": "data", "badge": "Sub-ms Reads" },
    { "id": "db", "component": "postgres", "label": "Transactional DB", "group": "data", "badge": "ACID Log" }
  ],
  "edges": [
    { "from": "gw", "to": "app", "kind": "request", "label": "gRPC Request" },
    { "from": "app", "to": "cache", "kind": "read", "label": "Cache Lookups" },
    { "from": "app", "to": "db", "kind": "write", "label": "Durable Write" }
  ],
  "takeaway": {
    "title": "Architecture Boundary Invariant",
    "body": "Stateless compute nodes scale horizontally while shielding underlying database clusters behind fast cache layers."
  }
}
```

```mermaid
graph TD
    User[Customer/App] -->|POST /payments| API[Payment API]
    API -->|1. Check/Lock Key| Redis[(Idempotency Redis)]
    API -->|2. Create Intent| Ledger[(PostgreSQL Ledger)]
    API -->|3. Authorize| PSP[Payment Provider - Stripe/Adyen]
    PSP -->|4. Response| API
    API -->|5. Update Status| Ledger
    API -->|6. Cache Result| Redis
    Ledger -->|7. CDC Event| Kafka{Kafka}
    Kafka -->|8. Notify| Webhook[Webhook Service]
    Webhook -->|9. Callback| Merchant[Merchant Server]
    Recon[Reconciliation Job] -->|Daily Sync| PSP
    Recon -->|Verify| Ledger
```

**Observable outcome:** The idempotency check in Redis (step 1) prevents a duplicate upstream call if the client retries with the same key.

## Stage 5 — Deep Dive: Double-Entry Bookkeeping, Idempotency Key Lifecycle, and the Reconciliation Engine

- Three mechanisms define correctness in a payment system: double-entry bookkeeping that makes every financial state change auditable, idempotency that prevents duplicate charges from network retries, and reconciliation t…
- Internals: Double-Entry Bookkeeping and Why Integer Cents Matter In a double-entry ledger, money is never simply "updated in a row." Every movement of value produces at minimum two ledger entries: a debit from one accou…
- When a customer pays $100 to a merchant, the system writes: DEBIT $100 from the customer's account, CREDIT $100 to the merchant's escrow account.
- The fundamental accounting invariant is that the sum of all debits must equal the sum of all credits across the entire ledger at all times.

*Applies **system-design-hld-architectures** to the decision above.*

```mermaid
flowchart TD
    A["Request Received with Idempotency-Key"] --> B{"Key Exists in Redis?"}
    B -->|Yes| C["Return Cached Response Immediately"]
    B -->|No| D["Acquire Distributed Lock — 30s TTL"]
    D --> E["Write PENDING Intent to Ledger"]
    E --> F["Call External PSP"]
    F --> G{"PSP Response Received?"}
    G -->|Success| H["Update Ledger to AUTHORIZED"]
    G -->|Failure| I["Update Ledger to FAILED"]
    H --> J["Cache Success Response — 24h TTL"]
    I --> J
    J --> K["Release Lock and Return to Caller"]
```

**Observable outcome:** The fundamental accounting invariant is that the sum of all debits must equal the sum of all credits across the entire ledger at all times.

## Stage 6 — Real-World Payment Architectures: Stripe, PayPal, and Square

- Stripe built its idempotency key system as a first-class API primitive from day one.
- Every write endpoint in the Stripe API accepts an Idempotency-Key header.
- Stripe stores the idempotency key, the request fingerprint (URL + body hash), and the response body in a dedicated idempotency database with a 24-hour retention window.
- Critically, Stripe validates that the same idempotency key is not used with a different request body — preventing a class of bugs where a client reuses a key across conceptually different operations.

*Applies **caching** to the decision above.*

```architecture
{
  "version": 1,
  "title": "Real-World Payment Architectures: Stripe, PayPal, and Square - Subsystem Overview",
  "description": "Decoupled system architecture isolating ingress, stateless logic, and persistent datastores.",
  "view": "high-level",
  "pattern": "microservices",
  "direction": "LR",
  "groups": [
    { "id": "edge", "label": "Edge Traffic Ingress", "tone": "slate" },
    { "id": "services", "label": "Stateless Microservices", "tone": "blue" },
    { "id": "data", "label": "Datastore Infrastructure", "tone": "green" }
  ],
  "nodes": [
    { "id": "gw", "component": "api-gateway", "label": "API Gateway", "group": "edge", "badge": "TLS & Auth" },
    { "id": "app", "component": "workflow-service", "label": "Domain Service", "group": "services", "badge": "Stateless" },
    { "id": "cache", "component": "redis", "label": "Redis Cache", "group": "data", "badge": "Sub-ms Reads" },
    { "id": "db", "component": "postgres", "label": "Transactional DB", "group": "data", "badge": "ACID Log" }
  ],
  "edges": [
    { "from": "gw", "to": "app", "kind": "request", "label": "gRPC Request" },
    { "from": "app", "to": "cache", "kind": "read", "label": "Cache Lookups" },
    { "from": "app", "to": "db", "kind": "write", "label": "Durable Write" }
  ],
  "takeaway": {
    "title": "Architecture Boundary Invariant",
    "body": "Stateless compute nodes scale horizontally while shielding underlying database clusters behind fast cache layers."
  }
}
```

**Observable outcome:** Critically, Stripe validates that the same idempotency key is not used with a different request body — preventing a class of bugs where a client reuses a key across conceptually different operations.

## End-to-end outcome

A robust payment platform must handle several distinct user journeys and operational requirements.

## Alternatives considered

### Decision Guide: Choosing the Right Payment Guarantee Level for Each Operation

Decision Guide: Choosing the Right Payment Guarantee Level for Each Operation


## Failure and degraded modes

### Trade-offs and Failure Modes in Payment System Design

- **Detection:** The Double-Charge Failure Mode: The API calls the PSP, the PSP charges the card and returns success, but the network drops before the response reaches the API.
- **User or operator impact:** The Double-Charge Failure Mode: The API calls the PSP, the PSP charges the card and returns success, but the network drops before the response reaches the API.

### Lessons Learned from Payment System Failures

- **Detection:** The Knight Capital Incident: No Idempotency Guard on Algorithmic Orders.
- **User or operator impact:** The Knight Capital Incident: No Idempotency Guard on Algorithmic Orders.



Work through common HLD prompts from easier to more demanding.

Case Study Practice

A high-level design roadmap for architecture patterns, APIs, storage, scalability, and full system design case studies.