Skip to main content
← PROBLEMS / SYSTEM-DESIGN
An end-to-end domain learning journey

Design a Real-Time Chat, Typing & Presence
System (Slack / Discord / WhatsApp Scale).

Terminate stateful WebSockets at edge Gateway Pods backed by Redis session maps, sequence messages per channel with Snowflake/monotonic IDs into ScyllaDB/Cassandra, and route offline users to APNs/FCM.
Start the journey7 stages
FIRST PRINCIPLES. PRODUCTION SCALE.
The idea to keep with you

Design a real-time chat and presence platform supporting 50M concurrent WebSocket connections, 1M messages/sec, < 100 ms p99 message delivery, multi-device read-cursor sync, and both 1:1 DMs and 100K-member channels.

From patterns to decisions.

Terminate stateful WebSockets at edge Gateway Pods backed by Redis session maps, sequence messages per channel with Snowflake/monotonic IDs into ScyllaDB/Cassandra, and route offline users to APNs/FCM.

Full walkthrough · 7 stages
01
STAGE 01 · CORE INTUITIONfoundation

Scale Envelope & Latency Budget.

Design a real-time chat and presence platform supporting 50M concurrent WebSocket connections, 1M messages/sec, < 100 ms p99 message delivery, multi-device read-cursor sync, and both 1:1 DMs and 100K-member channels.

STAGE 01 · CORE INTUITION

Scale Envelope & Latency Budget

Scale EnvelopeIngress / BaselineLatency BudgetScale ConstraintValidation GateBottleneck SignalProduction OutcomeTarget SLAexposes

1.Stage Objective: Size throughput, storage, and p99 latency targets.

2.Level (Foundation): Design a real-time chat and presence platform supporting 50M concurrent WebSocket connections, 1M messages/sec, < 100 ms p99 message deliver

3.Operational Invariant: Size throughput, storage, and p99 latency targets.

Caption: Size throughput, storage, and p99 latency targets.

02
STAGE 02 · CORE MECHANICSfoundation

Core Architecture & Data Flow.

Each incoming message receives a time-ordered Snowflake ID (or per-channel sequence ID) so clients sort messages deterministically across devices without relying on client clocks. Messages persist asynchronously to ScyllaDB / Cassandra partitioned by (channel_id, time_bucket) to bound partition sizes.

STAGE 02 · CORE MECHANICS

Core Architecture & Data Flow

Core ArchitectureInput StateData FlowCore MechanismValidation GateInvariant CheckProduction OutcomeOutput Statetransforms

1.Stage Objective: Assemble the primary data path and storage layout.

2.Level (Foundation): Each incoming message receives a time-ordered **Snowflake ID** (or per-channel sequence ID) so clients sort messages deterministically acros

3.Operational Invariant: Assemble the primary data path and storage layout.

Caption: Assemble the primary data path and storage layout.

03
STAGE 03 · SAFE IMPLEMENTATIONintermediate

Gateway Routing & Offline Push Fallback.

def route_chat_message(presence_store, pubsub, push_queue, recipient_ids: list[str], msg: dict) -> None:
    gateways = presence_store.mget_gateways(recipient_ids)
    for uid, gw_id in zip(recipient_ids, gateways):
        if gw_id:
            pubsub.publish(f'gw:{gw_id}', {'user_id': uid, 'msg': msg})
        else:
            push_queue.enqueue({'user_id': uid, 'channel_id': msg['channel_id'], 'preview': msg['snippet']})
STAGE 03 · SAFE IMPLEMENTATION

Gateway Routing & Offline Push Fallback

Gateway RoutingValidated InputOffline PushFallbackAtomic StepValidation GateError / Retry GuardProduction OutcomeCommitted Resultexecutes

1.Stage Objective: Route messages to online recipient gateways and fall back to push notifications for offline devices.

2.Level (Intermediate): def route_chat_message(presence_store, pubsub, push_queue, recipient_ids: list[str], msg: dict) -> None:

3.Operational Invariant: Route messages to online recipient gateways and fall back to push notifications for offline devices.

Caption: Route messages to online recipient gateways and fall back to push notifications for offline devices.

04
STAGE 04 · SCALING & TRADE-OFFSintermediate

Sharding & Hot-Spot Mitigation.

Partition state deterministically and apply bounded-load or sub-bucket sharding for hot keys.

STAGE 04 · SCALING & TRADE-OFFS

Sharding & Hot-Spot Mitigation

ShardingHot PathHot-Spot MitigationPartition / IndexValidation GateLatency / RAM BoundProduction OutcomeScaled Statebounds

1.Stage Objective: Prevent single-partition bottlenecks at peak load.

2.Level (Intermediate): Partition state deterministically and apply bounded-load or sub-bucket sharding for hot keys.

3.Operational Invariant: Prevent single-partition bottlenecks at peak load.

Caption: Prevent single-partition bottlenecks at peak load.

05
STAGE 05 · FAULT ISOLATIONadvanced

Failure Modes & Graceful Degradation.

Apply circuit breakers, fallback caches, and dead-letter quarantine queues.

STAGE 05 · FAULT ISOLATION

Failure Modes & Graceful Degradation

Failure ModesFault TriggerGraceful DegradationIsolation BoundaryValidation GateFallback / QuarantineProduction OutcomeProtected Pathisolates

1.Stage Objective: Keep critical paths healthy during partial outages.

2.Level (Advanced): Apply circuit breakers, fallback caches, and dead-letter quarantine queues.

3.Operational Invariant: Keep critical paths healthy during partial outages.

Caption: Keep critical paths healthy during partial outages.

06
STAGE 06 · RECOVERY & CONSISTENCYadvanced

Consistency, Idempotency & Recovery.

Enforce atomic state transitions, fencing/idempotency keys, and fast snapshot recovery.

STAGE 06 · RECOVERY & CONSISTENCY

Consistency, Idempotency & Recovery

ConsistencyIdempotencyCheckpoint / LogRecoveryReplay / SyncValidation GateConsistency GateProduction OutcomeRecovered Stateverifies

1.Stage Objective: Guarantee correctness across retries and node crashes.

2.Level (Advanced): Enforce atomic state transitions, fencing/idempotency keys, and fast snapshot recovery.

3.Operational Invariant: Guarantee correctness across retries and node crashes.

Caption: Guarantee correctness across retries and node crashes.

07
STAGE 07 · OBSERVABILITY & GUARDRAILSadvanced

Telemetry, Circuit Breakers & Failure Modes.

Track p99 latency, retry/conflict ratios, and saturation metrics; fail fast with bounded retries and circuit breakers whenever invariants degrade.

STAGE 07 · OBSERVABILITY & GUARDRAILS

Production Guardrails

ProductionGuardrailsTelemetry SignalCore EngineSLO ThresholdGuard & VerifyCircuit BreakerProduction OutcomeSafe Degradationtrips

1.Stage Objective: Monitor tail latency, error budgets, and invariant violations in production.

2.Level (Advanced): Track p99 latency, retry/conflict ratios, and saturation metrics; fail fast with bounded retries and circuit breakers whenever invariants de

3.Operational Invariant: Bounded retries, telemetry alerts, and circuit breakers prevent cascading failures.

Caption: Bounded retries, telemetry alerts, and circuit breakers prevent cascading failures.

Practice · Explain the consequences

Practice: Make the trade-off yourself.

Question 1 of 3
Score: 0/3

Why partition ScyllaDB/Cassandra messages by `(channel_id, month_bucket)` instead of just `channel_id`?

Recommended Learning Flow