System Design HLD Example: Chat and Messaging Platform

Scenario context

A distributed chat system must balance low-latency delivery with strong per-conversation ordering. The architectural crux is the WebSocket Gateway for persistent stateful connections and Cassandra for append-heavy message storage partitioned by conversation. By using a Kafka-driven Fan-out model and Redis-based Presence heartbeats, the system can handle millions of concurrent users while ensuring no……

This walkthrough focuses on Scenario context.

Before making a design choice, clarify:

• A distributed chat system must balance low-latency delivery with strong per-conversation ordering.
• The architectural crux is the WebSocket Gateway for persistent stateful connections and Cassandra for append-heavy message storage partitioned by conversation.
• By using a Kafka-driven Fan-out model and Redis-based Presence heartbeats, the system can handle millions of concurrent users while ensuring no… This walkthrough focuses on Lessons Learned from Building Production Chat…

Concepts in play: system-design-hld-architectures, btree-vs-lsmtree, caching, consistency-replication

Stage 1 — End-to-end outcome

• After working through the stages, you should be able to explain: - Hot Conversation Partitions Are Real and Painful.
• • A Cassandra partition for a single high-traffic public server or channel — 10 million messages per day — concentrates all writes on one Cassandra node, creating a hot partition that degrades p99 write latency for that…
• • The Gateway tier is stateful; everything behind it — Message Service, Fan-out Workers, Presence Service — must be stateless and independently scalable.

Observable outcome: - The Gateway tier is stateful; everything behind it — Message Service, Fan-out Workers, Presence Service — must be stateless and independently scalable.

End-to-end outcome

After working through the stages, you should be able to explain:

• • • Chat is a stateful problem (persistent WebSocket connections) that must be solved with a stateless mindset (horizontal scaling behind the gateway).
• • The Gateway tier is stateful; everything behind it — Message Service, Fan-out Workers, Presence Service — must be stateless and independently scalable.

Alternatives considered

Alternatives considered

The source focuses on one primary design.

Failure and degraded modes

Failure and degraded modes

• Detection: The source does not isolate a separate failure mode.
• User or operator impact: The source does not isolate a separate failure mode.

• • • Chat is a stateful problem (persistent WebSocket connections) that must be solved with a stateless mindset (horizontal scaling behind the gateway).
• • The Gateway tier is stateful; everything behind it — Message Service, Fan-out Workers, Presence Service — must be stateless and independently scalable.
• After working through the stages, you should be able to explain: - Hot Conversation Partitions Are Real and Painful.