System Design HLD Example: Collaborative Document Editing (Google Docs)

Scenario context

Real-time collaborative editing relies on Operational Transformation (OT) or CRDTs to resolve concurrent edits without data loss. The core trade-off is Latency vs. Consistency: we use optimistic local updates for zero-latency typing and a centralized OT server to ensure all clients eventually converge to the exact same document state.

Imagine two users, Alice and Bob, are editing the same sentence: "The…

This walkthrough focuses on The "Hello World" Race Condition.

Before making a design choice, clarify:

• Imagine two users, Alice and Bob, are editing the same sentence: "The cat is happy." Alice wants to change it to "The fat cat is happy," so she inserts "fat " at position 4.
• Simultaneously, Bob wants to change it to "The cat is very happy," so he inserts "very " at position 11.
• Alice's Edit: Server applies INSERT("fat ", pos:4) .

Concepts in play: system-design-hld-architectures, caching, database-sharding, consistency-replication

Stage 1 — Real-Time Collaboration: Use Cases & Requirements

• Actors - Author / Editor: Creates and modifies document content in real-time.
• • Collaborator: Views live changes and contributes simultaneously.
• • Viewer: Consumes the document without editing rights but still expects live updates.
• Functional Requirements - Real-Time Editing: Multiple users can edit the same document simultaneously.

Observable outcome: Functional Requirements - Real-Time Editing: Multiple users can edit the same document simultaneously.

Stage 2 — Basics: Operational Transformation (OT)

• Operational Transformation is the "classic" approach used by Google Docs.
• The core idea is that the server doesn't just apply an operation; it transforms it against all other operations that happened since the client's last known version.
• Every operation is represented as a triple: (Type, Position, Value) .
• When the server receives an operation based on version V , but the current server version is V+k , it runs the new operation through a transform function T(op new, op concurrent) for all $k$ operations.

Observable outcome: When the server receives an operation based on version V , but the current server version is V+k , it runs the new operation through a transform function T(op new, op concurrent) for all $k$ operations.

Stage 3 — Mechanics: The Transform Matrix

• To implement OT, you must define how every operation type interacts with every other operation type.
• Insert: If User A inserts at position 5 and User B inserts at position 10, no change is needed for A, but B's position must be incremented by the length of A's insert.
• Delete: If a character is deleted before an insert, the insert position must be decremented.
• Delete: If both users delete the same character, the second operation becomes a "no-op" to avoid double-deletion.

Observable outcome: Delete: If both users delete the same character, the second operation becomes a "no-op" to avoid double-deletion.

Stage 4 — Estimations & Design Goals

• The Math of Real-Time Sync - Average Typist: 60 words per minute $\approx 5$ characters per second.
• • Active Users per Doc: 10 (average) to 1,000 (hot docs).
• • Traffic per Doc: 10 users 5 chars/sec = 50 operations/sec .
• • Fan-out: Every 1 operation must be broadcast to the other 9 users.

Observable outcome: - Fan-out: Every 1 operation must be broadcast to the other 9 users.

Stage 5 — High-Level Design: The Centralized OT Architecture

• The following architecture utilizes a centralized "Orderer" to ensure all operations are sequenced correctly.
• The diagram above captures the full topology.
• The critical insight is the separation of concerns : WebSocket Servers handle raw TCP connections and serialization, the OT Service is the single stateful arbiter that runs all transformations, and Redis Pub/Sub is the…

graph TD
    UserA((User A)) <--> WSS1[WS Server Shard 1]
    UserB((User B)) <--> WSS1
    UserC((User C)) <--> WSS2[WS Server Shard 2]

    WSS1 <--> LB[Internal Load Balancer]
    WSS2 <--> LB

    LB <--> OTS[OT Service: The Arbiter]
    OTS --> PDB[(Document Store: Postgres)]
    OTS --> RC[(Active Doc Cache: Redis)]

    OTS --> PubSub[Redis Pub/Sub]
    PubSub --> WSS1
    PubSub --> WSS2

Observable outcome: The critical insight is the separation of concerns : WebSocket Servers handle raw TCP connections and serialization, the OT Service is the single stateful arbiter that runs all transformations, and Redis Pub/Sub is the…

Stage 6 — Deep Dive: How the OT Server Guarantees Document Convergence

• The "magic" of collaborative editing lies entirely inside the OT Service labeled "The Arbiter" in the architecture diagram.
• Understanding its internals reveals why a simple message broker is not enough—you need a stateful transformation engine with an ordered operation log.
• Internals: The Operation Log and Version Clock The OT Server maintains two critical data structures for each active document: an ordered operation log and a document snapshot .
• The operation log is an append-only ledger of every transformation ever applied to the document.

Observable outcome: The operation log is an append-only ledger of every transformation ever applied to the document.

End-to-end outcome

After working through the stages, you should be able to explain:

• Lesson 1 — Start with a single OT node and shard by document ID from day one.
• Adding document-level routing as an afterthought means rewriting the client session resume protocol.

Alternatives considered

OT vs. CRDT: A Decision Framework for Your Architecture

Use this guide to choose the right synchronization strategy before committing to either path.

Failure and degraded modes

Consistency vs. Latency: Trade-offs That Define Collaborative Systems

• Detection: Critical Failure Mode — The Thundering Herd on Reconnect: If a hot document has 1,000 simultaneous users and the OT service node crashes, all 1,000 WebSocket connections reconnect at the same instant.
• User or operator impact: Critical Failure Mode — The Thundering Herd on Reconnect: If a hot document has 1,000 simultaneous users and the OT service node crashes, all 1,000 WebSocket connections reconnect at the same instant.