NoSQL Partitioning: How Cassandra, DynamoDB, and MongoDB Split Data

Scenario context

Every NoSQL database hides a partitioning engine behind a deceptively simple API. Cassandra uses a consistent hashing ring where a Murmur3 hash of your partition key selects a node — virtual nodes (vnodes) make rebalancing smooth. DynamoDB manages partitioning completely on your behalf, splitting physical partitions at 10 GB automatically and redistributing throughput via adaptive capacity — but a structurally hot p…

This walkthrough focuses on Worked Scenarios: Diagnosing and Fixing Common Partition Problems.

Before making a design choice, clarify:

• Scenario 1 — Cassandra: wide partition on a gaming channel.
• A Discord-style messaging service stores messages with PRIMARY KEY (channel id, message id) .
• After six months, the general channel in a popular server has 50 million messages — the partition is approaching 400 MB.

Concepts in play: database-sharding, consistency-replication, database-indexing, btree-vs-lsmtree

Stage 1 — Visualizing Partition Routing Across All Three Systems

• The first diagram traces a write request through Cassandra's consistent hash ring, from partition key to vnode assignment to replication across three nodes.
• Follow each numbered step to see where the routing decision is made and how replication propagates automatically.
• The ring lookup is resolved from Node B's local in-memory token map — no centralised routing service is involved.
• The replication fan-out is parallel: Nodes C and D receive the write simultaneously with Node B.

flowchart TD
    A["Write Request (partition key: channel_id = gaming-news)"] --> B["Murmur3 Hash (applied to partition key value)"]
    B --> C["Token Computed (64-bit integer on the ring)"]
    C --> D["Ring Lookup (find vnode owning this token range)"]
    D --> E["Primary Node Identified (Node B owns this vnode)"]
    E --> F["Coordinator Routes to Node B"]
    F --> G["Replication Factor = 3 (forward to next 2 clockwise nodes)"]
    G --> H["Node B (primary replica — write persisted)"]
    G --> I["Node C (replica 2 — write persisted)"]
    G --> J["Node D (replica 3 — write persisted)"]
    H --> K["Quorum Satisfied (2 of 3 acknowledged — return to client)"]
    I --> K
    J --> K

flowchart TD
    A["Write Request (PK: product_id = top-seller, SK: order_ts)"] --> B["Internal Hash Function (applied to partition key)"]
    B --> C["Partition Map Lookup (identifies physical partition P7)"]
    C --> D{"Partition P7 approaching 10 GB?"}
    D -->|"No — capacity available"| E{"Is partition key structurally hot?"}
    D -->|"Yes — 10 GB limit reached"| F["Automatic Split (P7 → P7a and P7b)"]
    F --> G["Throughput Divided Equally (each half gets half of P7 WCU/RCU allocation)"]
    G --> E
    E -->|"No — traffic is balanced"| H["Write Succeeds (response returned)"]
    E -->|"Yes — throttling risk"| I["Adaptive Capacity Triggered (shift unused RCU/WCU from cold partitions)"]
    I --> J{"Spike transient or structural?"}
    J -->|"Transient — burst resolved"| H
    J -->|"Structural — hot key permanent"| K["ProvisionedThroughputExceededException (write sharding required at app layer)"]

Observable outcome: The replication fan-out is parallel: Nodes C and D receive the write simultaneously with Node B.

Stage 2 — Real-World Applications: Discord, Netflix, and Retail Scale

• Discord — Cassandra for message history.
• Discord's message storage uses Cassandra with a composite partition key of (channel id, week bucket) .
• The week bucket component is derived from the message timestamp — each week of messages for a channel becomes its own partition.
• This design caps per-partition size at roughly one week's volume of messages per channel, preventing the unbounded growth that would occur with channel id alone as the partition key.

Observable outcome: This design caps per-partition size at roughly one week's volume of messages per channel, preventing the unbounded growth that would occur with channel id alone as the partition key.

End-to-end outcome

After working through the stages, you should be able to explain:

• The partition key is a permanent architectural contract.
• Changing it post-launch requires a full schema rebuild — weeks of migration work on a live system.
• Cassandra uses a consistent hashing ring.
• Murmur3 maps partition key values to tokens; vnodes distribute token ranges across nodes for smooth rebalancing.

Alternatives considered

Decision Guide: Which NoSQL Partitioning Model Fits Your Use Case

Decision Guide: Which NoSQL Partitioning Model Fits Your Use Case

Failure and degraded modes

Trade-offs and Failure Modes in NoSQL Partitioning

• Detection: The most expensive anti-pattern across all three systems is choosing a partition key based on convenience (the field you already query) without verifying its cardinality and access distribution.
• User or operator impact: The most expensive anti-pattern across all three systems is choosing a partition key based on convenience (the field you already query) without verifying its cardinality and access distribution.