Outbox Table Partitioning: Scaling Event Delivery Without Locking
When a microservice hits the outbox table, it can choke your event pipeline—learn how partitioning keeps the flow smooth.
Partition your outbox table to avoid write contention and speed up event reads—just remember to keep partitions balanced and keep your cleanup job in sync!
🎯 System Scenario & Requirements
System Goal
Partition your outbox table to avoid write contention and speed up event reads—just remember to keep partitions balanced and keep your cleanup job in sync!
Actors & Components
- Domain Service
- Partitioned Outbox
- Event Publisher
- Outbox Table (4 partitions)
Functional Requirements
- Partition your outbox table to avoid write contention and speed up event reads—just remember to keep partitions balanced and keep your cleanup job in sync!
- 🚀 You’re building a microservice that writes domain events into a single outbox table as part of a transaction.
- Under load, every write hits the same table, causing lock contention, slow commits, and a backlog of events for the publisher.
Constraints & Assumptions
- Operational latency, consistency levels, and network partition boundaries must be enforced.
⚡ Execution Stages & Decision Trace
Stage 1 — Partitioning Strategy
Use table partitioning (PostgreSQL, MySQL, or a sharded NoSQL table) to spread writes across multiple physical segments.
Common strategies: Example: PostgreSQL hash partition
Deep Dive
Deep Dive: Stage 1 — Partitioning Strategy
Use table partitioning (PostgreSQL, MySQL, or a sharded NoSQL table) to spread writes across multiple physical segments. Common strategies:
| Strategy | How it works | When to use |
| Hash‑by‑entity | Partition key = hash(entity_id) | Many small entities, even distribution |
| Time‑window | Partition key = date_trunc('hour', event_time) | Temporal bursts, easy retention |
| Tenant‑ID | Partition key = tenant_id | Multi‑tenant SaaS with per‑tenant isolation |
Example: PostgreSQL hash partition
CREATE TABLE outbox (
id BIGSERIAL PRIMARY KEY,
tenant_id BIGINT NOT NULL,
event_type TEXT NOT NULL,
payload JSONB NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
) PARTITION BY HASH (tenant_id);
CREATE TABLE outbox_p0 PARTITION OF outbox FOR VALUES WITH (MODULUS 4, REMAINDER 0);
CREATE TABLE outbox_p1 PARTITION OF outbox FOR VALUES WITH (MODULUS 4, REMAINDER 1);
-- … up to p3
Observable outcome: Common strategies: Example: PostgreSQL hash partition
Deep Dive: Stage 2 — Architecture Kit Diagram
Observable outcome: You can explain how “Architecture Kit Diagram” changes the architecture or implementation path.
Deep Dive: Stage 3 — Trade‑offs & Gotchas
| Factor | Benefit | Risk |
| Write speed | Fewer locks per partition | Skewed partitions → hot spot |
| Read parallelism | Publisher can poll multiple partitions concurrently | Complexity of managing multiple partitions |
| Cleanup | Drop old partitions instead of deleting rows | Must coordinate cleanup with publisher to avoid missing events |
| Schema evolution | Add columns to all partitions | Migration scripts must target each partition |
Observable outcome: You can explain how “Trade‑offs & Gotchas” changes the architecture or implementation path.
Deep Dive: Stage 4 — Quick Checklist
- Choose a stable partition key that distributes writes evenly.
- Monitor partition sizes; add or drop partitions as needed.
- Implement idempotent reads in the publisher to handle duplicate reads from partitions.
- Schedule cleanup (e.g.,
DROP TABLE outbox_p0for old data) after confirming delivery.
By partitioning the outbox, you keep your transaction fast, your event stream smooth, and your system ready to scale.
Observable outcome: By partitioning the outbox, you keep your transaction fast, your event stream smooth, and your system ready to scale.
📊 System Outcome & Trade-offs
Observed User & Operator Behavior
User: Clients observe deterministic state without lost updates or race conditions.
Operator: Operators can verify stage outcomes against the scenario requirements.
Key Architectural Trade-offs
- System guarantees are traded against replication overhead and failover convergence times.
Key learnings
- Use table partitioning (PostgreSQL, MySQL, or a sharded NoSQL table) to spread writes across multiple physical segments.
- Apply event-driven-architecture to Architecture Kit Diagram.
- Apply event-driven-architecture to Trade‑offs & Gotchas.
- Choose a stable partition key that distributes writes evenly.
Apply Concepts from this Scenario
Test your understanding of the architecture decisions and trade-offs presented in this case study.