Transactional Outbox vs. Change Data Capture: Choosing the Right Event Pipeline

Scenario context

🎯 Pick transactional outbox for guaranteed atomicity and minimal latency, or CDC for a lightweight, log‑driven approach when you can tolerate small gaps. You’ve built a service that updates a user profile and must publish an UserUpdated event. If you write the event to the broker after the DB commit, a crash in between can leave the database updated but no event sent—breaking downstream consumers. You need a patt…

This walkthrough focuses on Problem: “Lost Events” in Microservices.

Before making a design choice, clarify:

• You’ve built a service that updates a user profile and must publish an UserUpdated event.
• If you write the event to the broker after the DB commit, a crash in between can leave the database updated but no event sent—breaking downstream consumers.
• You need a pattern that keeps the DB write and event publication transactionally consistent .

Concepts in play: event-driven-architecture, transaction-isolation-distributed-acid, message-queues, consistency-replication

Stage 1 — Solution: Transactional Outbox

• With an outbox, the service writes both the domain record and a new row in an outbox table inside the same transaction.
• A separate “outbox worker” polls this table, serializes the payload, and pushes it to Kafka (or another broker).
• Because the DB commit guarantees both rows exist, the worker will never see a partially written state.

-- Inside a single transaction
INSERT INTO users (id, name) VALUES (:id, :name);
INSERT INTO outbox (id, aggregate_id, payload, created_at)
VALUES (uuid_generate_v4(), :id, jsonb_build_object('type','UserUpdated','id',:id), now());

# pseudo‑worker
for event in outbox_table.fetch_unprocessed():
    broker.publish(event.payload)
    mark_as_sent(event.id)

Observable outcome: Because the DB commit guarantees both rows exist, the worker will never see a partially written state.

Stage 2 — Trade‑offs

Observable outcome: You can explain how “Trade‑offs” changes the architecture or implementation path.

Stage 3 — When to Use Which

• • Transactional Outbox : When exactly‑once delivery and strong consistency are non‑negotiable (e.g., financial transactions).
• • CDC : When you can tolerate occasional retries or gaps, or you already have a log‑driven architecture in place (e.g., micro‑services that read from a shared event store).

Observable outcome: - CDC : When you can tolerate occasional retries or gaps, or you already have a log‑driven architecture in place (e.g., micro‑services that read from a shared event store).

Stage 4 — Architecture Kit Diagram

• { "version": 1, "title": "Transactional Outbox vs CDC Pipelines", "description": "Side‑by‑side view of the two event publishing patterns", "view": "data-flow", "pattern": "reactive-pull", "direction": "LR", "groups": […

Observable outcome: { "version": 1, "title": "Transactional Outbox vs CDC Pipelines", "description": "Side‑by‑side view of the two event publishing patterns", "view": "data-flow", "pattern": "reactive-pull", "direction": "LR", "groups": […

Stage 5 — Bottom Line

• Choose the outbox when you can’t afford a single lost event; lean on CDC when you need minimal latency and can handle eventual consistency.
• The right choice hinges on your service’s tolerance for gaps versus the cost of extra infrastructure.

Observable outcome: The right choice hinges on your service’s tolerance for gaps versus the cost of extra infrastructure.

End-to-end outcome

You’ve built a service that updates a user profile and must publish an UserUpdated event.

Alternatives considered

The source focuses on one primary design. A second architecture is not compared, so no alternative is invented here.

Failure and degraded modes

The source does not isolate a separate failure mode. Limits that matter for the scenario are called out in the stage decisions above.

• With an outbox, the service writes both the domain record and a new row in an outbox table inside the same transaction.
• Apply transaction-isolation-distributed-acid to Trade‑offs.
• • Transactional Outbox : When exactly‑once delivery and strong consistency are non‑negotiable (e.g., financial transactions).
• { "version": 1, "title": "Transactional Outbox vs CDC Pipelines", "description": "Side‑by‑side view of the two event publishing patterns", "view": "data-flow", "pattern": "reactive-pull", "direction": "LR", "groups": […