Distributed Transactions: 2PC, Saga, and XA Explained

Scenario context

Distributed transactions require you to choose a consistency model before choosing a protocol. 2PC and XA give atomic all-or-nothing commits but block all participants on coordinator failure. Saga gives eventual consistency with explicit compensation and survives coordinator crashes — at the cost of designing compensating transactions for every step. The Outbox pattern and idempotency are mandatory, not optional: th…

This walkthrough focuses on The Charge-Without-Order Incident: Why ACID Transactions Don't Cross Service Boundaries.

Before making a design choice, clarify:

• In 2018, a major e-commerce platform discovered that customers had been charged for orders that were never created.
• The payment service completed its database write successfully.
• The order service received the request and started writing — but the process crashed before it committed.

Concepts in play: transaction-isolation-distributed-acid, event-driven-architecture, caching, api-gateway

Stage 1 — Two-Phase Commit: How a Blocking Protocol Achieves Distributed Atomicity

• Two-Phase Commit uses a coordinator — one service or middleware node — that orchestrates a global commit across multiple participants (individual databases or services).
• The protocol unfolds in two strict phases.
• Phase 1 — Prepare: The coordinator sends a PREPARE message to every participant.
• Each participant writes its data to a durable write-ahead log, acquires all needed locks, and votes.

Observable outcome: Each participant writes its data to a durable write-ahead log, acquires all needed locks, and votes.

Stage 2 — XA Transactions: The Database Industry Standard for 2PC

• XA is the X/Open distributed transaction standard that specifies how a transaction manager coordinates multiple resource managers (databases) using the 2PC protocol.
• In Java, the Java Transaction API (JTA) is the primary implementation.
• Most relational databases — PostgreSQL, MySQL, Oracle, IBM DB2 — support XA.
• In a JTA setup, the application server acts as the transaction manager, automatically detecting and enlisting each XADataSource connection opened within a UserTransaction.begin() / commit() block.

Observable outcome: In a JTA setup, the application server acts as the transaction manager, automatically detecting and enlisting each XADataSource connection opened within a UserTransaction.begin() / commit() block.

Stage 3 — Deep Dive: The Saga Pattern — Long-Running Transactions Without Global Locks

• A Saga decomposes a distributed transaction into a sequence of local transactions , each fully committed by exactly one service.
• If a later step fails, the saga triggers compensating transactions in reverse order — new, forward-moving database writes that undo the business effect of prior steps.
• A saga never holds locks across services.
• Each local transaction commits and releases its locks immediately.

sequenceDiagram
    participant T1 as Transaction 1
    participant DB as Database
    participant T2 as Transaction 2

    T1->>DB: Read X (value=100)
    T2->>DB: Read X (value=100)
    T1->>DB: Write X = 50
    T2->>DB: Write X = 75
    DB->>DB: Conflict!

Observable outcome: Each local transaction commits and releases its locks immediately.

Stage 4 — Visualizing Distributed Transaction Flows

• The Two-Phase Commit Protocol in Sequence This sequence diagram shows the nominal path: both participants vote yes, the coordinator writes its commit decision to its own WAL (making the decision durable), then broadcast…
• The uncertainty window is the gap between the coordinator writing its commit record and each participant receiving the COMMIT message — if the coordinator crashes inside this window, participants are stuck with locks he…
• The Saga Compensation Flow: Happy Path and Failure Recovery This flowchart shows both the happy path (all three local transactions succeed, saga ends cleanly) and the failure path (payment fails after inventory was alre…
• The compensation chain runs in strict reverse: ReleaseInventory fires before CancelOrder because the inventory reservation was the earlier step — releasing it first restores the earlier invariant before the order record…

sequenceDiagram
    participant C as Coordinator
    participant P1 as Payment DB
    participant P2 as Order DB

    C->>P1: PREPARE
    C->>P2: PREPARE
    P1-->>C: VOTE_COMMIT
    P2-->>C: VOTE_COMMIT
    Note over C: All votes = COMMIT  write commit record to WAL
    C->>P1: COMMIT
    C->>P2: COMMIT
    P1-->>C: ACK
    P2-->>C: ACK
    Note over P1,P2: Both participants commit and release locks

flowchart TD
    A[ Order Placed] --> B[Reserve Inventory local tx  commits immediately]
    B -->|" InventoryReservedEvent"| C[Authorize Payment local tx  commits immediately]
    C -->|" PaymentAuthorizedEvent"| D[Create Shipment local tx  commits immediately]
    D --> E[ Order Confirmed  Saga Complete]

    B -->|" OutOfStockEvent"| F[Cancel Order no prior steps to compensate]
    F --> I[ Saga Ended  Insufficient Stock]

    C -->|" PaymentDeclinedEvent"| G[Release Inventory  compensating tx  reverse Step 1]
    G --> H[Cancel Order  compensating tx  close saga]
    H --> J[ Saga Compensated  Payment Declined]

Observable outcome: The compensation chain runs in strict reverse: ReleaseInventory fires before CancelOrder because the inventory reservation was the earlier step — releasing it first restores the earlier invariant before the order record…

Stage 5 — Real-World Applications: How Major Systems Handle Distributed Transaction Consistency

• Uber's Trip Lifecycle — Choreography Saga: Uber's core trip creation uses a choreography-based saga across Matching, Trip, Payment, and Driver Pay services.
• Each service emits domain events rather than calling upstream services directly.
• When a payment pre-authorization fails, the Trip service receives a PaymentFailedEvent and compensates by releasing the matched driver and cancelling the trip.
• There is no central orchestrator — the workflow emerges from event reactions.

Observable outcome: There is no central orchestrator — the workflow emerges from event reactions.

Stage 6 — The Outbox Pattern: Atomically Publishing Events Alongside Database Writes

• The Saga pattern immediately creates a dual-write problem : a service must both commit its local transaction to its own database and publish an event to a message broker (Kafka, RabbitMQ) so downstream saga participants…
• These are two separate writes.
• If the service crashes after the database commit but before the broker publish completes, the event is silently lost and the saga stalls permanently.
• The Outbox pattern solves this by reducing the dual-write to a single atomic database transaction: The outbox table lives in the same database as the service's business data.

Observable outcome: The Outbox pattern solves this by reducing the dual-write to a single atomic database transaction: The outbox table lives in the same database as the service's business data.

End-to-end outcome

In 2018, a major e-commerce platform discovered that customers had been charged for orders that were never created.

Alternatives considered

Decision Guide: Choosing Your Distributed Transaction Strategy

When eventual consistency is acceptable: In most e-commerce, booking, and logistics workflows, a gap of seconds to minutes of cross-service inconsistency is a modeled, expected state — not data corruption.

Failure and degraded modes

Trade-offs and Failure Modes: 2PC, XA, Saga Orchestration, and Saga Choreography Compared

• Detection: Trade-offs and Failure Modes: 2PC, XA, Saga Orchestration, and Saga Choreography Compared
• User or operator impact: Trade-offs and Failure Modes: 2PC, XA, Saga Orchestration, and Saga Choreography Compared

Lessons From Production Distributed Transaction Failures

• Detection: Don't retrofit 2PC onto microservices.
• User or operator impact: Don't retrofit 2PC onto microservices.