A Guide to Raft, Paxos, and Consensus Algorithms
How distributed state machines elect leaders, replicate logs, and survive split-brain network partitions
Trace how Raft consensus maintains a replicated log across a 5-node cluster, handles randomized election timeouts, and guarantees safety during network splits.
🎯 System Scenario & Requirements
System Goal
Maintain an immutable, linearizable consensus log across 5 distributed cluster nodes during network partitions.
Actors & Components
- Raft Leader (Node A)
- Follower Nodes (Nodes B, C, D, E)
- Replicated State Machine
Functional Requirements
- Leader Completeness: If a log entry is committed in a given term, that entry will be present in logs of leaders for all higher terms.
- Election Safety: At most one leader can be elected per term.
- Majority Quorum: Progress requires at least 3 of 5 active nodes.
Constraints & Assumptions
- Network delay variance can cause split-vote elections if timeouts are uniform.
⚡ Execution Stages & Decision Trace
Stage 1: Leader Election via Term Votes
Node A heartbeat timer expires. Node A transitions to Candidate, increments Term to 4, and requests votes.
Nodes B and C grant votes to Node A (first candidate seen in Term 4). Node A collects majority (3 votes).
📊 System Outcome & Trade-offs
Observed User & Operator Behavior
User: State machine updates execute with linearizable consistency.
Operator: Cluster automatically re-elects new leader in under 300ms following node crash.
Key Architectural Trade-offs
- Strict majority consensus prevents split-brain but requires 3 active nodes out of 5 to accept writes.
🔄 Evaluated Alternatives
Multi-Paxos
Equivalent safety guarantees but higher implementation complexity compared to Raft decomposed state machine.
⚠️ Failure Modes & Degraded Operation
Cluster Network Partition (3 vs 2)
Majority partition (3 nodes) continues committing log entries; minority partition (2 nodes) cannot reach quorum and rejects client writes.
Key learnings
- Consensus exists to agree on one order of decisions despite crashes and slow nodes, not to make every replica identically fresh at every moment.
- Leader-based protocols (Raft) are easier to reason about operationally; understand election, log replication, and commit index as the core loop.
- A majority quorum is the durability and safety boundary — minority replicas cannot safely acknowledge committed state alone.
- Timeouts and leader leases are availability knobs; set them for your RTT and failure detector, not from defaults copied across regions.
- Clients must retry with idempotency: leadership changes and reconfigurations will abort in-flight proposals.