**The "Request Hedging" Pattern: Betting on Latency to Win Reliability**

Scenario context

Request hedging sends duplicate requests to multiple replicas after a timeout, trading extra load for lower latency and higher reliability. ⏱️🔄 Why it matters: It’s a simple way to mask slow nodes without complex coordination. Imagine you’re building a distributed key-value store. Most reads return in 50ms, but 1% take 500ms—enough to ruin p99 SLAs. Retries help, but waiting for a timeout first means yo…

This walkthrough focuses on The Problem: The "Long Tail" of Latency.

Before making a design choice, clarify:

• Imagine you’re building a distributed key-value store.
• Most reads return in 50ms , but 1% take 500ms —enough to ruin p99 SLAs.
• Retries help, but waiting for a timeout first means you’re already late.

Concepts in play: circuit-breaker, consistency-replication

Stage 1 — The Solution: Hedge Your Bets

• Instead of waiting for a timeout, send the same request to multiple replicas simultaneously after a short delay (e.g., 20ms).
• The first successful response wins, and the others are canceled.
• This works because: - Temporal locality : If one node is slow, others are likely fast.
• • No coordination : No need for consensus or leader election.

Observable outcome: - No coordination : No need for consensus or leader election.

Stage 2 — Code in Action: Hedging with gRPC

• func (c Client) Get(key string) (string, error) { ctx, cancel := context.WithTimeout(context.Background(), 100 time.Millisecond) defer cancel() // First attempt resp, err := c.primary.Get(ctx, key) if err == nil { retur…

func (c *Client) Get(key string) (string, error) {
    ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
    defer cancel()

    // First attempt
    resp, err := c.primary.Get(ctx, key)
    if err == nil {
        return resp, nil
    }

    // Hedge after 20ms
    time.Sleep(20 * time.Millisecond)
    hedgedResp, hedgedErr := c.replica.Get(ctx, key)
    if hedgedErr == nil {
        return hedgedResp, nil
    }

    return "", fmt.Errorf("both attempts failed: %v, %v", err, hedgedErr)
}

Observable outcome: func (c Client) Get(key string) (string, error) { ctx, cancel := context.WithTimeout(context.Background(), 100 time.Millisecond) defer cancel() // First attempt resp, err := c.primary.Get(ctx, key) if err == nil { retur…

Stage 3 — When to Use It

• Key Takeaway : Hedging is like buying insurance—you pay a small premium (extra load) to avoid catastrophic losses (slow responses).
• Use it when latency is critical and operations are safe to repeat.

Observable outcome: Use it when latency is critical and operations are safe to repeat.

End-to-end outcome

Imagine you’re building a distributed key-value store.

Alternatives considered

The source focuses on one primary design. A second architecture is not compared, so no alternative is invented here.

Failure and degraded modes

Trade-offs: Load vs. Latency

• Detection: - Pros : Cuts tail latency by 50-90% in practice (e.g., Google’s tail-at-scale).
• User or operator impact: - Pros : Cuts tail latency by 50-90% in practice (e.g., Google’s tail-at-scale).

• Instead of waiting for a timeout, send the same request to multiple replicas simultaneously after a short delay (e.g., 20ms).
• func (c Client) Get(key string) (string, error) { ctx, cancel := context.WithTimeout(context.Background(), 100 time.Millisecond) defer cancel() // First attempt resp, err := c.primary.Get(ctx, key) if err == nil { retur…
• Key Takeaway : Hedging is like buying insurance—you pay a small premium (extra load) to avoid catastrophic losses (slow responses).