Real-time presence uses periodic WebSocket heartbeats into TTL-backed Redis keys (`SETEX 45s`) with online/offline hysteresis, paired with hybrid push/pull fan-out for small vs. massive channels.
TL;DR30-second simplified summary▾
Broadcasting online/offline status changes for 50M concurrent users to every contact on every flaky mobile reconnect triggers a quadratic fan-out storm (O(N * F) events/sec).
Production presence systems decouple Heartbeat Ingestion (clients ping every 30s, refreshing SETEX presence:user_id 45s in Redis) from Lazy Presence Subscriptions (clients only subscribe to visible friends/active DM headers) and apply Offline Hysteresis (wait 45s+ before flipping offline so brief tunnel drops don't flicker).
For message delivery, small groups/DMs (<= 500 members) push directly via Redis Pub/Sub to sticky WebSocket gateways, while 100K-member Discord guilds switch to Hybrid/Pull or sharded guild relays.
From patterns to decisions.
Real-time presence uses periodic WebSocket heartbeats into TTL-backed Redis keys (`SETEX 45s`) with online/offline hysteresis, paired with hybrid push/pull fan-out for small vs. massive channels.
Core Intuition & Failure Mode.
Real-time presence uses periodic WebSocket heartbeats into TTL-backed Redis keys (SETEX 45s) with online/offline hysteresis, paired with hybrid push/pull fan-out for small vs. massive channels.
Core Intuition & Failure Mode
1.Stage Objective: Understand the baseline failure mode.
2.Level (Foundation): Real-time presence uses periodic WebSocket heartbeats into TTL-backed Redis keys (`SETEX 45s`) with online/offline hysteresis, paired with h
3.Operational Invariant: Understand the baseline failure mode.
Caption: Understand the baseline failure mode.
Architectural Mechanics.
Production presence systems decouple Heartbeat Ingestion (clients ping every 30s, refreshing SETEX presence:user_id 45s in Redis) from Lazy Presence Subscriptions (clients only subscribe to visible friends/active DM headers) and apply Offline Hysteresis (wait 45s+ before flipping offline so brief tunnel drops don't flicker).
Architectural Mechanics
1.Stage Objective: Trace the core mechanism end-to-end.
2.Level (Foundation): Production presence systems decouple **Heartbeat Ingestion** (clients ping every 30s, refreshing `SETEX presence:user_id 45s` in Redis) from
3.Operational Invariant: Trace the core mechanism end-to-end.
Caption: Trace the core mechanism end-to-end.
Heartbeat TTL & Presence Batch Reader.
def refresh_presence(redis_client, user_id: str, gateway_id: str, ttl_sec: int = 45) -> None:
if not user_id or not gateway_id:
raise ValueError('user_id and gateway_id are required')
redis_client.set(f'presence:{user_id}', gateway_id, ex=ttl_sec)
def get_visible_presence(redis_client, friend_ids: list[str]) -> dict[str, bool]:
if not friend_ids:
return {}
keys = [f'presence:{uid}' for uid in friend_ids[:200]] # Cap batch size
vals = redis_client.mget(keys)
return {uid: (val is not None) for uid, val in zip(friend_ids[:200], vals)}Heartbeat TTL & Presence Batch Reader
1.Stage Objective: Refresh presence TTLs cleanly and batch-query visible contacts without fan-out storms.
2.Level (Intermediate): def refresh_presence(redis_client, user_id: str, gateway_id: str, ttl_sec: int = 45) -> None:
3.Operational Invariant: Refresh presence TTLs cleanly and batch-query visible contacts without fan-out storms.
Caption: Refresh presence TTLs cleanly and batch-query visible contacts without fan-out storms.
Throughput & Latency Trade-Offs.
Evaluate asymptotic bounds, cache locality, and tail-latency impact under peak load.
Throughput & Latency Trade-Offs
1.Stage Objective: Quantify operational complexity and memory bounds.
2.Level (Intermediate): Evaluate asymptotic bounds, cache locality, and tail-latency impact under peak load.
3.Operational Invariant: Quantify operational complexity and memory bounds.
Caption: Quantify operational complexity and memory bounds.
Blast-Radius & Failure Isolation.
Isolate shared resources, apply bounded queues, and prevent retry amplification.
Blast-Radius & Failure Isolation
1.Stage Objective: Contain failures without cascading across tenants or nodes.
2.Level (Advanced): Isolate shared resources, apply bounded queues, and prevent retry amplification.
3.Operational Invariant: Contain failures without cascading across tenants or nodes.
Caption: Contain failures without cascading across tenants or nodes.
Durability, Consistency & Recovery.
Use idempotent state transitions, atomic commits, and deterministic replay.
Durability, Consistency & Recovery
1.Stage Objective: Preserve invariants across crashes and restarts.
2.Level (Advanced): Use idempotent state transitions, atomic commits, and deterministic replay.
3.Operational Invariant: Preserve invariants across crashes and restarts.
Caption: Preserve invariants across crashes and restarts.
Telemetry, Circuit Breakers & Failure Modes.
Track p99 latency, retry/conflict ratios, and saturation metrics; fail fast with bounded retries and circuit breakers whenever invariants degrade.
Production Guardrails
1.Stage Objective: Monitor tail latency, error budgets, and invariant violations in production.
2.Level (Advanced): Track p99 latency, retry/conflict ratios, and saturation metrics; fail fast with bounded retries and circuit breakers whenever invariants de
3.Operational Invariant: Bounded retries, telemetry alerts, and circuit breakers prevent cascading failures.
Caption: Bounded retries, telemetry alerts, and circuit breakers prevent cascading failures.
Practice: Make the trade-off yourself.
Why do presence systems use a 45-second TTL with 30-second heartbeats?