Skip to main content
← PROBLEMS / SYSTEM-DESIGN
An end-to-end domain learning journey

Design a Multi-Region Session
& Token Revocation Service.

Pair short-lived stateless JWTs (5–15 min TTL) with an in-memory edge revocation filter, an authoritative multi-region session store, and async cross-region push invalidation with bounded retries and DLQ.
Start the journey7 stages
FIRST PRINCIPLES. PRODUCTION SCALE.
The idea to keep with you

Design a globally distributed session and JWT revocation service for 100M active users across 4 regions. Edge API gateways handle 2M auth checks/sec and must validate tokens with < 2 ms p99 overhead, while security-critical revocations (password reset, compromised device, admin force-logout) must propagate globally within 5 seconds—even during cross-region network partitions or cache-node failures.

From patterns to decisions.

Pair short-lived stateless JWTs (5–15 min TTL) with an in-memory edge revocation filter, an authoritative multi-region session store, and async cross-region push invalidation with bounded retries and DLQ.

Full walkthrough · 7 stages
01
STAGE 01 · REQUIREMENTS & MATHfoundation

Read/Write Asymmetry & Revocation Memory Footprint.

  • Read Path: 2M auth checks/sec globally (~500K RPS/region) at < 2 ms p99.
  • Write Path: ~1,000–5,000 revocations/sec (logouts, password resets, risk triggers).
  • Bounded State: With a 10-minute access token TTL, even 5,000 revocations/sec produces at most 5,000 × 600 = 3M active deny entries (~100 MB in compact hash/Redis memory) before TTL eviction.
02
STAGE 02 · TOKEN DESIGNfoundation

Short-Lived Access Tokens + User Session Epoch.

Include two compact claims in every 10-minute access JWT:

  • jti (Token ID): Enables surgical revocation of a single compromised device/session.
  • ver (User Security Epoch): A monotonically increasing integer per user. On password reset or account takeover, incrementing ver in one record invalidates all outstanding access and refresh tokens for that user without enumerating every jti.
03
STAGE 03 · EDGE HOT PATHintermediate

Sub-2 ms Edge Verification via Local Filter + Regional Redis.

Each Edge Gateway maintains an in-memory map of (revoked_jti -> exp) and (user_id -> min_valid_epoch) synced via streaming deltas from its regional Redis cluster:

  1. Verify JWT signature locally with cached JWKS.
  2. Check local in-memory filter (O(1), < 50 µs).
  3. If the edge filter is stale (last_sync_age > 5s), fall back to Regional Redis for high-risk endpoints (payments, password change) while allowing low-risk reads if configured.

Edge Gateway Sub-2 ms Token Verification Path

Inbound JWTLocal JWKS VerifyIn-Memory FilterRegional Redis
04
STAGE 04 · WRITE & PROPAGATIONintermediate

Authoritative Store, Cross-Region Stream & Push Invalidation.

When a user revokes a session or resets their password:

  1. Authoritative Write: Update the multi-region DB (or revoke refresh token family) transactionally.
  2. Publish Event: Emit { type: 'REVOKE_JTI' | 'BUMP_EPOCH', id, ttl } to a cross-region replicated stream (e.g., Kafka MirrorMaker / DynamoDB Streams).
  3. Regional Fan-Out: Workers in all 4 regions write to Regional Redis (SETEX with remaining JWT TTL) and push via pub/sub to Edge Gateways.
05
STAGE 05 · SAFE PROPAGATION CODEintermediate

Idempotent Regional Invalidation Worker with Bounded Retry & DLQ.

Each regional fan-out worker must apply epoch updates monotonically (never overwrite a newer epoch with a stale out-of-order event) and route persistent failures to a DLQ:

import time, random

def apply_revocation_event(redis_client, dlq, event, max_retries=4):
    for attempt in range(max_retries):
        try:
            if event['type'] == 'REVOKE_JTI':
                ttl = max(1, int(event['exp'] - time.time()))
                redis_client.setex(f"rev:jti:{event['jti']}", ttl, '1')
            elif event['type'] == 'BUMP_EPOCH':
                # Monotonic Lua/atomic guard so out-of-order events never regress min_epoch
                redis_client.eval(
                    "local cur = tonumber(redis.call('GET', KEYS[1]) or '0'); "
                    "if tonumber(ARGV[1]) > cur then redis.call('SET', KEYS[1], ARGV[1], 'EX', ARGV[2]) end;",
                    1, f"rev:epoch:{event['user_id']}", str(event['epoch']), str(event['ttl'])
                )
            return True
        except RedisTransientError:
            if attempt == max_retries - 1:
                dlq.publish(event, reason='regional_redis_write_failed')
                return False
            time.sleep(min(2.0, (0.05 * (2 ** attempt)) + random.uniform(0, 0.05)))
06
STAGE 06 · RESILIENCEadvanced

Handling Cross-Region Partitions & Refresh Token Rotation.

  • Refresh Token Reuse Detection: Refresh tokens are always validated against the authoritative store; if a rotated/revoked refresh token is replayed, immediately bump the user's epoch.
  • Partition Mode: If a region loses stream connectivity (sync_lag > 10s), high-risk endpoints (/transfer, /password) fail closed to the authoritative store while non-sensitive reads stay available.
07
STAGE 07 · GOTCHASadvanced

Clock Skew, Out-of-Order Epochs & Cache Stampedes.

  • Clock Skew: Add a 30–60s buffer to revocation SETEX TTLs so a token accepted by a slightly skewed edge clock cannot slip past an expired deny entry.
  • Out-of-Order Events: Always compare new_epoch > current_epoch atomically in Redis Lua.
  • JWKS Stampedes: Cache JWKS with background refresh + jitter rather than synchronous fetch on unknown kid.
Practice · Explain the consequences

Practice: Make the trade-off yourself.

Question 1 of 3
Score: 0/3

A user clicks 'Log out of all devices' after an account compromise, and they have 14 active sessions across mobile and web. How can the service revoke all 14 access JWTs in O(1) state?

Recommended Learning Flow