Your Redis 7 primary serves 85,000 OPS at 0.8ms p99 latency. A feature-flag/catalog cache stores 2.5 million entries inside a single Redis Hash (`catalog:items`, ~480 MB RSS). Whenever a daily refresh job runs `DEL catalog:items` or its 24-hour TTL expires, all clients experience 820ms-1,200ms latency spikes, triggering 200ms SocketTimeoutExceptions and cascading application retries. Redesign the invalidation mechanism and data layout so cache refreshes and expirations never stall the Redis event loop above 2ms.
From patterns to decisions.
Your Redis 7 primary serves 85,000 OPS at 0.8ms p99 latency. A feature-flag/catalog cache stores 2.5 million entries inside a single Redis Hash (`catalog:items`, ~480 MB RSS). Whenever a daily refresh job runs `DEL catalog:items` or its 24-hour TTL expires, all clients experience 820ms-1,200ms latency spikes, triggering 200ms SocketTimeoutExceptions and cascading application retries. Redesign the invalidation mechanism and data layout so cache refreshes and expirations never stall the Redis event loop above 2ms.
Redis's aeMain event loop is.
Redis's aeMain event loop is single-threaded; DEL synchronously walks and frees all 2.5M hash entries before processing the next queued command.
| Choice | What it buys | What it costs |
|---|---|---|
| Trade-off 1 | Synchronous DEL reclaims memory immediately, whereas UNLINK defers reclamation by a few milliseconds to bio_lazy_free. |
Step 1
Recognize that Redis executes commands sequentially on one main thread.
Step 2
Note that DEL is O(N) for collection types (Hash, Set, ZSet, List), where N = 2,500,000.
Step 3
Connect the 820ms stall to 200ms client SocketTimeoutExceptions and retry amplification.
Alternative 1
Chunked HSCAN + HDEL in batches of 500 fields.
Failure Mode 1
200ms client timeouts trigger application retries, multiplying load when Redis unfreezes.
# Anti-pattern: blocks Redis aeMain for ~820ms on 2.5M fields
# redis_client.delete("catalog:items")
# Safe: O(1) unlink (~0.08ms), memory freed in bio_lazy_free thread
# redis_client.unlink("catalog:items")
Shard into 64 buckets via.
Shard into 64 buckets via CRC32, swap an active_ver pointer atomically, and reclaim old buckets with UNLINK (or chunked HSCAN+HDEL for legacy cleanup).
| Choice | What it buys | What it costs |
|---|---|---|
| Trade-off 1 | 64 buckets keep each key ~7.5 MB, eliminating big-key stalls on both reads and TTL expirations. | |
| Trade-off 2 | Requires 2x memory (~960 MB) briefly during Blue-Green refresh. |
Step 1
Compute bucket = zlib.crc32(item_id.encode()) % 64.
Step 2
Write new version buckets in 500-entry pipelines, flip active_ver, and UNLINK old buckets.
Step 3
Use HSCAN COUNT 500 + HDEL with 1ms sleep for safe in-place legacy big-key removal.
Alternative 1
Storing each item as an independent top-level string key (catalog:item:{id}) if per-item TTLs are needed.
Failure Mode 1
Forgetting to set lazyfree-lazy-expire yes still causes stalls if a bucket's TTL expires.
import time
import zlib
from typing import Dict, Iterable
NUM_BUCKETS = 64
def bucket_key(version: str, item_id: str) -> str:
bucket = zlib.crc32(item_id.encode("utf-8")) % NUM_BUCKETS
return f"catalog:{version}:{bucket:02d}"
def refresh_catalog_blue_green(r, new_ver: str, old_ver: str, items: Iterable[tuple[str, str]]) -> None:
pipe = r.pipeline(transaction=False)
count = 0
for item_id, payload in items:
pipe.hset(bucket_key(new_ver, item_id), item_id, payload)
count += 1
if count % 500 == 0:
pipe.execute()
if count % 500 != 0:
pipe.execute()
r.set("catalog:active_ver", new_ver)
old_keys = [f"catalog:{old_ver}:{b:02d}" for b in range(NUM_BUCKETS)]
r.unlink(*old_keys)
def chunked_delete_big_hash(r, key: str, batch_size: int = 500) -> None:
cursor = 0
while True:
cursor, data = r.hscan(key, cursor=cursor, count=batch_size)
if data:
r.hdel(key, *data.keys())
if cursor == 0:
break
time.sleep(0.001)
r.unlink(key)
64 sharded Hashes use ~40%.
64 sharded Hashes use ~40% less RAM via listpack/dict overhead sharing and allow fast versioned UNLINK, whereas individual keys are required only when items need independent TTLs.
| Choice | What it buys | What it costs |
|---|---|---|
| Trade-off 1 | Top-level keys support per-item EXPIRE; sharded Hashes save ~240 MB RAM and simplify Blue-Green version swaps. |
Compare memory overhead
2.5M top-level keys add 2.5M redisObject + dictEntry headers (~240 MB extra overhead).
Check TTL granularity
catalog refreshes expire together, so bucket-level TTL + version pointer is ideal.
Alternative 1
1,024 smaller Hashes using listpack encoding (hash-max-listpack-entries 512) for even tighter memory.
Failure Mode 1
Deleting 2.5M individual keys requires SCAN + UNLINK loops instead of 64 keys.
# 2.5M top-level keys: ~96 bytes metadata overhead each = ~240 MB extra RAM
# 64 sharded Hashes: 64 top-level keys, shared dict overhead, O(1) version pointer swap
Enable replica-lazy-flush yes in redis.
Enable replica-lazy-flush yes in redis.conf so the replica flushes its old dataset asynchronously in bio_lazy_free before loading the incoming RDB.
| Choice | What it buys | What it costs |
|---|---|---|
| Trade-off 1 | Async flush on replica briefly doubles memory until bio_lazy_free finishes. |
Step 1
Identify that during full resync, a replica must flush its entire existing keyspace before loading the primary's RDB.
Step 2
With replica-lazy-flush no, that flush blocks the replica's main thread for seconds, failing health checks.
Alternative 1
Diskless replication (repl-diskless-sync yes) to speed up replica catch-up.
Failure Mode 1
Read replicas serving traffic during resync fail readiness probes if flush blocks synchronously.
# Required redis.conf directives:
# lazyfree-lazy-eviction yes
# lazyfree-lazy-expire yes
# lazyfree-lazy-server-del yes
# replica-lazy-flush yes
Run redis-cli --bigkeys -i 0.
Run redis-cli --bigkeys -i 0.1 against a read replica (never the primary) or sample MEMORY USAGE key SAMPLES 5 during off-peak hours.
| Choice | What it buys | What it costs |
|---|---|---|
| Trade-off 1 | Running `--bigkeys` on a replica with `-i 0.1` adds <1% CPU overhead and zero primary impact. |
Step 1
Target a read replica with -i 0.1 (100ms sleep every 100 SCAN iterations).
Step 2
Monitor LATENCY LATEST and INFO commandstats for del/expire max_usec spikes.
Step 3
Alert when any key exceeds 10 MB or 10,000 elements.
Alternative 1
Offline RDB analysis (rdbtools) on a nightly backup snapshot.
Failure Mode 1
Running MEMORY USAGE without SAMPLES 5 on a 2.5M-element key performs an O(N) full traversal.
# Safe replica-only sampling check
def sample_key_bytes(replica_conn, key: str) -> int:
return replica_conn.memory_usage(key, samples=5) or 0
Theory & Tips and Tricks
Practical heuristics & interview traps
- 01Interview anchor: Replace DEL with async UNLINK, enable lazyfree-lazy-expire/eviction, and shard multi-million-entry collections into 64 hash buckets (< 8 MB each) or delete in 500-field HSCAN+HDEL batches.
- 02Trade-off tip: Top-level keys support per-item EXPIRE; sharded Hashes save ~240 MB RAM and simplify Blue-Green version swaps.
- 03Watch out for: Read replicas serving traffic during resync fail readiness probes if flush blocks synchronously.
Redis Big Key Event Loop Freezes — Normal → Load / Fault → Recovery
Replace DEL with async UNLINK, enable lazyfree-lazy-expire/eviction, and shard multi-million-entry collections into 64 hash buckets (< 8 MB each) or delete in 500-field HSCAN+HDEL batches.
Putting Preventing Redis Big-Key Event Loop Freezes Together
Replace DEL with async UNLINK, enable lazyfree-lazy-expire/eviction, and shard multi-million-entry collections into 64 hash buckets (< 8 MB each) or delete in 500-field HSCAN+HDEL batches.
Putting Preventing Redis Big-Key Event Loop Freezes Together
- Compute bucket = zlib.crc32(item_id.encode()) % 64.
- Write new version buckets in 500-entry pipelines, flip active_ver, and UNLINK old buckets.
- Use HSCAN COUNT 500 + HDEL with 1ms sleep for safe in-place legacy big-key removal.
Interview Readiness Check
Enable replica-lazy-flush yes in redis.conf so the replica flushes its old dataset asynchronously in bio_lazy_free before loading the incoming RDB.
Walk through bottlenecks, back-of-the-envelope math, and failure recovery.
Make the trade-off yourself.
Practice Check
Test your understanding
When is ACID Transaction Fundamentals the right default for a production team?
Preventing Redis Big-Key Event Loop Freezes
Replace DEL with async UNLINK, enable lazyfree-lazy-expire/eviction, and shard multi-million-entry collections into 64 hash buckets (< 8 MB each) or delete in 500-field HSCAN+HDEL batches.
