Hybrid RAG combines dense HNSW vector search with sparse BM25 lexical search via Reciprocal Rank Fusion (RRF), enforces document-level ACLs via pre-filtered Roaring Bitmaps, and reranks Top-50 -> Top-5 with a Cross-Encoder.
TL;DR30-second simplified summary▾
Pure vector search excels at semantic paraphrases but fails on exact product SKUs, error codes (ERR_504_GATEWAY), and acronyms; pure BM25 catches exact keywords but misses synonyms. Hybrid RAG runs both in parallel and merges rankings with Reciprocal Rank Fusion (RRF): score(d) = sum(1 / (k + rank_i(d))).
In enterprise deployments, security is non-negotiable: if User A asks about Q4 layoffs and only 5 of 10M documents are accessible to User A, post-filtering Top-50 global results often yields 0 authorized chunks! Instead, pre-filtering with a Roaring Bitmap of the user's AD/Okta group IDs inside Qdrant/Milvus/Elasticsearch guarantees 100% authorized Top-K recall.
Finally, a Cross-Encoder Reranker scores (query, chunk) jointly for Top-50 -> Top-5 precision before citation-grounded LLM synthesis.
From patterns to decisions.
Hybrid RAG combines dense HNSW vector search with sparse BM25 lexical search via Reciprocal Rank Fusion (RRF), enforces document-level ACLs via pre-filtered Roaring Bitmaps, and reranks Top-50 -> Top-5 with a Cross-Encoder.
Core Intuition & Failure Mode.
Hybrid RAG combines dense HNSW vector search with sparse BM25 lexical search via Reciprocal Rank Fusion (RRF), enforces document-level ACLs via pre-filtered Roaring Bitmaps, and reranks Top-50 -> Top-5 with a Cross-Encoder.
Core Intuition & Failure Mode
1.Stage Objective: Understand the baseline failure mode.
2.Level (Foundation): Hybrid RAG combines dense HNSW vector search with sparse BM25 lexical search via Reciprocal Rank Fusion (RRF), enforces document-level ACLs
3.Operational Invariant: Understand the baseline failure mode.
Caption: Understand the baseline failure mode.
Architectural Mechanics.
In enterprise deployments, security is non-negotiable: if User A asks about Q4 layoffs and only 5 of 10M documents are accessible to User A, post-filtering Top-50 global results often yields 0 authorized chunks! Instead, pre-filtering with a Roaring Bitmap of the user's AD/Okta group IDs inside Qdrant/Milvus/Elasticsearch guarantees 100% authorized Top-K recall.
Architectural Mechanics
1.Stage Objective: Trace the core mechanism end-to-end.
2.Level (Foundation): In enterprise deployments, security is non-negotiable: if User A asks about Q4 layoffs and only 5 of 10M documents are accessible to User A,
3.Operational Invariant: Trace the core mechanism end-to-end.
Caption: Trace the core mechanism end-to-end.
Reciprocal Rank Fusion (RRF) Merger.
def reciprocal_rank_fusion(dense_ids: list[str], sparse_ids: list[str], k: int = 60, top_n: int = 50) -> list[tuple[str, float]]:
if k <= 0 or top_n <= 0:
raise ValueError('k and top_n must be positive')
scores: dict[str, float] = {}
for rank, doc_id in enumerate(dense_ids, start=1):
scores[doc_id] = scores.get(doc_id, 0.0) + 1.0 / (k + rank)
for rank, doc_id in enumerate(sparse_ids, start=1):
scores[doc_id] = scores.get(doc_id, 0.0) + 1.0 / (k + rank)
return sorted(scores.items(), key=lambda x: x[1], reverse=True)[:top_n]Reciprocal Rank Fusion (RRF) Merger
1.Stage Objective: Fuse dense and sparse rankings deterministically without needing score calibration.
2.Level (Intermediate): def reciprocal_rank_fusion(dense_ids: list[str], sparse_ids: list[str], k: int = 60, top_n: int = 50) -> list[tuple[str, float]]:
3.Operational Invariant: Fuse dense and sparse rankings deterministically without needing score calibration.
Caption: Fuse dense and sparse rankings deterministically without needing score calibration.
Throughput & Latency Trade-Offs.
Evaluate asymptotic bounds, cache locality, and tail-latency impact under peak load.
Throughput & Latency Trade-Offs
1.Stage Objective: Quantify operational complexity and memory bounds.
2.Level (Intermediate): Evaluate asymptotic bounds, cache locality, and tail-latency impact under peak load.
3.Operational Invariant: Quantify operational complexity and memory bounds.
Caption: Quantify operational complexity and memory bounds.
Blast-Radius & Failure Isolation.
Isolate shared resources, apply bounded queues, and prevent retry amplification.
Blast-Radius & Failure Isolation
1.Stage Objective: Contain failures without cascading across tenants or nodes.
2.Level (Advanced): Isolate shared resources, apply bounded queues, and prevent retry amplification.
3.Operational Invariant: Contain failures without cascading across tenants or nodes.
Caption: Contain failures without cascading across tenants or nodes.
Durability, Consistency & Recovery.
Use idempotent state transitions, atomic commits, and deterministic replay.
Durability, Consistency & Recovery
1.Stage Objective: Preserve invariants across crashes and restarts.
2.Level (Advanced): Use idempotent state transitions, atomic commits, and deterministic replay.
3.Operational Invariant: Preserve invariants across crashes and restarts.
Caption: Preserve invariants across crashes and restarts.
Telemetry, Circuit Breakers & Failure Modes.
Track p99 latency, retry/conflict ratios, and saturation metrics; fail fast with bounded retries and circuit breakers whenever invariants degrade.
Production Guardrails
1.Stage Objective: Monitor tail latency, error budgets, and invariant violations in production.
2.Level (Advanced): Track p99 latency, retry/conflict ratios, and saturation metrics; fail fast with bounded retries and circuit breakers whenever invariants de
3.Operational Invariant: Bounded retries, telemetry alerts, and circuit breakers prevent cascading failures.
Caption: Bounded retries, telemetry alerts, and circuit breakers prevent cascading failures.
Practice: Make the trade-off yourself.
Why is ACL pre-filtering superior to post-filtering in enterprise RAG?