Skip to main content
← CONCEPTS / AI & MACHINE LEARNING SYSTEMS
An end-to-end domain learning journey

Hybrid RAG: RRF Reranking
& Document-Level ACL Filtering.

Hybrid RAG combines dense HNSW vector search with sparse BM25 lexical search via Reciprocal Rank Fusion (RRF), enforces document-level ACLs via pre-filtered Roaring Bitmaps, and reranks Top-50 -> Top-5 with a Cross-Encoder.
Start the journey7 sections
FIRST PRINCIPLES. PRODUCTION SCALE.
The idea to keep with you

Hybrid RAG combines dense HNSW vector search with sparse BM25 lexical search via Reciprocal Rank Fusion (RRF), enforces document-level ACLs via pre-filtered Roaring Bitmaps, and reranks Top-50 -> Top-5 with a Cross-Encoder.

TL;DR30-second simplified summary
▾

Pure vector search excels at semantic paraphrases but fails on exact product SKUs, error codes (ERR_504_GATEWAY), and acronyms; pure BM25 catches exact keywords but misses synonyms. Hybrid RAG runs both in parallel and merges rankings with Reciprocal Rank Fusion (RRF): score(d) = sum(1 / (k + rank_i(d))).

In enterprise deployments, security is non-negotiable: if User A asks about Q4 layoffs and only 5 of 10M documents are accessible to User A, post-filtering Top-50 global results often yields 0 authorized chunks! Instead, pre-filtering with a Roaring Bitmap of the user's AD/Okta group IDs inside Qdrant/Milvus/Elasticsearch guarantees 100% authorized Top-K recall.

Finally, a Cross-Encoder Reranker scores (query, chunk) jointly for Top-50 -> Top-5 precision before citation-grounded LLM synthesis.

From patterns to decisions.

Hybrid RAG combines dense HNSW vector search with sparse BM25 lexical search via Reciprocal Rank Fusion (RRF), enforces document-level ACLs via pre-filtered Roaring Bitmaps, and reranks Top-50 -> Top-5 with a Cross-Encoder.

Full walkthrough · 7 stages
01
STAGE 01 · CORE INTUITIONfoundation

Core Intuition & Failure Mode.

Hybrid RAG combines dense HNSW vector search with sparse BM25 lexical search via Reciprocal Rank Fusion (RRF), enforces document-level ACLs via pre-filtered Roaring Bitmaps, and reranks Top-50 -> Top-5 with a Cross-Encoder.

STAGE 01 · CORE INTUITION

Core Intuition & Failure Mode

Core IntuitionIngress / BaselineFailure ModeScale ConstraintValidation GateBottleneck SignalProduction OutcomeTarget SLAexposes

1.Stage Objective: Understand the baseline failure mode.

2.Level (Foundation): Hybrid RAG combines dense HNSW vector search with sparse BM25 lexical search via Reciprocal Rank Fusion (RRF), enforces document-level ACLs

3.Operational Invariant: Understand the baseline failure mode.

Caption: Understand the baseline failure mode.

02
STAGE 02 · CORE MECHANICSfoundation

Architectural Mechanics.

In enterprise deployments, security is non-negotiable: if User A asks about Q4 layoffs and only 5 of 10M documents are accessible to User A, post-filtering Top-50 global results often yields 0 authorized chunks! Instead, pre-filtering with a Roaring Bitmap of the user's AD/Okta group IDs inside Qdrant/Milvus/Elasticsearch guarantees 100% authorized Top-K recall.

STAGE 02 · CORE MECHANICS

Architectural Mechanics

ArchitecturalMechanicsInput StateCore EngineCore MechanismGuard & VerifyInvariant CheckProduction OutcomeOutput Statetransforms

1.Stage Objective: Trace the core mechanism end-to-end.

2.Level (Foundation): In enterprise deployments, security is non-negotiable: if User A asks about Q4 layoffs and only 5 of 10M documents are accessible to User A,

3.Operational Invariant: Trace the core mechanism end-to-end.

Caption: Trace the core mechanism end-to-end.

03
STAGE 03 · SAFE IMPLEMENTATIONintermediate

Reciprocal Rank Fusion (RRF) Merger.

def reciprocal_rank_fusion(dense_ids: list[str], sparse_ids: list[str], k: int = 60, top_n: int = 50) -> list[tuple[str, float]]:
    if k <= 0 or top_n <= 0:
        raise ValueError('k and top_n must be positive')
    scores: dict[str, float] = {}
    for rank, doc_id in enumerate(dense_ids, start=1):
        scores[doc_id] = scores.get(doc_id, 0.0) + 1.0 / (k + rank)
    for rank, doc_id in enumerate(sparse_ids, start=1):
        scores[doc_id] = scores.get(doc_id, 0.0) + 1.0 / (k + rank)
    return sorted(scores.items(), key=lambda x: x[1], reverse=True)[:top_n]
STAGE 03 · SAFE IMPLEMENTATION

Reciprocal Rank Fusion (RRF) Merger

Reciprocal RankFusion RValidated InputCore EngineAtomic StepGuard & VerifyError / Retry GuardProduction OutcomeCommitted Resultexecutes

1.Stage Objective: Fuse dense and sparse rankings deterministically without needing score calibration.

2.Level (Intermediate): def reciprocal_rank_fusion(dense_ids: list[str], sparse_ids: list[str], k: int = 60, top_n: int = 50) -> list[tuple[str, float]]:

3.Operational Invariant: Fuse dense and sparse rankings deterministically without needing score calibration.

Caption: Fuse dense and sparse rankings deterministically without needing score calibration.

04
STAGE 04 · SCALING & TRADE-OFFSintermediate

Throughput & Latency Trade-Offs.

Evaluate asymptotic bounds, cache locality, and tail-latency impact under peak load.

STAGE 04 · SCALING & TRADE-OFFS

Throughput & Latency Trade-Offs

ThroughputHot PathLatency Trade-OffsPartition / IndexValidation GateLatency / RAM BoundProduction OutcomeScaled Statebounds

1.Stage Objective: Quantify operational complexity and memory bounds.

2.Level (Intermediate): Evaluate asymptotic bounds, cache locality, and tail-latency impact under peak load.

3.Operational Invariant: Quantify operational complexity and memory bounds.

Caption: Quantify operational complexity and memory bounds.

05
STAGE 05 · FAULT ISOLATIONadvanced

Blast-Radius & Failure Isolation.

Isolate shared resources, apply bounded queues, and prevent retry amplification.

STAGE 05 · FAULT ISOLATION

Blast-Radius & Failure Isolation

Blast-RadiusFault TriggerFailure IsolationIsolation BoundaryValidation GateFallback / QuarantineProduction OutcomeProtected Pathisolates

1.Stage Objective: Contain failures without cascading across tenants or nodes.

2.Level (Advanced): Isolate shared resources, apply bounded queues, and prevent retry amplification.

3.Operational Invariant: Contain failures without cascading across tenants or nodes.

Caption: Contain failures without cascading across tenants or nodes.

06
STAGE 06 · RECOVERY & CONSISTENCYadvanced

Durability, Consistency & Recovery.

Use idempotent state transitions, atomic commits, and deterministic replay.

STAGE 06 · RECOVERY & CONSISTENCY

Durability, Consistency & Recovery

DurabilityConsistencyCheckpoint / LogRecoveryReplay / SyncValidation GateConsistency GateProduction OutcomeRecovered Stateverifies

1.Stage Objective: Preserve invariants across crashes and restarts.

2.Level (Advanced): Use idempotent state transitions, atomic commits, and deterministic replay.

3.Operational Invariant: Preserve invariants across crashes and restarts.

Caption: Preserve invariants across crashes and restarts.

07
STAGE 07 · OBSERVABILITY & GUARDRAILSadvanced

Telemetry, Circuit Breakers & Failure Modes.

Track p99 latency, retry/conflict ratios, and saturation metrics; fail fast with bounded retries and circuit breakers whenever invariants degrade.

STAGE 07 · OBSERVABILITY & GUARDRAILS

Production Guardrails

ProductionGuardrailsTelemetry SignalCore EngineSLO ThresholdGuard & VerifyCircuit BreakerProduction OutcomeSafe Degradationtrips

1.Stage Objective: Monitor tail latency, error budgets, and invariant violations in production.

2.Level (Advanced): Track p99 latency, retry/conflict ratios, and saturation metrics; fail fast with bounded retries and circuit breakers whenever invariants de

3.Operational Invariant: Bounded retries, telemetry alerts, and circuit breakers prevent cascading failures.

Caption: Bounded retries, telemetry alerts, and circuit breakers prevent cascading failures.

Practice · Explain the consequences

Practice: Make the trade-off yourself.

Question 1 of 3
Score: 0/3

Why is ACL pre-filtering superior to post-filtering in enterprise RAG?

Recommended Learning Flow