ANN Index Types Explained: When to Choose Flat, HNSW, IVF, or IVF-PQ

Scenario context

If your dataset is small and correctness is critical, use Flat. If you need high recall with low latency and enough RAM, use HNSW. If your corpus is huge and memory is your bottleneck, use IVF-PQ. Start with HNSW for most RAG systems, then move to IVF or IVF-PQ when scale and cost force compression.

Observable outcome: Compute approximate similarity over those candidates.

Stage 4 — Architecture and Workflow: How ANN Index Selection Fits into a Retrieval Stack

• <!-- This section is now handled by backmatter mermaidDiagram.

Observable outcome: <!-- This section is now handled by backmatter mermaidDiagram.

Stage 5 — Deep Dive: Tuning Knobs That Actually Move Outcomes

• You can spend weeks tuning random parameters.
• Focus on the small set that directly moves latency-recall behavior.
• The Internals HNSW Parameters - M : number of bi-directional edges per node.
• • Higher M improves recall but increases memory and build time.

Observable outcome: - Higher M improves recall but increases memory and build time.

Stage 6 — Practical Example 1: Building and Querying HNSW with hnswlib

• This example demonstrates a realistic online-serving setup: build an HNSW index, tune efSearch , and query top-k neighbors.
• Watch how efSearch changes recall-latency behavior.
• What to observe: - Increasing efSearch usually raises recall and latency together.
• • Memory consumption is driven heavily by M and corpus size.

import numpy as np
import hnswlib
import time

# Synthetic dataset: 1M vectors in production is common, kept small here for demo speed.
num_elements = 100_000
dim = 384
rng = np.random.default_rng(42)

data = rng.normal(size=(num_elements, dim)).astype(np.float32)
queries = rng.normal(size=(100, dim)).astype(np.float32)

# Build HNSW index
index = hnswlib.Index(space='cosine', dim=dim)
index.init_index(max_elements=num_elements, ef_construction=200, M=32)
index.add_items(data, np.arange(num_elements))
index.set_ef(80)  # query-time breadth; tune this for recall/latency

# Query benchmark
k = 10
start = time.perf_counter()
labels, distances = index.knn_query(queries, k=k)
elapsed_ms = (time.perf_counter() - start) * 1000

print(f'Queried {len(queries)} queries, top-{k}, total_time_ms={elapsed_ms:.2f}')
print('First query nearest IDs:', labels[0][:5])

Observable outcome: - Memory consumption is driven heavily by M and corpus size.

End-to-end outcome

After working through the stages, you should be able to explain:

• • Index choice is workload design, not library preference.
• • Always benchmark against Flat before accepting ANN tradeoffs.
• • Flat: exact, simple, expensive at scale.
• • HNSW: best default for online RAG when recall matters.

Alternatives considered

Vector Index Comparison: Latency, Recall, Memory, and Operations

Quick takeaway: - HNSW wins many production workloads where quality matters and RAM is acceptable.

Failure and degraded modes

The source does not isolate a separate failure mode. Limits that matter for the scenario are called out in the stage decisions above.