A multi-tenant e-commerce catalog cluster on Elasticsearch 8.11 (12 data nodes, 64 GB RAM / 31 GB JVM heap each, 20 primary shards with 1 replica) experiences 18-second G1GC stop-the-world pauses and HTTP 503 circuit-breaker rejections whenever scrapers or export jobs paginate past page 500 (`from=50000, size=100`). Redesign the pagination architecture, query limits, and client export path to eliminate coordinator OOMs and GC pauses while preserving deterministic ordering under concurrent writes.
From patterns to decisions.
A multi-tenant e-commerce catalog cluster on Elasticsearch 8.11 (12 data nodes, 64 GB RAM / 31 GB JVM heap each, 20 primary shards with 1 replica) experiences 18-second G1GC stop-the-world pauses and HTTP 503 circuit-breaker rejections whenever scrapers or export jobs paginate past page 500 (`from=50000, size=100`). Redesign the pagination architecture, query limits, and client export path to eliminate coordinator OOMs and GC pauses while preserving deterministic ordering under concurrent writes.
In scatter-gather execution, every shard.
In scatter-gather execution, every shard must compute and return from + size sort tuples because shards do not know the global rank distribution.
| Choice | What it buys | What it costs |
|---|---|---|
| Trade-off 1 | Capping `max_result_window` at 2000 protects JVM heap immediately but breaks any legacy client expecting `from > 2000`. |
Trace the Query Phase
coordinator broadcasts from + size = 50,100 to all 20 primary/replica shards.
Calculate per-query merge memory
20 shards * 50,100 entries = 1,002,000 ScoreDoc tuples allocated on the coordinator.
Step 3
Connect large contiguous array allocations to G1GC humongous region pressure and stop-the-world Full GC pauses.
Alternative 1
Using _scroll (deprecated for real-time user queries and pins old segments without flexible sorting).
Failure Mode 1
Raising max_result_window to 100,000+ merely hides the validation error while guaranteeing coordinator OOMs under concurrency.
# Index template guardrail to block deep offset pagination
index_patterns: ["catalog-*"]
template:
settings:
index:
max_result_window: 2000
number_of_shards: 20
number_of_replicas: 1
A Python service that opens.
A Python service that opens a 2-minute PIT, paginates with search_after + product_id tie-breaker, refreshes the PIT via the response, and closes it on completion.
| Choice | What it buys | What it costs |
|---|---|---|
| Trade-off 1 | Using `product_id` instead of `_shard_doc` adds doc-values lookup overhead on ties but allows seamless recovery if the PIT expires. |
Step 1
Open a PIT handle with keep_alive=2m before the first page.
Query `/_search` passing `pit
{id, keep_alive} and track_total_hits: false`.
Step 3
Use product_id as a deterministic tie-breaker so cursor recovery works even if a PIT expires.
Step 4
Close the PIT with DELETE /_pit in a finally block for batch exporters.
Alternative 1
Using _shard_doc when PIT expiration recovery is not required, as Elasticsearch appends it automatically and reads it faster.
Forgetting `track_total_hits
false` forces Lucene to traverse all matching postings to compute exact totals.
import base64, json
from elasticsearch import Elasticsearch
def fetch_page(es: Elasticsearch, index: str, cursor: str | None, size: int = 100):
if cursor:
state = json.loads(base64.urlsafe_b64decode(cursor))
pit_id, search_after = state["pit_id"], state["sort"]
else:
pit = es.open_point_in_time(index=index, keep_alive="2m")
pit_id, search_after = pit["id"], None
body = {
"size": size,
"pit": {"id": pit_id, "keep_alive": "2m"},
"track_total_hits": False,
"sort": [{"updated_at": "desc"}, {"product_id": "asc"}],
}
if search_after:
body["search_after"] = search_after
resp = es.search(body=body)
hits = resp["hits"]["hits"]
next_cursor = None
if len(hits) == size:
next_state = {"pit_id": resp["pit_id"], "sort": hits[-1]["sort"]}
next_cursor = base64.urlsafe_b64encode(json.dumps(next_state).encode()).decode()
return hits, next_cursor
search_after + PIT is the.
search_after + PIT is the only pattern offering O(size) heap, snapshot consistency, and safe stateless API cursors.
| Choice | What it buys | What it costs |
|---|---|---|
| Trade-off 1 | Hybrid routing preserves clickable page numbers 1-20 for UX while protecting the cluster from deep-offset abuse. |
Step 1
Keep from/size strictly for UI pages 1-20 (offset <= 2000) where users click page numbers.
Step 2
Use search_after + PIT for infinite scroll, API cursors, and parallel sliced exports.
Step 3
Eliminate _scroll due to stateful server-side context pinning and lack of real-time query flexibility.
Alternative 1
Cursor-only APIs from page 1, simplifying backend code at the cost of removing page-number buttons.
Failure Mode 1
Exposing raw pit_id without HMAC signing or size validation allows malicious clients to pass size=10000.
# Hybrid router: allow shallow page jumps up to 2000, enforce cursor beyond
MAX_OFFSET = 2000
def validate_pagination(offset: int, size: int, cursor: str | None):
if cursor is None and (offset + size) > MAX_OFFSET:
raise ValueError(f"Deep offset {offset + size} exceeds {MAX_OFFSET}; use cursor pagination.")
Open PITs pin old Lucene.
Open PITs pin old Lucene segments after merges, causing disk bloat and file-descriptor exhaustion if left unclosed or uncapped.
| Choice | What it buys | What it costs |
|---|---|---|
| Trade-off 1 | Reopening a PIT on expiry sacrifices strict point-in-time isolation across the boundary but prevents export job failure. |
Step 1
Cap keep_alive at 2 minutes and monitor indices.search.open_contexts per node.
Step 2
Isolate bulk exports to dedicated read-only replica/search nodes or throttled async job queues.
Step 3
Handle search_context_missing_exception by reopening a PIT and resuming from the deterministic (updated_at, product_id) sort key.
Alternative 1
Taking a daily snapshot to S3 and running offline exports via Spark/DuckDB instead of hitting the live search cluster.
Failure Mode 1
Leaving orphaned PITs open with keep_alive=30m prevents Lucene from reclaiming deleted documents, filling NVMe disks.
from elasticsearch import BadRequestError, NotFoundError
def safe_fetch_page(es, index, cursor, size=100):
try:
return fetch_page(es, index, cursor, size)
except (BadRequestError, NotFoundError) as exc:
if "search_context_missing_exception" in str(exc) and cursor:
state = json.loads(base64.urlsafe_b64decode(cursor))
fresh_pit = es.open_point_in_time(index=index, keep_alive="2m")["id"]
rebuilt = base64.urlsafe_b64encode(json.dumps({"pit_id": fresh_pit, "sort": state["sort"]}).encode()).decode()
return fetch_page(es, index, rebuilt, size)
raise
Use PIT slice partitioning (slice.
Use PIT slice partitioning (slice: {id, max}) aligned to a divisor of the shard count, combined with token-bucket rate limiting.
| Choice | What it buys | What it costs |
|---|---|---|
| Trade-off 1 | Slicing with `max=10` speeds up exports 10x, but unthrottled workers can saturate data-node disk I/O and CPU. |
Step 1
Open a single shared PIT for the entire export job so all workers read the exact same snapshot.
Step 2
Partition the export into max=5 or max=10 slices (dividing the 20 shards evenly) so each worker scans a subset of documents.
Step 3
Close the shared PIT once all slice workers finish.
Alternative 1
Reading directly from the source-of-truth Kafka topic or database CDC stream for downstream warehouse syncs.
Failure Mode 1
Setting max_slices larger than number_of_shards forces expensive _id hash filtering per document instead of per-shard partitioning.
def fetch_slice(es: Elasticsearch, pit_id: str, slice_id: int, max_slices: int, size: int = 500):
search_after = None
while True:
body = {
"size": size,
"pit": {"id": pit_id, "keep_alive": "2m"},
"slice": {"id": slice_id, "max": max_slices},
"sort": [{"_shard_doc": "asc"}],
}
if search_after:
body["search_after"] = search_after
hits = es.search(body=body)["hits"]["hits"]
if not hits:
break
yield hits
search_after = hits[-1]["sort"]
Theory & Tips and Tricks
Practical heuristics & interview traps
- 01Interview anchor: Replace offset-based
from/sizewith statelesssearch_afterpaired with a Point-in-Time (_pit) ID and a tie-breaker sort key (_shard_docorid) to keep memory O(size) per page. - 02Trade-off tip: Hybrid routing preserves clickable page numbers 1-20 for UX while protecting the cluster from deep-offset abuse.
- 03Watch out for: Leaving orphaned PITs open with
keep_alive=30mprevents Lucene from reclaiming deleted documents, filling NVMe disks.
Elasticsearch Search After Vs Deep Pagination — Normal → Load / Fault → Recovery
Replace offset-based from/size with stateless searchafter paired with a Point-in-Time (pit) ID and a tie-breaker sort key (sharddoc or id) to keep memory O(size) per page.
Putting Fixing Elasticsearch Deep Pagination with search_after Together
Replace offset-based `from/size` with stateless `search_after` paired with a Point-in-Time (`_pit`) ID and a tie-breaker sort key (`_shard_doc` or `id`) to keep memory O(size) per page.
Putting Fixing Elasticsearch Deep Pagination with search_after Together
- Open a PIT handle with
keep_alive=2mbefore the first page. - Query
/_searchpassingpit: {id, keep_alive}andtrack_total_hits: false. - Use
product_idas a deterministic tie-breaker so cursor recovery works even if a PIT expires. - Close the PIT with
DELETE /_pitin afinallyblock for batch exporters.
Interview Readiness Check
Open PITs pin old Lucene segments after merges, causing disk bloat and file-descriptor exhaustion if left unclosed or uncapped.
Walk through bottlenecks, back-of-the-envelope math, and failure recovery.
Make the trade-off yourself.
Practice Check
Test your understanding
When is ACID Transaction Fundamentals the right default for a production team?
Fixing Elasticsearch Deep Pagination with search_after
Replace offset-based from/size with stateless search_after paired with a Point-in-Time (_pit) ID and a tie-breaker sort key (_shard_doc or id) to keep memory O(size) per page.
