Skip to main content
Concepts/Distributed Systems & Storage/sql-query-tuning-indexes

SQL Query Tuning: Index Seeks, Scans, Clustered & Covering Indexes

Eliminate slow table scans and high-cost Key Lookups using B-Tree index seeks, clustered primary keys, and composite covering indexes.

XLinkedInReddit
advanced

๐Ÿ’ก Why It Matters

Unindexed or poorly indexed queries degrade from 2ms to 10,000+ms under concurrent traffic, triggering database connection pool exhaustion and 100% CPU spikes. Analyzing execution plans (EXPLAIN ANALYZE) and replacing expensive Key Lookups with covering B-Tree index seeks is the highest-leverage performance optimization in relational databases.

๐ŸŒ Real-World Production Usages & Examples

E-Commerce User Order History Tuning

Scenario: Querying orders by user_id and created_at with status filter across 40M orders. Existing index was only on created_at, causing a 6-second table scan.

Outcome: Added composite covering index on (user_id, created_at) INCLUDE (status, total_amount), shifting execution to an Index Only Seek and dropping latency to 1.4ms.

FinTech Payment Ledger Bookmark Lookup Elimination

Scenario: A secondary index on merchant_id matched 5,000 rows, but required 5,000 Key Lookups to fetch transaction_id, currency, and amount, burning 85% CPU.

Outcome: Added INCLUDE (currency, amount) to the secondary index. Eliminated 5,000 random page reads and reduced p99 query latency from 420ms to 2.1ms.

โœ“ When to Use

  • โ€ขTables with >100,000 rows where point lookups or range queries suffer from full table scans (Seq Scan).
  • โ€ขQueries where execution plans reveal high-cost 'Key Lookup', 'RID Lookup', or 'Bitmap Heap Scan' operations consuming >70% of query time.
  • โ€ขHigh-frequency read APIs where queries can be fulfilled completely by index leaf pages (Covering Index).
  • โ€ขDesigning primary keys where range scans or time-series sorting align with the physical clustered index order.

โœ• When NOT to Use

  • โ€ขSmall lookup tables (<1,000 rows) where sequential memory page reads are faster than traversing B-Tree branch pages.
  • โ€ขWrite-heavy tables with high ingestion throughput where maintaining multiple secondary indexes creates severe write amplification and page splits.
  • โ€ขQueries returning >20-25% of table rows, where the optimizer's tipping point mathematically favors sequential scans over random single-page lookups.

๐Ÿ”‘ Core Architecture & Rules

  • 1Clustered Index: Determines the physical sort order of table rows on disk. Leaf nodes are the actual table data pages (e.g. InnoDB clustered primary key). A table can have only one clustered index.
  • 2Secondary (Non-Clustered) Index: A separate B-Tree where leaf nodes contain indexed keys plus a row pointer (Clustered Key in MySQL/SQL Server, or physical Tuple ID/RID in PostgreSQL heap tables).
  • 3Index Seek vs Index Scan: An Index Seek navigates root-to-leaf via binary search to find exact boundaries in O(log N); an Index Scan reads all leaf pages in sequence in O(N).
  • 4Key / RID Lookup (Bookmark Lookup): Occurs when a secondary index seek matches rows, but the query requests additional columns not present in the index, forcing extra random I/O lookups to the clustered index or heap.
  • 5Covering Index: Contains all columns needed for a query (in the key or via INCLUDE clause), allowing the database engine to return results entirely from the index tree (Index Only Scan) with zero heap accesses.
  • 6Leftmost Prefix Rule: Composite indexes (col1, col2, col3) can only be sought if WHERE clauses filter on leading columns in left-to-right order without skipping.

โš–๏ธ Architectural Trade-Off Analysis

Read Speed vs Write Amplification

PROCovering indexes eliminate heap lookups and deliver sub-millisecond reads.
CONEvery INSERT, UPDATE, or DELETE must update both the table and all secondary B-Tree indexes, increasing write I/O and transaction log volume.

Clustered Key: Sequential ID vs UUIDv4

PROSequential keys (BIGINT AUTO_INCREMENT / IDENTITY) append data to the rightmost leaf page with zero page splits.
CONRandom UUIDv4 primary keys cause random disk writes, high B-Tree page splits (50% fill factor), and massive index bloat.

Index Selectivity & The Tipping Point

PROIndex seeks excel when row selectivity is high (<5% of table rows).
CONWhen a query matches >15-20% of rows, random single-page lookups cost more than sequential multi-page reads, forcing the optimizer to revert to a full scan.

โš ๏ธ Failure Modes & Mitigations

Non-SARGable WHERE Expressions

Wrapping indexed columns in functions (e.g. WHERE DATE(created_at) = '2026-10-13' or WHERE LOWER(email) = ...) prevents the B-Tree from performing an Index Seek, forcing a full index scan.

Mitigation: Rewrite filters to isolate columns: WHERE created_at >= '2026-10-13 00:00:00' AND created_at < '2026-10-14 00:00:00', or define expression/functional indexes.

Implicit Type Conversion

Comparing a VARCHAR column with an integer literal (WHERE account_code = 10492) forces the database to cast every row's string to integer, disabling index seeks.

Mitigation: Ensure ORM mappings and client queries pass matching parameter data types (WHERE account_code = '10492').

Over-Indexing & B-Tree Page Splits

Creating 15+ secondary indexes on a high-concurrency table causes leaf page fragmentation, locks during page splits, and slow write transactions.

Mitigation: Audit unused indexes using pg_stat_user_indexes or sys.dm_db_index_usage_stats and consolidate overlapping composite indexes.

๐Ÿ› ๏ธ Example Usages

Reviewed practical implementations and concrete code walkthroughs for SQL Query Tuning: Index Seeks, Scans, Clustered & Covering Indexes.

No reviewed practical examples currently published for this concept. Queued for technical review.

โฌกExplore Related Concepts & Guides

Curated technical reading and direct concept relationships.

โšก Next Step ยท Apply Concept

Ready to apply SQL Query Tuning: Index Seeks, Scans, Clustered & Covering Indexes?

Test your understanding with a short scenario about real system constraints and trade-offs.

Share this concept

Share with fellow engineers or bookmark for system design discussions.

Share:XLinkedInReddit