Postgres Index-Only Scans: Covering Indexes, INCLUDE, and the Visibility Map
Postgres · Intermediate · 0 min read · published
Overview
An index-only scan lets Postgres answer a query entirely from an index, never touching the heap (table) pages. It only works when every column the query needs is present in the index and the visibility map marks the containing heap pages as all-visible. The INCLUDE clause (Postgres 11+) lets you add non-key payload columns to a B-tree index so it becomes "covering" without bloating the search key or breaking uniqueness semantics.
flowchart TD
A[Query arrives] --> B{All needed columns<br/>in the index?}
B -- No --> C[Index Scan:<br/>probe index, then fetch heap tuple]
B -- Yes --> D{Heap page marked<br/>all-visible in VM?}
D -- No --> E[Index Only Scan<br/>with Heap Fetches > 0]
D -- Yes --> F[Index Only Scan<br/>Heap Fetches: 0]
E --> G[Visibility check<br/>reads heap page]
C --> H[Rows returned]
G --> H
F --> H
I[VACUUM / autovacuum] -.->|sets all-visible bits| D
Pros
- Eliminates random heap I/O for the hottest read paths — often a 5–20x latency drop on wide tables.
INCLUDEcolumns are stored only in leaf pages, so the index stays shallow and inner pages stay dense compared to adding them as key columns.- Works with
UNIQUEindexes:CREATE UNIQUE INDEX ... (email) INCLUDE (tenant_id)keeps uniqueness onemailalone. - Great for count/aggregate queries and for pagination that only needs id + sort key.
Cons
- Every extra column widens the index, increasing write amplification and slowing HOT-update eligibility (updating an indexed key column forces a new index entry).
- Depends on the visibility map: a write-heavy table with lazy autovacuum shows high
Heap Fetchesand the benefit evaporates. INCLUDEcolumns cannot be used for filtering with index quals in the same way key columns can — they're payload only (though the planner can still apply them as a filter after fetching the tuple from the index).- Expression indexes generally can't produce index-only scans for the underlying column.
Real-World Example
A multi-tenant SaaS dashboard renders an order list: 40M-row orders table, query selects id, status, total_cents filtered by tenant and ordered by created_at. The existing index on (tenant_id, created_at) forces 500 random heap fetches per page load.
-- Before: Index Scan + 500 heap fetches
EXPLAIN (ANALYZE, BUFFERS)
SELECT id, status, total_cents
FROM orders
WHERE tenant_id = 42
ORDER BY created_at DESC
LIMIT 50;
-- Make the index covering: keys drive the search + sort, INCLUDE carries payload
CREATE INDEX CONCURRENTLY idx_orders_tenant_created_cov
ON orders (tenant_id, created_at DESC)
INCLUDE (status, total_cents);
-- Ensure the visibility map is current so Heap Fetches drops to 0
VACUUM (ANALYZE) orders;
-- After: "Index Only Scan using idx_orders_tenant_created_cov ... Heap Fetches: 0"
-- Keep the VM fresh on this hot table:
ALTER TABLE orders SET (autovacuum_vacuum_scale_factor = 0.02);
Real-world challenge
Your team added `INCLUDE (status, total_cents)` to a covering index on a 40M-row `orders` table. For two weeks the dashboard endpoint returned in 8ms. After a nightly batch job started running `UPDATE orders SET status = 'archived' WHERE created_at < now() - interval '1 year'` (touching ~2M rows), the same endpoint climbed to 300ms every morning and slowly recovered over the day. `EXPLAIN ANALYZE` still shows `Index Only Scan`. Diagnose and fix.
Diagnosis
- The plan shape is unchanged (
Index Only Scan), so this is not a planner regression — look at the scan's counters:
EXPLAIN (ANALYZE, BUFFERS)
SELECT id, status, total_cents FROM orders
WHERE tenant_id = 42 ORDER BY created_at DESC LIMIT 50;
-- Heap Fetches: 41,332 shared read=39k
- High
Heap Fetchesright after the batch job means the 2M updated rows cleared the all-visible bits across a huge swath of heap pages. Confirm with:
SELECT relname, n_dead_tup, last_autovacuum,
pg_relation_size(relid) / 8192 AS heap_pages,
(SELECT count(*) FROM pg_visibility_map('orders') v WHERE v.all_visible) AS av_pages
FROM pg_stat_user_tables WHERE relname = 'orders';
- The slow recovery over the day is autovacuum eventually catching up — with the default
autovacuum_vacuum_scale_factor = 0.2, 2M dead tuples on 40M rows doesn't even cross the threshold, so only later activity triggers it.
Root cause
The batch UPDATE also rewrites the index entry for every row because status is an INCLUDE column — non-key payload columns still participate in index maintenance, so a HOT update is impossible. That produces both index bloat and mass visibility-map invalidation.
Fix
- Vacuum explicitly at the end of the batch job so the visibility map is rebuilt before morning traffic:
-- inside the batch job, after the UPDATE commits
VACUUM (ANALYZE, PARALLEL 4) orders;
- Lower the autovacuum threshold on this hot table:
ALTER TABLE orders SET (autovacuum_vacuum_scale_factor = 0.02, autovacuum_vacuum_cost_delay = 0); - Batch the update in chunks (e.g. 50k rows with a short sleep) instead of one 2M-row transaction, so vacuum can reclaim between chunks.
- Longer term: reconsider putting a frequently-mutated column like
statusin the covering index, or move archived rows to a partition so the archival step becomes a partition detach instead of a mass UPDATE.