Postgres Index-Only Scans: Covering Indexes, INCLUDE, and the Visibility Map

Postgres · Intermediate · 0 min read · published

Overview

An index-only scan lets Postgres answer a query entirely from an index, never touching the heap (table) pages. It only works when every column the query needs is present in the index and the visibility map marks the containing heap pages as all-visible. The INCLUDE clause (Postgres 11+) lets you add non-key payload columns to a B-tree index so it becomes "covering" without bloating the search key or breaking uniqueness semantics.

flowchart TD
    A[Query arrives] --> B{All needed columns<br/>in the index?}
    B -- No --> C[Index Scan:<br/>probe index, then fetch heap tuple]
    B -- Yes --> D{Heap page marked<br/>all-visible in VM?}
    D -- No --> E[Index Only Scan<br/>with Heap Fetches > 0]
    D -- Yes --> F[Index Only Scan<br/>Heap Fetches: 0]
    E --> G[Visibility check<br/>reads heap page]
    C --> H[Rows returned]
    G --> H
    F --> H
    I[VACUUM / autovacuum] -.->|sets all-visible bits| D

Pros

Cons

Real-World Example

A multi-tenant SaaS dashboard renders an order list: 40M-row orders table, query selects id, status, total_cents filtered by tenant and ordered by created_at. The existing index on (tenant_id, created_at) forces 500 random heap fetches per page load.

-- Before: Index Scan + 500 heap fetches
EXPLAIN (ANALYZE, BUFFERS)
SELECT id, status, total_cents
FROM orders
WHERE tenant_id = 42
ORDER BY created_at DESC
LIMIT 50;

-- Make the index covering: keys drive the search + sort, INCLUDE carries payload
CREATE INDEX CONCURRENTLY idx_orders_tenant_created_cov
  ON orders (tenant_id, created_at DESC)
  INCLUDE (status, total_cents);

-- Ensure the visibility map is current so Heap Fetches drops to 0
VACUUM (ANALYZE) orders;

-- After: "Index Only Scan using idx_orders_tenant_created_cov ... Heap Fetches: 0"
-- Keep the VM fresh on this hot table:
ALTER TABLE orders SET (autovacuum_vacuum_scale_factor = 0.02);

Real-world challenge

Your team added `INCLUDE (status, total_cents)` to a covering index on a 40M-row `orders` table. For two weeks the dashboard endpoint returned in 8ms. After a nightly batch job started running `UPDATE orders SET status = 'archived' WHERE created_at < now() - interval '1 year'` (touching ~2M rows), the same endpoint climbed to 300ms every morning and slowly recovered over the day. `EXPLAIN ANALYZE` still shows `Index Only Scan`. Diagnose and fix.

Diagnosis

  1. The plan shape is unchanged (Index Only Scan), so this is not a planner regression — look at the scan's counters:
EXPLAIN (ANALYZE, BUFFERS)
SELECT id, status, total_cents FROM orders
WHERE tenant_id = 42 ORDER BY created_at DESC LIMIT 50;
-- Heap Fetches: 41,332   shared read=39k
  1. High Heap Fetches right after the batch job means the 2M updated rows cleared the all-visible bits across a huge swath of heap pages. Confirm with:
SELECT relname, n_dead_tup, last_autovacuum,
       pg_relation_size(relid) / 8192 AS heap_pages,
       (SELECT count(*) FROM pg_visibility_map('orders') v WHERE v.all_visible) AS av_pages
FROM pg_stat_user_tables WHERE relname = 'orders';
  1. The slow recovery over the day is autovacuum eventually catching up — with the default autovacuum_vacuum_scale_factor = 0.2, 2M dead tuples on 40M rows doesn't even cross the threshold, so only later activity triggers it.

Root cause

The batch UPDATE also rewrites the index entry for every row because status is an INCLUDE column — non-key payload columns still participate in index maintenance, so a HOT update is impossible. That produces both index bloat and mass visibility-map invalidation.

Fix

-- inside the batch job, after the UPDATE commits
VACUUM (ANALYZE, PARALLEL 4) orders;