PostgreSQL's recovery_prefetch explained: how WAL replay read-ahead works

PostgreSQL consultant Christophe Pettus published a detailed technical breakdown of recovery_prefetch, a PostgreSQL 15 feature that lets the WAL replay startup process ask the operating system to read data pages before they're needed. On simulated cloud storage with 1ms read latency, recovery_prefetch reduced crash recovery time from 378 seconds to 82 seconds at default settings, and to 30 seconds with increased concurrency. The parameter defaults to try on Linux and macOS, uses posix_fadvise(POSIX_FADV_WILLNEED), and can be changed on a running standby with SIGHUP reload.

How recovery_prefetch works and its limits

• Mechanism: The prefetcher wraps the WAL reader, decoding records ahead of replay and calling posix_fadvise(WILLNEED) for main-fork blocks not in shared_buffers; Linux supports this via POSIX_FADV_WILLNEED, macOS added fcntl(F_RDADVISE) in PostgreSQL 18, Windows silently does nothing with try
• Companion parameters: wal_decode_buffer_size (default 512kB) controls how far ahead in WAL to decode; maintenance_io_concurrency (16 on PG 18+, 10 on 15-17) caps outstanding prefetches
• Skip categories: Blocks already in shared_buffers (hit), full-page images (skip_fpw), zero-fill blocks (skip_init), not-yet-created relations (skip_new), and recent duplicate lookups (skip_rep) are not prefetched
• Hidden costs: The prefetcher cannot see the kernel page cache, so on RAM-fitting workloads it wastes system calls and in-flight slots on no-ops; with full_page_writes on, first touches after checkpoints carry whole pages and get skipped
• Direct I/O incompatibility: With debug_io_direct = data, prefetching silently does nothing; PostgreSQL 18's async I/O subsystem does not change replay's single-block-at-a-time behavior
• History: Originated from Sean Chittenden's 2018 pg_prefaulter external tool; committed for PostgreSQL 14 in 2021, reverted for bugs, then committed again for 15 in 2022 with default try

Why it matters

For PostgreSQL users running standbys on network-attached cloud storage, recovery_prefetch offers substantial recovery time improvements without application changes, though its benefits diminish on systems where the working set fits in RAM or where full_page_writes dominates the WAL stream.