PostgreSQL memtable extension implements cache invalidation for memory management

A technical deep dive by Mats Kindahl details the implementation of cache invalidation within the memtable extension for PostgreSQL. The extension utilizes a custom table access method that maps table arenas to backend memory, which can lead to significant memory retention if not properly managed.

Without invalidation, a backend that has read a table will continue to hold a mapping to that table's arena even after another session has dropped or truncated it. This implementation leverages PostgreSQL's internal relcache invalidation mechanism to ensure that stale mappings are detached, allowing the system to reclaim memory from dropped relations without waiting for backend disconnections.

Technical implementation of relcache invalidation

• Memory Leak Prevention: The extension prevents DSM (Dynamic Shared Memory) segments from being pinned indefinitely by ensuring that mappings are detached when a relation changes.
• Relcache Callback Integration: The implementation uses CacheRegisterRelcacheCallback to register a callback function that triggers whenever pg_class changes via DROP or TRUNCATE operations.
• Cache Inspection Tools: A new set-returning function memtable_cache_entries and a corresponding view memtable_cache allow administrators to inspect current backend cache entries, including relation IDs and mapping status.
• Invalidation Logic: The memtable_relcache_callback handles both specific relation invalidations and global invalidations (where relid is InvalidOid) by iterating through the local state cache and removing entries via memtable_forget_entry.

Why it matters

For database engineers and system administrators, this ensures that custom PostgreSQL access methods can manage large memory arenas safely. It prevents 'ghost' memory usage in connection pools where long-lived sessions might otherwise prevent the reclamation of memory from deleted tables.