FOSSWire
Search/
Big Tech FOSS

Meta releases RocksDB 11.13.2 with data loss fixes and manifest tracking

Brief written by the FOSSWire AI editor from Meta RocksDB Key-Value Store How we write

1 min read

Key Takeaway

RocksDB 11.13.2 introduces experimental manifest-based options tracking and resolves critical data loss bugs related to range tombstone conversion.

Summary

Meta has released RocksDB version 11.13.2, an update to its high-performance embeddable key-value store. This release focuses heavily on data integrity, addressing several bugs in previous versions that could lead to accidental data loss during external file ingestion or specific iterator operations.

Beyond stability fixes, the update introduces an experimental feature for tracking options file numbers within the MANIFEST, improving how database configurations are persisted and recovered after crashes. The release also includes performance optimizations for performance context counters and fixes for asynchronous I/O handling in MultiGet operations.

Technical improvements in RocksDB 11.13.x

  • Data Integrity: Fixed critical data loss scenarios where min_tombstones_for_range_conversion could incorrectly insert range tombstones, deleting live keys during external file ingestion or when using specific ReadOptions::read_tier settings.
  • Configuration Management: Introduced an experimental DBOptions::track_options_file_number_in_manifest rollout option for C and Java bindings, allowing the database to durably commit OPTIONS file numbers to the MANIFEST for safer crash recovery.
  • I/O and Concurrency: Resolved a bug where MultiGet with async_io returned NotSupported during io_uring failures by implementing a fallback to synchronous reads, and fixed a corruption issue in blob direct writes during concurrent column family flushes.
  • Resource Accounting: Addressed multiple memory and size leaks in SstFileManager trash accounting that occurred when file deletions failed or directories were renamed, which previously required a restart to clear.
  • Performance: Optimized performance context counters and timers, reducing the cost of counter updates by approximately 60% when kEnableCount is active.

Why it matters

As a foundational component for many large-scale distributed systems and databases, these stability and data integrity fixes are vital for production environments. The new manifest tracking and offline repair tools via ldb provide system administrators and database engineers with more robust recovery options during catastrophic failures.

What to do

  • Update to RocksDB 11.13.2 if you use min_tombstones_for_range_conversion with external file ingestion or ReadOptions::read_tier settings, as earlier versions have critical data loss bugs.
  • Enable the experimental DBOptions::track_options_file_number_in_manifest option if you want safer crash recovery through MANIFEST-tracked OPTIONS file numbers in C or Java.
  • Watch for hard background errors if you have max_bgerror_resume_count <= 0, as compaction or flush IO errors now stop writes until DB::Resume() is called.
  • Check your SstFileManager trash accounting if you use rate limiting, as previous versions leaked trash sizes until restart when deletions failed or directories were renamed.

Original reference: github.com →