RocksDB has historically been known for poor performance when tombstones accumulate. This has become a common problem within Meta, and the community has raised it as well. Here, we introduce an optimization that attempts to convert contiguous tombstones into a range tombstone during scans. As a resu...

RocksDB 11.0 added CompactionOptionsFIFO::max_data_files_size and CompactionOptionsFIFO::use_kv_ratio_compaction for a specific but important shape of workload: FIFO compaction, integrated BlobDB, large values, point lookups, and data that naturally expires by TTL or by a bounded data-size budget. T...

TL;DR Blob Direct Write moves large-value separation earlier in RocksDB’s write path. When enable_blob_files and enable_blob_direct_write are enabled, values at or above min_blob_size can be written directly to blob files during a write, while the WAL and memtable store a compact BlobIndex reference...

Background RocksDB can offload compaction work to remote workers through the CompactionService API. In this model, the primary RocksDB instance selects the input files and sends a serialized CompactionServiceInput to a worker; the remote worker runs DB::OpenAndCompact(), writes output SSTs to output...

For workloads with uniformly distributed keys, RocksDB now supports interpolation search for SST index blocks as an alternative to the default binary search. The idea Binary search always splits the remaining range in half: 1 mid = low + (high - low) / 2 That’s Θ(log n) probes regardless of the data...

This is the story of how a RocksDB unit test I added four years ago, a mini-stress test you might call it, revealed a novel hardware bug in a newer CPU. It was scary enough to be assigned a “high severity” CVE. Background: Unique Identifiers About four years ago, we added unique identifiers to SST f...

Modern concurrent data structures increasingly rely on atomic operations to avoid the overhead of locking. A valuable but under-utilized technique for maximizing the effectiveness of atomic operations is bit packing—fitting multiple logical fields into a single atomic variable for algorithmic simpli...

The upcoming RocksDB 10.7 release includes a major revamp of parallel compression that dramatically reduces the feature’s CPU overhead by up to 65% while maintaining or improving throughput for compression-heavy workloads. We expect this to broaden the set of workloads that could benefit from parall...

Context RocksDB performs a variety of IO operations—user reads, background compactions, flushes, database opens, and verification tasks. Treating all these operations the same makes it difficult for file system implementers to optimize performance, prioritize latency-sensitive IOs, and diagnose bott...

Context / Problem Modern RocksDB deployments often run in environments with strict memory constraints—cloud VMs, containers, or hosts with hundreds of DB instances. Unpredictable memory usage can lead to out-of-memory (OOM) errors, degraded performance, or even service outages. Historically, while t...

Dear RocksDB Community, We want to share an update about the bug that allowed our bug bounty researcher to update the release note title in August 2024 involving the RocksDB open-source repository on GitHub. This issue was found and responsibly disclosed to us by an external bug bounty researcher th...

Java Foreign Function Interface (FFI) Evolved Binary has been working on several aspects of how the Java API to RocksDB can be improved. The recently introduced FFI features in Java provide significant opportunities for improving the API. We have investigated this through a prototype implementation....

RocksDB Java API Performance Improvements Evolved Binary has been working on several aspects of how the Java API to RocksDB can be improved. Two aspects of this which are of particular importance are performance and the developer experience. We have built some synthetic benchmark code to determine w...

TL:DR Tiered storage is now natively supported in the RocksDB with the option last_level_temperature, time-aware Tiered storage feature guarantees the recently written data are put in the hot tier storage with the option preclude_last_level_data_seconds. Background RocksDB Tiered Storage assigns a d...

TL;DR By cutting the compaction output file earlier and allowing larger than targeted_file_size to align the compaction output files to the next level files, it can reduce WA (Write Amplification) by more than 10%. The feature is enabled by default after the user upgrades RocksDB to version 7.8.0+. ...