facebook/rocksdb

Features

• Key-Value - Implements a high-performance storage engine that persists data as key-value pairs optimized for flash and RAM.
• Log-Structured Merge-Trees - Uses a Log-Structured Merge-Tree architecture to buffer writes in memory and flush them to sorted immutable files.
• Column Family Management - Organizes data into multiple logical partitions with independent settings and compaction strategies.
• Compaction Strategies - Provides configurable compaction strategies to balance write amplification and read efficiency in the LSM-tree.
• Data Partitioning - Divides the keyspace into independent namespaces with separate memtables and compaction settings for logical data organization.
• Atomic Transactions - Ensures consistency across multiple operations using optimistic and pessimistic atomic transaction patterns.
• Sorted String Tables - Persists data in immutable sorted files containing indices and bloom filters for fast point lookups and range scans.
• High-Performance Databases - Engineered for extreme write throughput and low-latency retrieval using multi-threaded compaction.
• Key-Value Persistence Stores - Provides a durable data store for large-scale datasets with atomic transactions and column families.
• Key-Value Stores - Serves as a persistent, embeddable key-value store integrated directly into applications.
• Large-Scale Dataset Management - Manages multi-terabyte datasets efficiently using multi-threaded compactions.
• Tiered Compaction Strategies - Merges smaller sorted files into larger ones across multiple levels to reclaim space and minimize write amplification.
• Write-Ahead Logging - Records every mutation to a sequential on-disk log to ensure data durability and recovery after crashes.
• Sorted Memory Buffers - Buffers incoming writes in a sorted memory structure (MemTable) before flushing them as immutable sorted string tables.
• Point-In-Time Snapshots - Generates consistent point-in-time snapshots of the database for backup or analysis purposes.
• Custom Grouped Operations - Defines custom merge operators to aggregate multiple values for a single key during reads or compaction.
• Flash Storage Managers - Manages data placement and write patterns to maximize SSD and zoned storage efficiency.
• Data Caching - Improves read throughput by storing frequently accessed data on local disks or SSDs.
• Data Compression Algorithms - Reduces the storage footprint of keys and values using compression algorithms to balance CPU and disk usage.
• Caching and Performance - Reduces latency and improves read throughput by caching frequently accessed data on local hardware.
• Compaction Filters - Applies custom logic to drop or transform keys and values while the database reorganizes internal files.
• Expiration Policies - Automatically removes old data based on a configurable time-to-live (TTL) threshold.
• Distributed System Metadata - Provides a durable and efficient storage layer for tracking metadata in large-scale distributed systems.
• In-Memory Caches - Implements a memory cache to store frequently accessed data and index blocks, reducing disk I/O latency.
• Key-Value Store Iterators - Provides forward and backward iterators for traversing sorted key-value pairs with snapshot isolation.
• Range Deletions - Removes all keys within a specified range using tombstone markers that are physically purged during compaction.
• Local State Stores - Enables fast access for stream processing by storing application state on local SSDs.
• Secondary Indexes - Provides secondary indexing capabilities to enable efficient lookups on columns other than the primary key.
• Storage Amplification Tuning - Allows tuning of write, read, and space amplification to optimize performance for specific hardware and workloads.
• Storage Integrity Validation - Verifies checksums for files and memtables to detect and report storage corruption.
• System Metadata Tracking - Tracks filesystem and object metadata for large-scale distributed systems across billions of entries.
• Plugin-Extensible Runtimes - Allows injecting custom logic via external plugins to modify internal storage engine operations.
• Custom Memory Allocators - Utilizes custom memory allocators to optimize heap usage and minimize resource contention during high-throughput operations.
• Zoned Block Device Support - Optimizes data placement for zoned storage devices to improve write efficiency and device lifespan.
• Language Bindings - Provides third-party bindings to connect the storage engine to various programming languages across different application stacks.
• Internal Telemetry Monitors - Monitors internal operational state including read/write latency, cache hit rates, and compaction times.
• Database Engines - Embeddable persistent key-value store for fast storage.
• Database Systems - Embedded key-value store optimized for fast storage.
• Databases - High-performance embedded key-value store.
• Databases and Data - High-performance embedded key-value store.

Star history