Pebble is an embedded key-value storage engine written in Go, designed as a library that provides durable, write-optimized data persistence directly within applications. It organizes data using a log-structured merge-tree (LSM-tree) structure, where writes are first buffered in an in-memory skiplist memtable and persisted to a write-ahead log before being flushed to block-based SSTable files on disk. The engine supports atomic batch commits, configurable write synchronization, and automatic background compaction that merges and rewrites sorted runs to reclaim space and maintain read performance.
The storage engine distinguishes itself through support for range keys, which allow key-value pairs to apply to contiguous ranges of keyspace and be interleaved with point keys during iteration. It also includes property-based filtering, where user-defined key-value properties attached to SSTable blocks and files enable iteration to skip irrelevant data blocks and entire files during scans. Pebble provides a storage format migrator that can upgrade on-disk database files to newer physical formats at runtime, using either background or blocking compaction to apply the change.
The engine offers comprehensive key-value operations including point lookups, range scans in both forward and reverse directions, and batch write operations that group multiple mutations into a single atomic commit. It includes built-in benchmarking tools for measuring read, write, and mixed workload throughput and latency under realistic access patterns, using configurable key distributions and value sizes.