microsoft/FASTER
FASTER
FASTER is a high-throughput key-value store that combines an in-memory data store with a hybrid memory-disk storage engine, enabling datasets larger than available RAM. It uses a latch-free, cache-optimized index for concurrent point lookups and heavy updates, and records all mutations to a persistent append-only log on disk with checksum validation and group-commit checkpointing for crash recovery.
The system supports multi-key transactional workloads through atomic multi-key locking, ensuring transactional consistency without coarse-grained contention. It exposes the key-value store to remote clients over a custom TCP protocol that scales linearly with the number of concurrent connections, and provides atomic value merging for read-modify-write operations.
FASTER includes a non-blocking checkpoint-recovery model that restores consistent state after crashes, and provides high-performance iterators for reading through the persistent log sequentially. The append-only log engine supports frequent low-latency commits and saturates disk bandwidth, while the hybrid storage layer seamlessly spills cold data to fast local or cloud storage.
