Snapchat/KeyDB
KeyDB
KeyDB is a multithreaded in-memory key-value store and distributed cache. It functions as a NoSQL database utilizing multi-version concurrency control to execute non-blocking queries and scans.
The project is a multithreaded fork of Redis that maintains protocol compatibility while utilizing a multithreaded architecture to scale across multi-core hardware. It distinguishes itself with flash-tiered storage, allowing the system to offload data from primary RAM to SSD or flash storage to increase total capacity.
The system supports high availability through active-active mesh replication and multi-master synchronization. Additional capabilities include cloud-native snapshotting for backups, Transport Layer Security for encrypted communication, and subkey-level expiration for nested data structures.
The server can be operated as a background daemon with configuration managed via startup arguments or files, and it includes a dedicated command-line interface for data management.
Features
- In-Memory Data Stores - Functions as a high-performance in-memory key-value store for low-latency data access.
- Multi-Threaded Request Handling - Distributes network IO and query parsing across multiple CPU cores to handle concurrent requests without blocking.
- Data Replication - Synchronizes data across distributed nodes to ensure high availability and enable load balancing.
- Active-Active Database Clusters - Implements a peer-to-peer topology allowing concurrent read and write operations across multiple database nodes.
- Database Replication - Synchronizes data across multiple instances in a mesh topology to enable load balancing and high availability.
- Distributed Caching - Implements a distributed caching system with active-active mesh replication for high availability.
- Flash Extensions - Offloads infrequently accessed data from primary RAM to flash storage to increase total capacity.
- Tiered Memory Management - Expands database storage beyond available RAM by offloading data to local SSD or flash storage.
- Multi-Master Replication - Implements a synchronization system where multiple instances simultaneously accept reads and writes.
- Multi-Version Concurrency Control - Utilizes multi-version concurrency control to allow non-blocking read operations during concurrent updates.
- Multithreaded Key-Value Stores - Extends the Redis protocol with a multithreaded architecture to utilize multi-core hardware more efficiently.
- NoSQL Databases - Implements a NoSQL database utilizing multi-version concurrency control for non-blocking operations.
- Redis Compatible Stores - Implements a caching layer that is protocol-compatible with Redis clients while leveraging multi-core hardware.
- SSD Storage Extensions - Moves data to local SSD volumes to increase storable information beyond available RAM.
- Key-Value - Provides a high-performance multithreaded in-memory store for key-value pairs.
- Thread Scaling - Distributes network IO and query parsing across multiple CPU cores to handle concurrent requests.
- Concurrency Control - Uses multi-version concurrency control to execute non-blocking queries and scans during active traffic.
- Subkey Expirations - Controls the lifetime of individual keys nested within larger data structures.
- Cloud Storage Backups - Provides a mechanism to dump and load database snapshots directly to and from remote cloud storage buckets.
- Database Snapshots - Provides native capabilities to dump and load database state directly to and from remote cloud storage buckets.
- Nested Element Expirations - Tracks and enforces individual time-to-live values for nested elements within larger composite data structures.
- Caching and Messaging - Multi-threaded fork of the popular in-memory database.
