microsoft/garnet

Features

• Distributed Key-Value Stores - Scales data across a sharded cluster with replication and dynamic key migration to ensure high availability and throughput.
• In-Memory Caches - Provides a high-performance remote cache store with sub-millisecond latency under heavy concurrent load.
• Latch-Free Shared Memory Engines - Processes parallel requests across multiple cores using latch-free data structures and epoch protection for sub-millisecond latency.
• Wire Protocol Serializers - Uses a standardized binary serialization format to ensure interoperability with remote cache clients and libraries.
• Lua Scripting - Executes Lua scripts atomically on the server to combine multiple operations in one round trip.
• Cluster Key Migration - Rebalances cluster load by migrating keys between nodes in a distributed cluster without incurring downtime.
• Complex Data Structure Stores - Natively supports diverse data structures including sorted sets, lists, bitmaps, and geospatial indexes.
• Latch-Free Indexes - Implements latch-free indexes and cache-line alignment to enable high-throughput multi-threaded access with minimal locking.
• Checkpoints and Recovery - Provides durable persistence through fast checkpoints and restore operations to ensure data recovery after restarts.
• Data Sharding - Organizes data across multiple nodes using sharding and replication to ensure high availability and scalability.
• Database Protocol Compatibility - Implements the Redis Serialization Protocol to ensure seamless interoperability with existing Redis client libraries and administration tools.
• Database Wire Protocol Implementations - Implements the RESP wire protocol to allow standard Redis clients to connect and execute commands.
• Distributed Sharding Architectures - Distributes keys across multiple nodes using a consistent hashing mechanism to scale storage and throughput linearly.
• Horizontal Scaling - Distributes keys across slots and nodes to enable linear horizontal scaling and data migration.
• In-Memory Databases with Persistence - Combines sub-millisecond memory performance with append-only logging and checkpoints for durable data recovery.
• Multi-Threaded Shared-Memory Engines - Utilizes a multi-threaded shared-memory architecture to enable parallel request processing for sub-millisecond latency.
• Primary-Replica Replication - Propagates data from lead nodes to read-only followers to improve read availability and provide fault tolerance.
• Read-Only Data Replication - Copies data from lead nodes to followers to improve read availability and ensure fault tolerance.
• Redis Compatible Stores - Implements the RESP wire protocol to ensure compatibility with the existing Redis ecosystem of clients and libraries.
• Vector Search Indexes - Utilizes specialized indexing for high-dimensional vectors to enable fast semantic search at scale.
• Server-Side Scripting - Executes logic-bearing scripts directly on the server to perform complex atomic operations and reduce network round trips.
• Slot-Based Routing - Routes operations to the correct node based on hash slot ownership to manage distributed request flow.
• Key-Value - Implements high-performance get, set, and expiration commands for string key-value pairs via the RESP protocol.
• Storage Tiering - Implements automated movement of cold data from system memory to SSD or cloud storage layers.
• Stored Procedures - Executes Lua scripts and C# stored procedures atomically on the server to perform complex multi-key operations.
• Transactional Key-Value Modifications - Processes multi-key operations atomically via client requests or server-side procedures to ensure data consistency.
• Vector Search - Provides a specialized database for storing and querying high-dimensional vectors using similarity search.
• Cluster Configuration Management - Manages cluster behavior, node membership, and slot assignments via direct commands and gossip protocols.
• Gossip Protocols - Uses peer-to-peer gossip communication to propagate node membership and slot assignment updates across the distributed network.
• Hash Slot Sharding - Distributes keys across cluster nodes using a consistent hashing scheme that maps each key to a specific slot.
• Multi-Threaded Request Handling - Distributes network IO and query processing across multiple CPU cores to maintain high throughput and low latency.
• Sharded Cluster Deployment - Distributes data across a sharded cluster with replication and dynamic key migration for high availability.
• Append-Only Persistence - Ensures data durability through sequential write-ahead logging and non-blocking checkpoints.
• Hash Maps - Stores collections of field-value pairs and supports atomic increments and range retrievals.
• Hash-Tag Key Mapping - Uses hashtags to map multiple keys to the same shard, enabling atomic multi-key operations.
• Command Batching - Groups multiple commands into a single request block to minimize network round trips and maximize throughput.
• Key Lifecycle Management - Manages key expiration timers, time-to-live calculations, and key renaming or deletion.
• Tiered Memory Management - Spans data across RAM and persistent flash storage to handle datasets larger than available system memory.
• Geospatial Search - Supports storing coordinate data and calculating distances or searching within geographic radii.
• Secondary Indexes - Builds secondary indexes over incoming data streams using subset hashing for fast lookups.
• Set Data Structures - Supports collections of unique elements with mathematical set operations like intersections and unions.
• Sorted Sets - Stores unique elements associated with numerical scores to maintain ordered lists for ranking.
• SSD Storage Extensions - Extends available memory by offloading colder datasets to local SSD or cloud storage when RAM is exceeded.
• Vector Indexing - Provides specialized approximate nearest neighbor indexing for high-dimensional vectors to enable fast semantic similarity search.
• Server Configuration Management - Provides interfaces to adjust runtime settings, monitor performance, and trigger manual data persistence.
• Pub-Sub Messaging - Facilitates asynchronous communication between clients using a publish and subscribe pattern.
• Linked Lists - Implements linear collections allowing elements to be pushed, popped, or retrieved by index.
• Bit Manipulation Techniques - Performs bit-level operations such as setting, clearing, and counting bits within string values.
• Atomic Transaction Grouping - Groups multiple discrete operations into a single atomic unit to ensure consistent updates and reduce network overhead.
• Atomic Integer Operations - Performs thread-safe increment and decrement operations on integer values directly on the server.

Star history