# microsoft/garnet

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/microsoft-garnet).**

11,885 stars · 668 forks · C# · MIT

## Links

- GitHub: https://github.com/microsoft/garnet
- Homepage: https://microsoft.github.io/garnet/
- awesome-repositories: https://awesome-repositories.com/repository/microsoft-garnet.md

## Topics

`cache` `cache-storage` `cluster` `concurrent` `hash-table` `key-value` `larger-than-memory` `low-latency` `persistent` `remote` `scalable` `store`

## Description

Garnet is a multi-threaded in-memory database and distributed key-value store. It functions as a high-performance remote cache store that implements the RESP wire protocol to maintain compatibility with existing Redis clients and libraries.

The project is distinguished by a shared-memory architecture that enables parallel request processing across multiple cores for sub-millisecond latency. It features a tiered storage system that automatically offloads colder data from system memory to SSD or cloud storage layers, and includes a specialized vector search database for high-dimensional similarity search.

The system covers a broad range of capabilities, including distributed clustering with horizontal slot sharding, primary-replica data replication, and durable persistence via append-only logging and non-blocking checkpointing. It supports various complex data types, server-side Lua scripting, and atomic multi-key transactions.

Administrative and security features include role-based access control, TLS connection encryption, and integration with external control planes for cluster state management and failover.

## Tags

### Data & Databases

- [Distributed Key-Value Stores](https://awesome-repositories.com/f/data-databases/data-engineering-infrastructure/data-persistence-storage/specialized-storage-engines/distributed-key-value-stores.md) — Scales data across a sharded cluster with replication and dynamic key migration to ensure high availability and throughput.
- [In-Memory Caches](https://awesome-repositories.com/f/data-databases/in-memory-caches.md) — Provides a high-performance remote cache store with sub-millisecond latency under heavy concurrent load. ([source](https://microsoft.github.io/garnet/docs/azure/overview))
- [Wire Protocol Serializers](https://awesome-repositories.com/f/data-databases/binary-serialization-formats/wire-protocol-serializers.md) — Uses a standardized binary serialization format to ensure interoperability with remote cache clients and libraries.
- [Lua Scripting](https://awesome-repositories.com/f/data-databases/caching-and-locking/atomic-task-locks/lua-scripting.md) — Executes Lua scripts atomically on the server to combine multiple operations in one round trip. ([source](https://microsoft.github.io/garnet/docs/commands/api-compatibility))
- [Cluster Key Migration](https://awesome-repositories.com/f/data-databases/cluster-key-migration.md) — Rebalances cluster load by migrating keys between nodes in a distributed cluster without incurring downtime. ([source](https://cdn.jsdelivr.net/gh/microsoft/garnet@main/README.md))
- [Complex Data Structure Stores](https://awesome-repositories.com/f/data-databases/complex-data-structure-stores.md) — Natively supports diverse data structures including sorted sets, lists, bitmaps, and geospatial indexes. ([source](https://microsoft.github.io/garnet/docs))
- [Latch-Free Indexes](https://awesome-repositories.com/f/data-databases/concurrent-write-optimizations/concurrent-caches/latch-free-indexes.md) — Implements latch-free indexes and cache-line alignment to enable high-throughput multi-threaded access with minimal locking. ([source](https://microsoft.github.io/garnet/docs/research/papers))
- [Checkpoints and Recovery](https://awesome-repositories.com/f/data-databases/data-engineering-infrastructure/data-persistence-storage/checkpoints-and-recovery.md) — Provides durable persistence through fast checkpoints and restore operations to ensure data recovery after restarts. ([source](https://microsoft.github.io/garnet/docs))
- [Data Sharding](https://awesome-repositories.com/f/data-databases/data-sharding.md) — Organizes data across multiple nodes using sharding and replication to ensure high availability and scalability. ([source](https://microsoft.github.io/garnet))
- [Database Protocol Compatibility](https://awesome-repositories.com/f/data-databases/database-protocol-compatibility.md) — Implements the Redis Serialization Protocol to ensure seamless interoperability with existing Redis client libraries and administration tools. ([source](https://microsoft.github.io/garnet/docs/azure/overview))
- [Database Wire Protocol Implementations](https://awesome-repositories.com/f/data-databases/database-wire-protocol-implementations.md) — Implements the RESP wire protocol to allow standard Redis clients to connect and execute commands.
- [Distributed Sharding Architectures](https://awesome-repositories.com/f/data-databases/distributed-sharding-architectures.md) — Distributes keys across multiple nodes using a consistent hashing mechanism to scale storage and throughput linearly.
- [Horizontal Scaling](https://awesome-repositories.com/f/data-databases/horizontal-scaling.md) — Distributes keys across slots and nodes to enable linear horizontal scaling and data migration. ([source](https://microsoft.github.io/garnet/docs/cluster/overview))
- [In-Memory Databases with Persistence](https://awesome-repositories.com/f/data-databases/in-memory-databases-with-persistence.md) — Combines sub-millisecond memory performance with append-only logging and checkpoints for durable data recovery.
- [Multi-Threaded Shared-Memory Engines](https://awesome-repositories.com/f/data-databases/in-memory-databases/multi-threaded-shared-memory-engines.md) — Utilizes a multi-threaded shared-memory architecture to enable parallel request processing for sub-millisecond latency.
- [Primary-Replica Replication](https://awesome-repositories.com/f/data-databases/primary-replica-replication.md) — Propagates data from lead nodes to read-only followers to improve read availability and provide fault tolerance.
- [Read-Only Data Replication](https://awesome-repositories.com/f/data-databases/read-only-data-replication.md) — Copies data from lead nodes to followers to improve read availability and ensure fault tolerance. ([source](https://microsoft.github.io/garnet/docs/cluster/overview))
- [Redis Compatible Stores](https://awesome-repositories.com/f/data-databases/redis-compatible-stores.md) — Implements the RESP wire protocol to ensure compatibility with the existing Redis ecosystem of clients and libraries.
- [Vector Search Indexes](https://awesome-repositories.com/f/data-databases/search-indexing-technologies/search-indexing/search-and-indexing/vector-search-indexes.md) — Utilizes specialized indexing for high-dimensional vectors to enable fast semantic search at scale. ([source](https://microsoft.github.io/garnet/docs/azure/overview))
- [Server-Side Scripting](https://awesome-repositories.com/f/data-databases/server-side-scripting.md) — Executes logic-bearing scripts directly on the server to perform complex atomic operations and reduce network round trips. ([source](https://microsoft.github.io/garnet/docs))
- [Slot-Based Routing](https://awesome-repositories.com/f/data-databases/slot-based-routing.md) — Routes operations to the correct node based on hash slot ownership to manage distributed request flow. ([source](https://microsoft.github.io/garnet/docs/cluster/overview))
- [Key-Value](https://awesome-repositories.com/f/data-databases/storage-engines/key-value.md) — Implements high-performance get, set, and expiration commands for string key-value pairs via the RESP protocol. ([source](https://cdn.jsdelivr.net/gh/microsoft/garnet@main/README.md))
- [Storage Tiering](https://awesome-repositories.com/f/data-databases/storage-tiering.md) — Implements automated movement of cold data from system memory to SSD or cloud storage layers.
- [Stored Procedures](https://awesome-repositories.com/f/data-databases/stored-procedures.md) — Executes Lua scripts and C# stored procedures atomically on the server to perform complex multi-key operations.
- [Transactional Key-Value Modifications](https://awesome-repositories.com/f/data-databases/transactional-key-value-modifications.md) — Processes multi-key operations atomically via client requests or server-side procedures to ensure data consistency. ([source](https://cdn.jsdelivr.net/gh/microsoft/garnet@main/README.md))
- [Vector Search](https://awesome-repositories.com/f/data-databases/vector-search.md) — Provides a specialized database for storing and querying high-dimensional vectors using similarity search.
- [Append-Only Persistence](https://awesome-repositories.com/f/data-databases/data-engineering-infrastructure/data-persistence-storage/persistence-durability/append-only-persistence.md) — Ensures data durability through sequential write-ahead logging and non-blocking checkpoints.
- [Hash Maps](https://awesome-repositories.com/f/data-databases/hash-maps.md) — Stores collections of field-value pairs and supports atomic increments and range retrievals. ([source](https://microsoft.github.io/garnet/docs/commands/api-compatibility))
- [Hash-Tag Key Mapping](https://awesome-repositories.com/f/data-databases/hash-tag-key-mapping.md) — Uses hashtags to map multiple keys to the same shard, enabling atomic multi-key operations. ([source](https://microsoft.github.io/garnet/docs/cluster/overview))
- [Command Batching](https://awesome-repositories.com/f/data-databases/high-throughput-ingestion-pipelines/command-batching.md) — Groups multiple commands into a single request block to minimize network round trips and maximize throughput. ([source](https://microsoft.github.io/garnet/docs/extensions/overview))
- [Key Lifecycle Management](https://awesome-repositories.com/f/data-databases/key-lifecycle-management.md) — Manages key expiration timers, time-to-live calculations, and key renaming or deletion. ([source](https://microsoft.github.io/garnet/docs/commands/api-compatibility))
- [Tiered Memory Management](https://awesome-repositories.com/f/data-databases/large-scale-dataset-management/tiered-memory-management.md) — Spans data across RAM and persistent flash storage to handle datasets larger than available system memory. ([source](https://microsoft.github.io/garnet/docs/research/papers))
- [Geospatial Search](https://awesome-repositories.com/f/data-databases/search-indexing/geospatial-search.md) — Supports storing coordinate data and calculating distances or searching within geographic radii. ([source](https://microsoft.github.io/garnet/docs/commands/api-compatibility))
- [Secondary Indexes](https://awesome-repositories.com/f/data-databases/secondary-indexes.md) — Builds secondary indexes over incoming data streams using subset hashing for fast lookups. ([source](https://microsoft.github.io/garnet/docs/research/papers))
- [Set Data Structures](https://awesome-repositories.com/f/data-databases/set-data-structures/set-data-structures.md) — Supports collections of unique elements with mathematical set operations like intersections and unions. ([source](https://microsoft.github.io/garnet/docs/commands/api-compatibility))
- [Sorted Sets](https://awesome-repositories.com/f/data-databases/sorted-sets.md) — Stores unique elements associated with numerical scores to maintain ordered lists for ranking. ([source](https://microsoft.github.io/garnet/docs/commands/api-compatibility))
- [SSD Storage Extensions](https://awesome-repositories.com/f/data-databases/ssd-storage-extensions.md) — Extends available memory by offloading colder datasets to local SSD or cloud storage when RAM is exceeded. ([source](https://microsoft.github.io/garnet/docs))
- [Vector Indexing](https://awesome-repositories.com/f/data-databases/vector-indexing.md) — Provides specialized approximate nearest neighbor indexing for high-dimensional vectors to enable fast semantic similarity search.

### Software Engineering & Architecture

- [Latch-Free Shared Memory Engines](https://awesome-repositories.com/f/software-engineering-architecture/shared-memory-architectures/latch-free-shared-memory-engines.md) — Processes parallel requests across multiple cores using latch-free data structures and epoch protection for sub-millisecond latency.
- [Hash Slot Sharding](https://awesome-repositories.com/f/software-engineering-architecture/hash-based-data-distribution/hash-slot-sharding.md) — Distributes keys across cluster nodes using a consistent hashing scheme that maps each key to a specific slot.
- [Multi-Threaded Request Handling](https://awesome-repositories.com/f/software-engineering-architecture/high-throughput-task-processing/network-request-processing/multi-threaded-request-handling.md) — Distributes network IO and query processing across multiple CPU cores to maintain high throughput and low latency.
- [Atomic Transaction Grouping](https://awesome-repositories.com/f/software-engineering-architecture/atomic-transaction-grouping.md) — Groups multiple discrete operations into a single atomic unit to ensure consistent updates and reduce network overhead. ([source](https://microsoft.github.io/garnet/docs/extensions/overview))
- [Atomic Integer Operations](https://awesome-repositories.com/f/software-engineering-architecture/integer-arithmetic/atomic-integer-operations.md) — Performs thread-safe increment and decrement operations on integer values directly on the server. ([source](https://microsoft.github.io/garnet/docs))

### DevOps & Infrastructure

- [Cluster Configuration Management](https://awesome-repositories.com/f/devops-infrastructure/cluster-configuration-management.md) — Manages cluster behavior, node membership, and slot assignments via direct commands and gossip protocols. ([source](https://microsoft.github.io/garnet/docs/cluster/overview))
- [Gossip Protocols](https://awesome-repositories.com/f/devops-infrastructure/cluster-coordination/gossip-protocols.md) — Uses peer-to-peer gossip communication to propagate node membership and slot assignment updates across the distributed network.
- [Server Configuration Management](https://awesome-repositories.com/f/devops-infrastructure/server-configuration-management.md) — Provides interfaces to adjust runtime settings, monitor performance, and trigger manual data persistence. ([source](https://microsoft.github.io/garnet/docs/commands/api-compatibility))

### System Administration & Monitoring

- [Sharded Cluster Deployment](https://awesome-repositories.com/f/system-administration-monitoring/cluster-management/shard-rebalancing/sharded-cluster-deployment.md) — Distributes data across a sharded cluster with replication and dynamic key migration for high availability. ([source](https://cdn.jsdelivr.net/gh/microsoft/garnet@main/README.md))

### Networking & Communication

- [Pub-Sub Messaging](https://awesome-repositories.com/f/networking-communication/pub-sub-messaging.md) — Facilitates asynchronous communication between clients using a publish and subscribe pattern. ([source](https://microsoft.github.io/garnet/docs))

### Programming Languages & Runtimes

- [Linked Lists](https://awesome-repositories.com/f/programming-languages-runtimes/programming-utilities/data-structure-type-helpers/data-structures/linear-collections/linked-lists.md) — Implements linear collections allowing elements to be pushed, popped, or retrieved by index. ([source](https://microsoft.github.io/garnet/docs/commands/api-compatibility))

### Scientific & Mathematical Computing

- [Bit Manipulation Techniques](https://awesome-repositories.com/f/scientific-mathematical-computing/numerical-mathematical-foundations/algorithms-and-complexity/algorithms/bit-manipulation-techniques.md) — Performs bit-level operations such as setting, clearing, and counting bits within string values. ([source](https://microsoft.github.io/garnet/docs/commands/api-compatibility))