Awesome GitHub RepositoriesKey-Value Stores

etcd is a distributed key-value store and configuration store designed to maintain a consistent set of data across a cluster of nodes. It functions as a reliable registry for storing and synchronizing critical settings and metadata used by distributed applications. The system implements the Raft consensus algorithm to ensure data consistency and leader election across servers. To protect data transfers and verify node identities, it utilizes a network security layer based on mutual TLS and client certificates. Its capabilities cover distributed configuration management, cluster state synchro

Consul is a distributed coordination service and service mesh tool used for service discovery, health monitoring, and cluster state management across dynamic networks. It provides a platform for locating network addresses of services and managing traffic across distributed infrastructure using DNS and HTTP interfaces. The project distinguishes itself through multi-datacenter network orchestration, enabling the federation of services across different regions using mesh gateways. It secures communication via a service mesh architecture that employs identity-based authorization and mutual TLS en

Redisson is a Java library and Redis client that functions as a distributed Java object mapper, caching provider, and locking framework. It maps Java collections and concurrency primitives to distributed implementations backed by Redis and Valkey, providing synchronous, asynchronous, and reactive APIs for interacting with these data stores. The project distinguishes itself by providing a comprehensive suite of distributed coordination tools, including a locking framework for managing semaphores and countdown latches across multiple application nodes. It also serves as a distributed messaging

TiKV is a cloud-native distributed transactional key-value store and storage engine. It provides a distributed database designed for horizontal scalability and strong consistency across a cluster of physical nodes. The system uses a Raft-based consensus mechanism to maintain data availability and state synchronization. It ensures ACID compliance for distributed transactions through a two-phase commit workflow and manages data distribution via multi-Raft sharding. The engine handles massive datasets using automated range splitting and cluster load balancing to distribute data across different

Badger is an embeddable key-value store written in Go that provides persistent data storage for byte keys and values. It is a persistent database that utilizes a tiered LSM tree storage model to optimize disk storage and retrieval efficiency. The system features an ACID transaction engine that ensures data integrity through serializable snapshot isolation and multi-version concurrency control. It also provides an encrypted key-value store with data-at-rest encryption and a managed encrypted key registry to secure stored information. The engine covers a broad set of capabilities including hig

Bolt is a single-file embedded key-value store for Go applications. It is an ACID transactional database that organizes data in B+trees on disk to provide efficient sorted key retrieval and range scans. The system uses a memory-mapped model to map the database file directly into the process address space for fast random-access reads. The project distinguishes itself through a multi-version concurrency control architecture that allows multiple simultaneous readers to access a consistent snapshot of data without blocking a writer. It employs a single-writer multi-reader locking model and uses a

Dice is a low-latency key-value store based on Valkey that functions as a hierarchical storage engine. It combines a high-speed memory store with a tiered cache system to manage data across multiple storage levels. The system extends memory capacity by implementing a disk-backed cache that automatically persists evicted items to disk and restores them during cache misses. This storage tiering balances retrieval speed and operational cost by distributing data between fast memory and slower disk storage. The engine includes a query subscription service that monitors specific data patterns and

Tile38 is an in-memory geospatial database that uses the Redis protocol for communication and query execution. It serves as a distributed spatial store for points and polygons, featuring high availability through leader-follower replication and disk-based persistence. The system includes a real-time geofencing engine that monitors virtual geographic boundaries and triggers webhooks when objects enter or exit specified areas. It further distinguishes itself with an embedded Lua scripting engine for server-side data transformations and a built-in vector tile server for efficient map visualizati

go-cache is a thread-safe, in-memory cache library for Go that stores arbitrary objects with per-item expiration timestamps. It provides a concurrent key-value store where multiple goroutines can safely read and write shared cached data without external synchronization, using a mutex-guarded map for access control. The library distinguishes itself through its expiration management and optional disk persistence. Each cached item carries its own time-to-live, and a background goroutine periodically purges expired entries. The cache can serialize its entire contents to disk using Go's gob encodi

This is a Golang client library for interacting with a cloud native distributed messaging system. It provides the necessary tools for Go applications to exchange messages using publish-subscribe and request-reply patterns, as well as specialized clients for managing persistent streams and distributed storage. The library includes a JetStream client for durable message streaming and replay, a Key-Value store client for managing distributed state with versioning and watchers, and an Object Store client for the storage and retrieval of large binary files via chunked delivery. The implementation

Pebble is an embedded key-value storage engine written in Go, designed as a library that provides durable, write-optimized data persistence directly within applications. It organizes data using a log-structured merge-tree (LSM-tree) structure, where writes are first buffered in an in-memory skiplist memtable and persisted to a write-ahead log before being flushed to block-based SSTable files on disk. The engine supports atomic batch commits, configurable write synchronization, and automatic background compaction that merges and rewrites sorted runs to reclaim space and maintain read performanc

FastStream is an asynchronous Python framework designed for building event-driven microservices. It provides a unified abstraction layer for interacting with various message brokers, enabling developers to manage event production and consumption through a consistent interface while maintaining access to native provider-specific features. The framework centers on a decorator-based routing model that binds application logic directly to broker topics, supported by a built-in dependency injection container that resolves resources at runtime. The framework distinguishes itself through its deep int

RoseDB ist eine eingebettete Key-Value-Datenbank und eine Log-strukturierte Storage-Engine. Sie fungiert als Bibliotheks-basierte Datenbank, die innerhalb eines Anwendungsprozesses läuft, um lokale Datenpersistenz ohne die Notwendigkeit eines separaten Datenbankservers bereitzustellen. Das Projekt implementiert einen Bitcask-artigen Store und nutzt ein Append-only-Log-Format, um den Schreibdurchsatz zu maximieren und die Suchzeit zu minimieren. Es stellt Datenkonsistenz sicher, indem mehrere Lese- und Schreiboperationen in atomaren Einheiten gruppiert werden. Die Speicherarchitektur basiert auf einem In-Memory-Key-Index für konstante Suchzeiten und verwendet Segment-basiertes Dateimanagement, um Datenkompression und die Wiederherstellung von Speicherplatz auf der Festplatte zu handhaben.

BuntDB is an embedded key-value store for Go applications, providing in-memory storage with optional disk persistence. It structures data using a B-tree for ordered key-value access and an R-tree for spatial indexing, allowing both range scans and geometric intersection queries. Support for indexing on nested JSON document fields enables efficient lookups by values within JSON objects, and per-key time-to-live (TTL) expiration automatically removes stale entries. The store uses copy-on-write transaction isolation, ensuring each transaction sees a consistent snapshot and changes are applied at

Riak ist ein dezentraler NoSQL-Key-Value-Store, der für hohe Verfügbarkeit und Fehlertoleranz entwickelt wurde. Es handelt sich um eine verteilte Datenbank im Dynamo-Stil, die konsistentes Hashing und Eventual Consistency nutzt, um Daten über einen Cluster von Knoten zu skalieren. Das System fungiert als geo-replizierter Datenspeicher, der Daten über mehrere physische Standorte hinweg synchronisiert, um Disaster Recovery zu ermöglichen und die Verfügbarkeit bei regionalen Ausfällen aufrechtzuerhalten. Es verwendet ein Multi-Master-Replikationsmodell, das Schreibvorgänge auf jedem Knoten im Cluster erlaubt. Die Plattform deckt breite Funktionsbereiche ab, einschließlich dezentraler Cluster-Koordinierung, hochverfügbarer Datenverwaltung und großskaliger Datenablaufsteuerung. Sie verwaltet die Datenbeständigkeit durch quorum-basierte Konsistenz und Schreibverifizierung, während sie spezialisierte Datenstrukturen verwendet, um gleichzeitige Updates automatisch zusammenzuführen.

Get your data in RAM. Get compute close to data. Enjoy the performance.

A simple, fast, embeddable, persistent key/value store written in pure Go. It supports fully serializable transactions and many data structures such as list, set, sorted set.

Example code from the book

Most advanced key-value database written in Go, extremely fast, compatible with LSM tree and B+ tree.

High-performance, columnar, in-memory store with bitmap indexing in Go