Awesome GitHub RepositoriesDistributed Consensus Stores

etcd is a distributed key-value store and configuration store designed to maintain a consistent set of data across a cluster of nodes. It functions as a reliable registry for storing and synchronizing critical settings and metadata used by distributed applications. The system implements the Raft consensus algorithm to ensure data consistency and leader election across servers. To protect data transfers and verify node identities, it utilizes a network security layer based on mutual TLS and client certificates. Its capabilities cover distributed configuration management, cluster state synchro

TiDB is a horizontally scalable, distributed SQL database designed to provide consistent transactional storage and high-performance analytical processing within a single unified architecture. It utilizes a decoupled compute-storage design and a distributed key-value storage layer to ensure horizontal scalability and efficient range-based queries. By employing a consensus-based replication algorithm, the system maintains high availability and automatic failover across multiple nodes and geographical regions. The platform distinguishes itself through its hybrid transactional and analytical proc

Vitess is a database clustering system for horizontal scaling of MySQL. It functions as a middleware layer that abstracts complex sharding and physical topology, allowing applications to interact with a distributed database environment through a unified interface. By intercepting and routing SQL queries across multiple shards, it enables large-scale data management while maintaining the appearance of a single database instance. The platform distinguishes itself through its ability to perform online schema migrations and distributed transaction coordination without requiring application downti

rqlite is a distributed relational database that replicates SQLite data across a cluster using the Raft consensus algorithm. It functions as a fault-tolerant storage system that provides high availability and a web API for executing SQL queries and managing relational data without requiring native database drivers. The system distinguishes itself by using an HTTP SQL interface to expose database operations and cluster management. It features a real-time change data capture stream that pushes database mutations to external HTTP endpoints via webhooks and supports the scaling of read throughput

TiKV is a cloud-native distributed transactional key-value store and storage engine. It provides a distributed database designed for horizontal scalability and strong consistency across a cluster of physical nodes. The system uses a Raft-based consensus mechanism to maintain data availability and state synchronization. It ensures ACID compliance for distributed transactions through a two-phase commit workflow and manages data distribution via multi-Raft sharding. The engine handles massive datasets using automated range splitting and cluster load balancing to distribute data across different

ZooKeeper is a distributed coordination service that provides a centralized system for managing configuration, naming, and synchronization across a cluster of distributed processes. It functions as a cluster consensus service, a distributed configuration store, and a distributed lock manager to maintain a consistent state across multiple network nodes. The service implements a consensus protocol to ensure data consistency and uses a replicated state machine to maintain identical copies of the system state across all servers. It provides a distributed lock management system to coordinate exclu

Nebula is a distributed graph database designed for storing and querying massive volumes of interconnected vertices and edges across a horizontally scalable cluster. It functions as a Kubernetes-native database and a distributed graph analytics engine, utilizing a Raft-based distributed store to ensure strong consistency and high availability. The system features an OpenCypher query engine for performing complex graph traversals and pattern matching. It distinguishes itself with a decoupled compute-storage architecture and a shared-nothing distributed design, allowing query processing and dat

OceanBase is a distributed SQL database designed for high availability and strong consistency across multiple nodes and regions. It functions as a hybrid transactional and analytical processing engine, allowing real-time analytics and transactions to execute on a single data copy. The system also serves as a vector database engine for indexing and querying vector data to power semantic search and recommendation systems. The platform features native compatibility layers for MySQL and Oracle, enabling the migration of legacy workloads without rewriting SQL code. It utilizes a Paxos-based distri

Cassandra is a distributed NoSQL database and wide-column store designed for high availability and linear scalability. It functions as a fault-tolerant distributed system that utilizes an LSM-tree storage engine to optimize write throughput and manage massive datasets. The system is a CQL-compliant database, using a structured query language to manage and retrieve tabular data stored across multiple nodes. It organizes information into rows and columns based on a flexible schema and primary keys. The project provides capabilities for horizontal database scaling, distributed data partitioning

Patroni is a high availability manager and cluster orchestrator for PostgreSQL. It functions as an automatic failover controller and replication manager that ensures continuous database availability by automating leader election and promoting standby nodes during failures. The system maintains a consistent cluster state by acting as a distributed consensus coordinator. It synchronizes configuration and manages leader elections through integration with distributed configuration stores such as etcd, ZooKeeper, or Consul. Its broader capabilities include managing both synchronous and asynchrono

Hazelcast is a distributed data platform that combines an in-memory data grid with a stream processing engine to support real-time analytics and event-driven applications. It functions as a partitioned, distributed key-value store that replicates data across cluster nodes to provide low-latency access and high availability. The platform also serves as a distributed SQL query engine, allowing users to execute standard SQL statements against both in-memory datasets and external data sources. What distinguishes Hazelcast is its use of a distributed consensus subsystem to maintain strongly consis

Pigsty es una plataforma integral de orquestación de infraestructura de bases de datos diseñada para automatizar el ciclo de vida completo de clústeres de PostgreSQL de alta disponibilidad. Funciona como un framework de infraestructura como código que gestiona la coordinación de clústeres, el aprovisionamiento de nodos y el descubrimiento de servicios a través de playbooks idempotentes. Al integrar mecanismos de consenso distribuido, la plataforma garantiza la conmutación por error automatizada y la aplicación de estado consistente en diversos entornos, incluyendo infraestructura bare metal y virtualizada. La plataforma se distingue por un sólido conjunto de capacidades operativas que se extienden más allá de la gestión estándar de bases de datos. Cuenta con una tubería de observabilidad integrada que agrega métricas, registros y trazas en paneles centralizados para la monitorización del rendimiento en tiempo real y el análisis de diagnóstico. Además, proporciona un framework de migración que emula protocolos de cable propietarios y sintaxis SQL, permitiendo la integración de cargas de trabajo de bases de datos empresariales heredadas en entornos relacionales modernos. El sistema cubre una amplia superficie funcional, incluyendo gestión avanzada de almacenamiento con clonación de copia en escritura para un despliegue rápido, y orquestación de múltiples bases de datos que coordina motores relacionales con almacenamiento en caché y almacenamiento de objetos. También incorpora endurecimiento de seguridad, copia de seguridad y recuperación automatizadas, y enrutamiento de tráfico a través de proxies en capas para desacoplar las conexiones de los clientes de la topología del clúster subyacente. El proyecto se distribuye como un modelo de espejo de paquetes autónomo, lo que permite un despliegue y una gestión de dependencias consistentes en entornos seguros o aislados.

Jocko es una plataforma de streaming de eventos cloud-native y un registro de commits distribuido implementado en Go. Funciona como un broker de mensajes distribuido que garantiza la durabilidad de los datos y la alta disponibilidad mediante la replicación de secuencias de registros a través de un clúster. El sistema está diseñado como un streamer de eventos sin Zookeeper, utilizando coordinación de consenso integrada para gestionar el estado del clúster y la elección de líder sin requerir servicios de coordinación externos. Implementa el protocolo de red de Kafka, lo que le permite comunicarse con clientes y herramientas del ecosistema existente. La plataforma proporciona capacidades para la gestión de registros distribuidos, incluyendo el uso de un registro de commits de solo adición (append-only) para la persistencia de datos. También incluye mecanismos automatizados de retención de datos que purgan registros antiguos basados en límites de tiempo o umbrales de espacio en disco. El proyecto se distribuye como un runtime de binario único.

sofa-jraft is a Java implementation of the Raft consensus algorithm. It serves as a distributed consensus engine and linearizable state machine designed to ensure high availability and data consistency across a cluster of nodes. The project provides a replicated key-value store and a coordination engine for managing distributed state. It distinguishes itself through support for multi-group consensus sharding to distribute traffic and a service provider interface that allows for custom log storage and entry encoding implementations. The system covers a wide range of distributed capabilities,

Trillian is a distributed, multi-tenant verifiable data store that maintains cryptographically verifiable logs and maps using Merkle tree structures. It functions as a scalable backend for transparency logs, providing a system where data integrity is ensured through append-only records and mathematical proofs of inclusion and consistency. The system distinguishes itself by decoupling core storage from application-specific logic through a personality layer, which handles admission criteria and data canonicalization. It employs a consensus-based leader election mechanism for high availability a