Awesome GitHub RepositoriesCluster Coordination

FoundationDB is an ACID-compliant distributed transactional key-value store. It functions as a scalable database engine that ensures strict serializability and data consistency across a cluster of servers using a shared-nothing architecture. The system is distinguished by its multi-region replication capabilities, allowing data to be synchronized across different datacenters for high availability and disaster recovery. It utilizes optimistic concurrency control to manage distributed transactions and employs a majority-based coordination system to maintain cluster state. The platform provides

Stalwart is a self-hosted email and collaboration infrastructure that provides an integrated mail server supporting SMTP, IMAP, POP3, and JMAP protocols. It functions as a comprehensive communication hub, combining email hosting with a collaboration server for shared calendars, contacts, and files. The system distinguishes itself through a distributed architecture that uses peer-to-peer cluster coordination to ensure high availability and fault tolerance. It features a built-in security suite that implements an S/MIME and OpenPGP email gateway alongside automated TLS certificate provisioning

Akka Core is an actor model framework and asynchronous concurrency library used for building scalable and resilient distributed systems. It provides a distributed computing platform and fault tolerant runtime that manages communication and state across networked nodes. The system uses location-transparent messaging and a cluster management system to organize nodes into high-availability architectures. This allows for the creation of elastic clusters that scale resources on demand and coordinate distributed workloads. The platform handles concurrent state management and distributed systems or

Kafka Manager is a web-based management interface and monitoring tool for Apache Kafka clusters. It serves as a central control plane for topic administration, consumer monitoring, and cluster health inspection. The project provides specialized utilities for data rebalancing and partition reassignment to distribute workloads across brokers. It also includes tools to optimize partition leadership by electing preferred replicas. The platform covers a broad range of administrative capabilities, including the creation and configuration of message topics, tracking of consumer offsets, and the col

Garnet is a multi-threaded in-memory database and distributed key-value store. It functions as a high-performance remote cache store that implements the RESP wire protocol to maintain compatibility with existing Redis clients and libraries. The project is distinguished by a shared-memory architecture that enables parallel request processing across multiple cores for sub-millisecond latency. It features a tiered storage system that automatically offloads colder data from system memory to SSD or cloud storage layers, and includes a specialized vector search database for high-dimensional similar

Redis is a high-performance in-memory key-value store that functions as a distributed cache, message broker, and NoSQL database. It provides sub-millisecond read and write access to data stored in RAM and can operate as a vector database for indexing high-dimensional embeddings. The system supports a wide range of data storage and synchronization primitives, including the management of strings, hashes, lists, sets, and JSON documents. It enables real-time data operations through atomic transactions, hybrid persistence using snapshots and append-only logs, and high-availability configurations

Accelerate is a PyTorch distributed training library that abstracts the boilerplate required to run models across multiple GPUs, TPUs, and CPUs. It functions as a deep learning model scaler and distributed hardware orchestrator, allowing the same training script to run on different hardware backends without modifying the core logic. The project provides a distributed training command line interface for configuring compute environments and launching jobs across single or multi-node clusters. It includes a mixed precision training framework to implement FP16 and BF16 precision, reducing memory

rkt is a pod-native container engine and runtime for Linux that executes containerized applications as isolated pods. It serves as an OCI container runtime and a Linux container manager, supporting the execution of images based on Open Container Initiative, appc, and Docker specifications. The project distinguishes itself by offering hardware-level container isolation, allowing pods to run within virtual machines using KVM or QEMU for a dedicated kernel. It further separates itself through secure container deployment practices, utilizing SELinux mandatory access control and TPM-backed integri

Alertmanager is a monitoring notification gateway and routing service that deduplicates, groups, and directs alerts to the correct receivers. It functions as a central manager for Prometheus alerts, using a hierarchical routing tree and label-based matchers to dispatch notifications to external services. The system employs a peer-to-peer mesh network to coordinate multiple instances in a high availability cluster, ensuring continuous alert processing. It features a dedicated inhibition engine and grouping mechanisms to reduce notification noise by suppressing redundant alerts when related iss

Disque is a distributed message broker that manages asynchronous communication between services through in-memory job queues. It operates as a cluster-aware job scheduler, providing at-least-once delivery guarantees with acknowledgment and retry mechanisms, and supports blocking queue consumers that efficiently wait for jobs to become available. The broker implements a job state machine that tracks each job through its lifecycle from queued to acknowledged, with configurable retry limits and time-to-live expiration to prevent unbounded queue growth. Jobs can be enqueued with delivery guarante

dockerlabs is a collection of educational labs and technical tutorials designed to teach the fundamentals of containerization and microservice architecture. It provides instructional material and hands-on exercises covering image optimization, security training, infrastructure setup, and cluster orchestration. The project features specific courses and guides focused on reducing image size through multi-stage builds, securing workloads via vulnerability scanning and encrypted networks, and deploying multi-node clusters with high availability using Swarm orchestration. The materials cover a br

Elsa Core is a workflow engine framework designed for defining, executing, and managing long-running business processes. It functions as a distributed workflow orchestrator and event-driven trigger system, capable of operating as a multi-tenant platform with secure data isolation. The project distinguishes itself through a flexible approach to workflow definitions, supporting a visual drag-and-drop designer, programmatic C# definitions, and portable JSON specifications. It provides a highly extensible architecture allowing for the development of custom activities and the use of a dynamic expr

Dieses Projekt ist eine chinesische Übersetzung des ursprünglichen Forschungspapiers, das das Raft-Konsensprotokoll detailliert beschreibt. Es dient als technische Forschungsübersetzung und Leitfaden für Konsensprotokolle und macht die Spezifikationen des Raft-Algorithmus für chinesischsprachige Personen zugänglich. Die Dokumentation deckt die Kernmechanismen verteilter Systeme ab, einschließlich Leader-Wahl, Log-Replikation und Sicherheitsprotokollen. Sie bietet eine detaillierte Erklärung, wie eine einzige Quelle der Wahrheit über mehrere Server hinweg aufrechterhalten wird, um eine fehlertolerante Cluster-Verwaltung zu erreichen. Das Material adressiert verteilte Zustandsmaschinen-Replikation und Log-Management. Es behandelt technische Konzepte wie Heartbeat-gesteuertes Liveness-Tracking, Wahlen mit randomisierten Timeouts, termbasierte Konsistenz-Versionierung und Snapshot-basiertes Log-Compaction.

Serf is a decentralized cluster coordination tool that manages node membership, failure detection, and event broadcasting across a distributed system without a central coordinator. Every node runs an identical agent process that independently handles membership, health monitoring, and event propagation through a peer-to-peer gossip protocol, creating a leaderless architecture where no single point of failure exists. The project implements the SWIM failure detection algorithm, where each node monitors a small random subset of peers to detect unreachable or failed nodes in real time. Custom use

Algodeck is an open-source collection of flash cards designed for reviewing algorithms, data structures, and system design concepts, specifically curated for technical interview preparation. The project organizes knowledge into atomic question-and-answer pairs and incorporates spaced repetition scheduling to optimize long-term memory retention. The flash card catalog covers a broad range of computer science topics, including classic sorting algorithms like quicksort and mergesort, data structure operations for arrays, trees, heaps, tries, and graphs, as well as bit manipulation techniques for

Cortex is an open-source, horizontally scalable metrics platform that ingests, stores, and queries Prometheus-compatible time-series data with multi-tenant isolation. It accepts metrics via Prometheus remote write and OpenTelemetry, executes PromQL queries against both recent and historical data, and provides a Prometheus-compatible alerting and recording rule engine with an integrated Alertmanager. The system is built as a set of independently scalable microservices that use hash-ring-based sharding, gossip-based cluster membership, and tenant-aware object storage to distribute workloads acro

protoactor-go is a framework for building concurrent and distributed systems in Go using the actor model. It provides a distributed actor system that enables isolated entities to communicate via asynchronous messaging and share state across a cluster. The framework implements a multi-language actor protocol, allowing interoperability between actors written in Go, C#, and Java. It further supports a virtual actor implementation, where actors are automatically instantiated across a network based on a unique identity. The system includes a supervision model for managing actor lifecycles and fau

Mimir ist eine Multi-Tenant-Zeitreihendatenbank und ein verteilter Metrik-Speicher für skalierbare Telemetrie. Es dient als Prometheus-kompatibles Backend und bietet Langzeitspeicherung sowie eine skalierbare Abfrage-Engine für massive Mengen an Zeitreihendaten. Das System ist für Multi-Tenant-Observability konzipiert und isoliert Telemetriedaten sowie Ressourcenlimits für unabhängige Teams oder Organisationen innerhalb eines einzigen Clusters. Es gewährleistet hohe Verfügbarkeit und Langlebigkeit durch Sharding und Replikation von Daten über einen verteilten Cluster hinweg und nutzt Objektspeicher zur Persistenz, um externe Datenbankabhängigkeiten zu eliminieren. Das Projekt deckt weitreichende Fähigkeiten ab, einschließlich globaler Metrik-Aggregation für regionsübergreifende Analysen und verteilter Abfrageausführung mittels Parallelisierung und Caching. Es integriert zudem Observability-Tools wie föderiertes Alerting, synthetisches Monitoring und KI-gestützte Incident-Resolution-Workflows zur Beschleunigung der Fehlerbehebung. Administrative Kontrollen umfassen Tenant-Ressourcenquoten, benutzerbezogene Ressourcen-Overrides und Shuffle-Sharding für Workload-Isolierung.

Jocko ist eine Cloud-native Event-Streaming-Plattform und ein verteilter Commit-Log, implementiert in Go. Es fungiert als verteilter Message-Broker, der Datenbeständigkeit und Hochverfügbarkeit durch die Replikation von Datensatzsequenzen über einen Cluster hinweg sicherstellt. Das System ist als Zookeeper-freier Event-Streamer konzipiert und nutzt eine integrierte Konsens-Koordination, um den Cluster-Zustand und die Leader-Wahl ohne externe Koordinationsdienste zu verwalten. Es implementiert das Kafka-Wire-Protokoll, wodurch es mit bestehenden Ökosystem-Clients und Tools kommunizieren kann. Die Plattform bietet Funktionen für das verteilte Log-Management, einschließlich der Verwendung eines Append-only-Commit-Logs für die Datenpersistenz. Sie enthält zudem automatisierte Mechanismen zur Datenaufbewahrung, die alte Datensätze basierend auf Zeitlimits oder Speicherplatzschwellenwerten bereinigen. Das Projekt wird als Single-Binary-Laufzeitumgebung vertrieben.

Akka.NET is an actor model framework used for building concurrent and distributed applications. It functions as a distributed computing platform and state manager that enables isolated actors to communicate via asynchronous message passing, ensuring thread-safe state management without manual locks. The project is distinguished by its decentralized coordination capabilities, including a distributed state manager that uses sharding and dynamic rebalancing to maintain high availability. It incorporates an event sourcing engine that persists state as a sequence of events in an append-only log an