Awesome GitHub RepositoriesDistributed Commit Logs

Kafka is a distributed event streaming platform designed for capturing, storing, and processing real-time data streams across interconnected nodes. It functions as a distributed commit log, providing a fault-tolerant storage mechanism that records state changes sequentially to ensure data consistency and durability across distributed environments. The platform distinguishes itself through a partitioned commit log architecture that enables horizontal scaling and parallel processing of data streams. It integrates a stream processing engine for continuous transformations and aggregations, while

This is a Raft consensus library and distributed consensus engine implemented in Go. It provides the primitives necessary to build fault-tolerant distributed services by implementing a replicated state machine that ensures a group of servers agree on a shared system state through leader election and log replication. The project distinguishes itself through a pluggable architecture for storage backends and snapshot storage, decoupling the consensus logic from physical persistence. It includes specialized mechanisms for leadership transfer, protocol version management to support rolling upgrade

immudb is a tamperproof database that maintains an immutable record of entries using cryptographic commit logging. It ensures verifiable database integrity by utilizing Merkle trees to generate membership and consistency proofs that detect unauthorized data alterations. The system employs a multi-model storage engine that unifies key-value, document, and relational data structures within a single immutable backend. It provides compatibility with the PostgreSQL wire protocol, allowing it to integrate with standard SQL clients, ORMs, and database tools. The project covers broad capabilities in

Orbit DB is a decentralized NoSQL database that utilizes conflict-free replicated data types to ensure eventual consistency across a network of nodes. It functions as a peer-to-peer data store that uses IPFS for content-addressing and synchronization, allowing for the maintenance of application state without a central server or authority. The system is built upon a cryptographically verifiable, immutable operation log, which serves as the foundation for custom decentralized data models. This architecture enables the implementation of various data storage patterns, including JSON document stor

OrbitDB is a decentralized data storage system that enables the creation of serverless databases residing across a network of peers. It functions as a peer-to-peer database that integrates with a content-addressed storage layer to distribute and replicate data without a central server. The system utilizes conflict-free replicated data types to ensure eventual consistency and state convergence across distributed nodes. It maintains an immutable record of updates using a directed acyclic graph to preserve causal ordering and cryptographic integrity. Access is managed through a decentralized ide

Sui is a blockchain platform featuring an object-centric state model and resource-oriented smart contracts. It utilizes parallel transaction execution to increase network throughput and supports programmable transaction blocks that bundle multiple operations into single atomic units. The platform distinguishes itself with a capability-based access control system and zero-knowledge login mechanisms, enabling users to authenticate via identity providers without seed phrases. It also implements deterministic object addressing to allow predictable state lookups and supports the creation of soulbo

该项目是一个可验证的数据库引擎，提供 SQL 加密证明系统，以确保存储信息及其检索的完整性。它作为一个支持证明的数据存储，利用加密承诺来防止数据篡改并保证查询结果的准确性。 该系统生成零知识证明，提供数学证据证明 SQL 查询已正确执行，而无需重新运行整个过程。这些加密证明可以导出以进行链上验证，允许智能合约或第三方以无需信任的方式验证数据库结果。 该引擎包括一个高性能分析 SQL 查询处理器，能够在数百万行数据上执行复杂的聚合和过滤，响应时间在亚秒级。它维护一个安全的数据摄取流水线，在输入期间创建防篡改哈希，以确保数据库状态的完整性。

Jocko 是一个云原生的事件流平台和分布式提交日志，使用 Go 语言实现。它作为一个分布式消息代理，通过在集群中复制记录序列来确保数据持久性和高可用性。 该系统被设计为无需 Zookeeper 的事件流平台，利用内置的共识协调来管理集群状态和领导者选举，无需外部协调服务。它实现了 Kafka 有线协议，允许其与现有的生态系统客户端和工具进行通信。 该平台提供了分布式日志管理能力，包括使用仅追加（append-only）的提交日志进行数据持久化。它还包含自动数据保留机制，可根据时间限制或磁盘空间阈值清除旧记录。 该项目以单一二进制文件的形式发布。

Trillian is a distributed, multi-tenant verifiable data store that maintains cryptographically verifiable logs and maps using Merkle tree structures. It functions as a scalable backend for transparency logs, providing a system where data integrity is ensured through append-only records and mathematical proofs of inclusion and consistency. The system distinguishes itself by decoupling core storage from application-specific logic through a personality layer, which handles admission criteria and data canonicalization. It employs a consensus-based leader election mechanism for high availability a