Awesome GitHub RepositoriesLog Compaction

This project serves as a comprehensive technical reference for the architecture and design of data-intensive applications. It provides a structured analysis of the fundamental principles required to build reliable, scalable, and maintainable software systems, covering the core trade-offs inherent in modern data infrastructure. The repository explores the mechanics of distributed data management, including strategies for replication, partitioning, and achieving consensus across multiple nodes. It details the design of storage engines, indexing techniques, and transaction management models, whi

NeDB is a JavaScript embedded NoSQL document store designed for Node.js and the browser. It functions as an in-memory data store with the option to persist documents to a local file system, ensuring data survives application restarts. The project utilizes a MongoDB-compatible API to perform data operations, allowing it to serve as a lightweight document indexing system and a persistent file database without requiring a separate database server. Capabilities include querying, inserting, updating, and deleting documents, as well as the ability to create indexes on specific fields to accelerate

ToyDB is a distributed SQL database that provides a system for storing and querying data across multiple nodes. It focuses on maintaining strong consistency and fault tolerance through the implementation of a distributed consensus algorithm. The project distinguishes itself by supporting historical data versioning, enabling time-travel queries to retrieve the state of the database from a specific point in the past. It utilizes multi-version concurrency control to manage ACID transactions and ensure data integrity during concurrent operations. The system covers relational data modeling with t

هذا المشروع هو ترجمة للغة الصينية للورقة البحثية الأصلية التي تفصل بروتوكول إجماع Raft. يعمل كترجمة بحثية تقنية ودليل لبروتوكول الإجماع، مما يجعل مواصفات خوارزمية Raft في متناول المتحدثين بالصينية. يغطي التوثيق الآليات الأساسية للأنظمة الموزعة، بما في ذلك انتخاب القائد، ونسخ السجل، وبروتوكولات السلامة. ويوفر شرحاً مفصلاً لكيفية الحفاظ على مصدر واحد للحقيقة عبر خوادم متعددة لتحقيق إدارة مجموعة متسامحة مع الأخطاء. تتناول المادة نسخ آلة الحالة الموزعة وإدارة السجل. وتغطي مفاهيم تقنية مثل تتبع الحيوية القائم على نبضات القلب، وانتخابات المهلة العشوائية، وإصدار الاتساق القائم على المصطلح، وضغط السجل القائم على اللقطات.

m3 is a distributed time series database designed for high-resolution metrics and high-cardinality data management. It functions as a scalable storage system and a multi-cluster query engine, providing a distributed metrics aggregator capable of downsampling and summarizing data before it is committed to storage. The project distinguishes itself through a coordinated cluster model using etcd for node membership and shard placement. It supports multiple ingestion protocols, including the Prometheus remote write protocol, InfluxDB line protocol, and Graphite Carbon plaintext protocol, and provi

braft is an embeddable C++ library that implements the Raft consensus algorithm, providing a distributed consensus engine for building fault-tolerant, replicated state machines. At its core, it manages leader election, log replication, cluster membership changes, and state machine synchronization across a cluster of nodes, ensuring strong consistency and data durability even in the face of node failures. The library distinguishes itself through a comprehensive set of mechanisms for reliable distributed coordination. It uses a randomized timeout-based leader election process with term manageme

sofa-jraft is a Java implementation of the Raft consensus algorithm. It serves as a distributed consensus engine and linearizable state machine designed to ensure high availability and data consistency across a cluster of nodes. The project provides a replicated key-value store and a coordination engine for managing distributed state. It distinguishes itself through support for multi-group consensus sharding to distribute traffic and a service provider interface that allows for custom log storage and entry encoding implementations. The system covers a wide range of distributed capabilities,

NutsDB is an ACID-compliant, embedded transactional storage engine that functions as both a disk-backed key-value store and an in-memory data structure store. It provides atomic and serializable transactions with commit and rollback capabilities to ensure strict data consistency for applications requiring a lightweight persistence layer. The engine distinguishes itself by supporting a variety of complex data types, including lists, sets, and sorted sets, alongside standard byte-slice storage. It implements a transactional storage model featuring hot backups and a compaction algorithm to maint