Awesome GitHub RepositoriesDistributed Systems

Developer Roadmap 是一个社区驱动的平台，提供结构化的、基于图谱的软件工程学习路径。它作为一个综合知识仓库，将技术领域组织成可视化序列，以指导专业技能获取和职业成长。 该项目通过协作生态系统脱颖而出，使用户能够贡献路线图、策划行业最佳实践并维护个人职业档案。它集成了诊断评估框架来评估技术熟练度，帮助开发者识别知识缺口，并通过有针对性的学习序列为专业面试做准备。 除了核心映射能力外，该平台还提供实用的项目创意和交互式辅导，以巩固工程概念。它为社区提供了一个共享资源、跟踪技能进步和导航复杂技术领域的中心化空间。

This project is a comprehensive, community-curated knowledge base designed to support software engineers in mastering both fundamental computer science principles and practical industry methodologies. It serves as a centralized reference library that aggregates technical resources, academic literature, and professional guidance to facilitate systematic skill acquisition across the entire software development lifecycle. What distinguishes this repository is its holistic approach to the engineering profession, which bridges the gap between theoretical knowledge and career-oriented development.

This project is a community-driven educational repository that serves as a comprehensive directory of university-level computer science video lectures. It provides a structured learning path for students and professionals, aggregating high-quality academic resources to facilitate self-paced study across a wide range of technical disciplines. The repository distinguishes itself through a collaborative maintenance model, utilizing version control workflows to allow contributors to expand and update the collection. Content is organized within a single, version-controlled document that leverages

This project is a comprehensive Java backend engineering guide and technical reference focused on high-concurrency design, distributed systems, and microservices architecture. It provides detailed strategies for decomposing monolithic applications, managing service discovery, and implementing the architectural patterns required for scalable backend environments. The repository distinguishes itself through an extensive collection of big data algorithmic references and database scaling strategies. It covers memory-efficient techniques for analyzing massive datasets, such as Top-K element extrac

This project is a curated knowledge repository that aggregates high-quality resources, technical documentation, and expert insights focused on distributed systems engineering. It serves as a community-driven learning resource designed to help developers navigate the complexities of building and maintaining large-scale software applications. The repository distinguishes itself through a hierarchical taxonomy that organizes vast amounts of technical information into a structured, searchable format. By utilizing markdown-based content curation and static indexing, the collection remains version-

Socket.io is a real-time communication engine that enables bidirectional, event-based data exchange between clients and servers. It provides a robust transport-agnostic protocol layer that automatically manages connection lifecycles, including heartbeat signals, automatic reconnection, and seamless fallback between WebSockets and HTTP long-polling. By maintaining persistent links, it ensures reliable messaging across diverse network environments. The project distinguishes itself through a scalable, distributed architecture that supports multi-node synchronization and room-based message routin

Scrapy is a comprehensive framework designed for automated web data extraction and large-scale crawling. It operates on an asynchronous, event-driven engine that manages non-blocking network requests and data processing tasks, allowing for the efficient retrieval of structured information from web documents using path-based selectors. The system distinguishes itself through a highly modular architecture that supports complex data collection workflows. Users can implement custom middleware and signal handlers to intercept and modify request flows, while a priority-based scheduler manages concu

This project serves as a comprehensive knowledge base and reference for distributed systems engineering and enterprise software architecture. It provides a structured collection of technical resources, design patterns, and methodologies intended to assist in the design, maintenance, and scaling of complex, high-performance software environments. The repository distinguishes itself by offering deep dives into core architectural concepts such as actor-based concurrency, aspect-oriented interception, and inversion-of-control containers. It emphasizes the practical application of distributed syst

MinIO is a software-defined, cloud-native object storage server designed to manage large volumes of unstructured data. It functions as a distributed storage cluster that aggregates multiple independent nodes into a unified, scalable pool, providing a high-performance infrastructure compatible with standard cloud storage protocols and application programming interfaces. The system utilizes a shared-nothing architecture that eliminates central metadata servers, relying instead on a decentralized hash table to map objects across the cluster. Data availability and resilience are maintained throug

etcd is a distributed key-value store and configuration store designed to maintain a consistent set of data across a cluster of nodes. It functions as a reliable registry for storing and synchronizing critical settings and metadata used by distributed applications. The system implements the Raft consensus algorithm to ensure data consistency and leader election across servers. To protect data transfers and verify node identities, it utilizes a network security layer based on mutual TLS and client certificates. Its capabilities cover distributed configuration management, cluster state synchro

etcd is a distributed, strongly consistent key-value store designed to provide reliable storage for critical system metadata and coordination primitives. It functions as a distributed consensus engine, utilizing a replicated log and leader-based state machine to ensure that all nodes in a cluster maintain a synchronized view of data. By providing atomic operations and linearizable reads and writes, it serves as a foundational component for distributed systems requiring high availability and fault tolerance. The system distinguishes itself through its multi-version concurrency control, which e

Cobalt is a cross-platform web application designed as a distributed service platform for managing media content downloading. It functions as a full-stack monorepo that integrates a backend API with a responsive frontend, providing a unified interface for users to fetch and save media files from various online platforms. The project utilizes a modular architecture where backend services, frontend interfaces, and shared logic are organized into decoupled packages within a single repository. This monorepo structure employs centralized workspace orchestration to manage dependencies and cross-pac

Rustfs is a distributed object storage system designed for high availability and horizontal scalability. It functions as a cluster-based platform that manages data across multiple nodes, providing a self-hosted infrastructure for large-scale storage requirements. The system is built to be container-native, utilizing an operator to automate deployment and management within orchestrated environments. It provides compatibility with standard object storage protocols, allowing existing applications and tools to interact with the storage layer through a translation interface. To ensure long-term re

This project is a human resources management system built using Spring Boot and Vue. It serves as a platform for managing employee records, professional titles, and organizational hierarchies. The system features a role-based access control framework that maps users to specific roles and resources to secure API endpoints and user interface elements. It includes a real-time communication hub utilizing WebSockets for internal corporate chat and system notifications, as well as a dedicated manager for defining and modifying nested organizational department structures. Additional capabilities co

System Design Academy is an educational platform and study guide focused on software architecture and distributed systems engineering. It provides a structured collection of technical playbooks, design patterns, and deep dives intended to assist engineers in mastering complex system design and preparing for technical interviews. The platform functions as a content management system that utilizes version-controlled markdown files to maintain and distribute engineering knowledge. By employing a static site generator, the project pre-renders all educational materials into plain HTML files to ens

Crawlee is a web scraping framework designed for building scalable, reliable, and distributed data extraction pipelines. It provides a unified interface for managing headless browser automation and lightweight HTTP requests, allowing developers to handle complex web navigation, dynamic content rendering, and large-scale data collection within a single, modular architecture. The project distinguishes itself through its resource-aware concurrency controller, which dynamically scales task execution based on real-time CPU and memory usage to prevent host machine exhaustion. It also features a rob

IPFS is a peer-to-peer hypermedia protocol and content-addressed storage system that identifies data by cryptographic hashes rather than network locations. It enables the creation of a decentralized web by organizing files and directories as directed acyclic graphs of linked content identifiers. The project differentiates itself through the use of a distributed hash table for locating peers and a system of signed records to map human-readable names to changing content. It also provides HTTP gateways that translate standard web requests into peer-to-peer queries, allowing decentralized data to

This project serves as a comprehensive technical reference for the architecture and design of data-intensive applications. It provides a structured analysis of the fundamental principles required to build reliable, scalable, and maintainable software systems, covering the core trade-offs inherent in modern data infrastructure. The repository explores the mechanics of distributed data management, including strategies for replication, partitioning, and achieving consensus across multiple nodes. It details the design of storage engines, indexing techniques, and transaction management models, whi

RocketMQ is a cloud-native distributed messaging platform and streaming engine. It functions as a distributed transactional queue that ensures atomicity between local transactions and message delivery, and serves as an MQTT IoT message broker to bridge lightweight device traffic into high-performance data streams. The system is distinguished by a Kubernetes-native architecture that decouples compute from storage to allow independent scaling of traffic and data retention. It utilizes a tiered storage model to offload older data to remote storage and employs quorum-based replication and automat

This project is a deep learning framework designed for constructing, training, and deploying neural networks across diverse hardware environments. It functions as a high-performance tensor computation library that provides both imperative and symbolic programming interfaces, allowing developers to balance flexible, step-by-step model building with the efficiency of compiled computation graphs. The framework distinguishes itself through a hybrid execution engine that integrates declarative graph compilation with imperative runtime logic. It supports scalable, distributed training across multip