Awesome GitHub RepositoriesDistributed Systems

SurrealDB is a multi-model database engine designed to store and query document, graph, relational, and vector data within a single ACID-compliant platform. It functions as an AI-native data store, integrating vector search, graph traversal, and machine learning model execution directly into its query layer. By providing a unified declarative query language, the platform eliminates the need for external middleware to synchronize data across different storage models. The platform distinguishes itself through its ability to manage agent memory and complex workflows natively. It allows developer

NSQ is a distributed, brokerless messaging platform designed for high-throughput, fault-tolerant communication. By utilizing a decentralized topology, it eliminates single points of failure and allows for horizontal scaling across clusters. The system organizes message streams into topics and channels, effectively decoupling producers from consumers to support both streaming and job-oriented workloads. The platform distinguishes itself through a lookup-service-based discovery mechanism that enables clients to dynamically locate producers at runtime without requiring centralized coordination.

Typesense is a distributed search engine designed to provide sub-millisecond query latency across massive datasets. It functions as both a high-performance indexing and retrieval engine and a comprehensive search experience platform, offering built-in typo tolerance and tools for managing relevance through synonym configuration, result curation, and complex filtering. The platform distinguishes itself by utilizing in-memory indexing to maintain high-throughput data retrieval and integrating vector database capabilities to support semantic similarity searches. It ensures data consistency and h

This project serves as a comprehensive technical reference for the architecture and design of data-intensive applications. It provides a structured analysis of the fundamental principles required to build reliable, scalable, and maintainable software systems, covering the core trade-offs inherent in modern data infrastructure. The repository explores the mechanics of distributed data management, including strategies for replication, partitioning, and achieving consensus across multiple nodes. It details the design of storage engines, indexing techniques, and transaction management models, whi

RocketMQ is a distributed messaging and streaming platform designed for building event-driven applications. It serves as middleware to decouple services using publish-subscribe and request-reply patterns, and functions as a transactional messaging system that ensures atomicity by linking message delivery to local transaction outcomes. The platform includes specialized capabilities as a Kubernetes-native message broker for container orchestration environments and an MQTT broker for ingesting event data from mobile applications and hardware terminals. The system covers high-throughput data str

Vector is a high-performance observability data pipeline designed to collect, transform, and route logs, metrics, and traces across distributed infrastructure. It functions as a modular engine that decouples data ingestion from processing and transmission, utilizing a component-based architecture to connect diverse sources to multiple destinations. The project distinguishes itself through a focus on reliability and flow control. It implements backpressure-aware data movement to prevent data loss during traffic spikes and utilizes disk-backed event buffering to ensure durability during network

Scala is a statically typed programming language and compiler that combines object-oriented and functional programming paradigms. It serves as a cross-platform runtime language capable of targeting the Java Virtual Machine and JavaScript to share logic between backend servers and web frontends. The project provides a functional programming framework with immutable data structures and higher-order functions to build reliable concurrent and distributed applications. It distinguishes itself through deep interoperability with Java and JavaScript ecosystems and the ability to transform code into n

dtm is a distributed transaction framework and polyglot transaction coordinator designed to maintain data consistency across microservices. It functions as a Saga orchestration engine and a two-phase message coordinator, ensuring that multi-service operations either succeed completely or roll back to a consistent state. The project distinguishes itself by supporting multiple consistency patterns, including Saga, TCC, XA, and outbox patterns, allowing users to select the appropriate model for their specific application requirements. It provides a polyglot integration layer via HTTP and gRPC, e

CRMEB is a comprehensive e-commerce platform built on ThinkPHP 6, designed as a headless system that delivers standardized APIs to various frontend clients. It provides a unified backend to synchronize product catalogs, orders, and customer data across web browsers, mobile applications, and mini-programs. The platform supports diverse commerce models, including multi-vendor marketplaces where independent merchants manage their own stores, centralized chain store networks, and social commerce frameworks featuring affiliate distribution and community group buying. It also integrates specialized

This project is a comprehensive knowledge base and study resource designed for mastering technical interviews. It provides structured guides, roadmaps, and curricula focused on data structures, algorithms, system design, and frontend engineering to help candidates prepare for software engineering screenings. The repository distinguishes itself by offering a holistic approach to professional advancement. Beyond technical drills, it includes a career development handbook covering resume optimization, salary benchmarking, and strategic negotiation coaching. It also provides detailed methodologie

This project is a self-hosted email verification system and API designed to validate email existence and clean mailing lists on private infrastructure. It functions as a deliverability tool that confirms if email addresses are reachable by communicating with mail servers via the SMTP protocol without sending actual messages. The system is distinguished by its high-volume SMTP infrastructure, which utilizes a stateless worker architecture and message queue task distribution to scale validation tasks. It includes an SMTP proxy gateway that routes requests through SOCKS5 proxies to mask server i

This project is a comprehensive educational resource and curriculum focused on site reliability engineering, distributed systems, and infrastructure operations. It provides technical guides, a systems engineering course, and instructional manuals designed to teach the principles of managing large-scale computing environments. The curriculum covers high-level architectural design for scalability and resilience, including fault-tolerant infrastructure, high-availability patterns, and microservices decomposition. It emphasizes the practical application of site reliability engineering through the

Concourse is a container-based continuous integration and delivery platform that functions as a distributed build system. It operates as a declarative pipeline orchestrator, using a central controller and multiple worker nodes to execute concurrent tasks within isolated containers. The system distinguishes itself by executing every build step in a separate container to ensure environment consistency and by defining software delivery sequences through portable, versionable configuration files. It provides a web-based pipeline visualizer to display the real-time status and progress of automated

Swarms 是一个多代理编排框架和自主代理工具包，旨在协调大语言模型代理。它作为一个用于管理代理关系的工作流引擎，提供了构建具有集成内存、工具调用能力和推理循环的自主代理的基础设施。 该框架的特色在于其多代理共识系统，利用投票、对抗性辩论和裁判代理来合成高质量的响应。它支持多种协作模式，包括导演-工作者层次结构、专家合成以及基于自然语言描述的自动化群体架构生成。 该系统涵盖了广泛的运营功能，包括通过领域特定语言进行基于图和顺序的工作流编排、针对不同模型提供商的统一接口，以及与 Model Context Protocol 的集成以实现动态工具发现。它还包括对检索增强生成、状态持久内存以及将代理功能公开为 Web 服务的能力的支持。 该项目提供用于代理管理的命令行界面，并支持通过 YAML 和模块化 markdown 技能文件进行配置。

Faust is a Python library for building distributed stream processing applications that integrate with Kafka. It functions as an asynchronous stream processor designed to handle high-throughput event streams and real-time data analysis using asynchronous functions. The system operates as a distributed stream processor and state store, utilizing sharding and partitioned topics to scale processing workloads horizontally across multiple worker nodes. It maintains state through a replicated key-value storage system backed by local databases to ensure high availability and fast recovery. The frame

Kill Bill is a subscription billing platform and usage-based billing engine designed to manage recurring invoicing and automated payment collection. It functions as a multi-tenant billing infrastructure, providing isolated environments for different organizational entities through a dedicated API. The system is distinguished by a plugin-based extension framework that allows for the integration of third-party payment gateways and custom business logic. It includes a payment gateway orchestrator to handle transactions and refunds, as well as a revenue recognition system to allocate contract rev

yii2_fecshop 是一个基于 Yii2 PHP 框架构建的综合性 B2C 电商平台和 CMS。它作为一个无头（headless）电商 API 和国际化店面解决方案，支持单商户商店和多商户市场，独立商家可以在其中管理自己的产品和订单。 该系统以多入口架构为特色，将管理、Web、移动和 API 界面分离，同时共享一个通用的服务层。它通过配置驱动的服务替换、事件驱动的功能扩展以及用于视觉自定义的优先级模板系统提供了极高的可扩展性。 该平台涵盖了一套广泛的电商功能，包括多货币定价、国际支付网关集成、全球物流管理和多语言内容本地化。它还包括目录和内容管理工具、具有 SEO 优化的自动化搜索索引，以及与外部 ERP 系统同步数据的能力。 其他运维功能包括基于角色的访问控制、区域 IP 过滤、用于商店维护的后台任务自动化，以及用于销售激励的分级多层分销跟踪系统。

Jocko 是一个云原生的事件流平台和分布式提交日志，使用 Go 语言实现。它作为一个分布式消息代理，通过在集群中复制记录序列来确保数据持久性和高可用性。 该系统被设计为无需 Zookeeper 的事件流平台，利用内置的共识协调来管理集群状态和领导者选举，无需外部协调服务。它实现了 Kafka 有线协议，允许其与现有的生态系统客户端和工具进行通信。 该平台提供了分布式日志管理能力，包括使用仅追加（append-only）的提交日志进行数据持久化。它还包含自动数据保留机制，可根据时间限制或磁盘空间阈值清除旧记录。 该项目以单一二进制文件的形式发布。

这是一个精选的算法模式、数据结构实现和系统设计笔记参考库。它作为 Java 算法问题集和竞赛编程指南，为 LeetCode 和 LintCode 等平台上的编码挑战提供了解决方案合集。 该库以其针对高级数据结构和算法策略的全面 Java 实现而著称。它包括用于解决复杂问题的详细参考资料，并附带时间和空间复杂度分析。 该项目涵盖了广泛的计算机科学基础知识，包括算法设计、数据结构实现和系统设计。其内容涵盖图论、动态规划、搜索与优化以及线性数据处理技术。它还包括关于基础设施可扩展性、性能缓存和软件架构模式的笔记。

Lilishop 是一个多商户 B2B2C 电子商务平台，设计为一个市场，第三方商家可以在统一的生态系统中管理独立的店面。它基于 Spring Cloud 微服务架构和 Spring Boot 后端构建，利用 Vue 和 Uniapp 在 Web、移动和微信小程序平台上提供响应式用户界面。 该平台通过集成的营销自动化工具（包括团购、砍价活动和基于佣金的联盟分销网络）实现差异化。它还具有一个全面的财务结算系统，通过支付宝和微信支付处理多渠道支付，并辅以自动佣金分配和商家账单。 该系统涵盖了广泛的零售业务，包括分层目录管理、自动优惠券分发和全渠道订单生命周期跟踪。它包括用于争议仲裁、售后服务审计和分面产品搜索索引的专用工具。 部署通过 Docker 和 Compose 脚本进行管理，以促进其分布式 API 服务和消费者的编排。