Awesome GitHub RepositoriesDistributed Systems

SurrealDB is a multi-model database engine designed to store and query document, graph, relational, and vector data within a single ACID-compliant platform. It functions as an AI-native data store, integrating vector search, graph traversal, and machine learning model execution directly into its query layer. By providing a unified declarative query language, the platform eliminates the need for external middleware to synchronize data across different storage models. The platform distinguishes itself through its ability to manage agent memory and complex workflows natively. It allows developer

NSQ is a distributed, brokerless messaging platform designed for high-throughput, fault-tolerant communication. By utilizing a decentralized topology, it eliminates single points of failure and allows for horizontal scaling across clusters. The system organizes message streams into topics and channels, effectively decoupling producers from consumers to support both streaming and job-oriented workloads. The platform distinguishes itself through a lookup-service-based discovery mechanism that enables clients to dynamically locate producers at runtime without requiring centralized coordination.

Typesense is a distributed search engine designed to provide sub-millisecond query latency across massive datasets. It functions as both a high-performance indexing and retrieval engine and a comprehensive search experience platform, offering built-in typo tolerance and tools for managing relevance through synonym configuration, result curation, and complex filtering. The platform distinguishes itself by utilizing in-memory indexing to maintain high-throughput data retrieval and integrating vector database capabilities to support semantic similarity searches. It ensures data consistency and h

This project serves as a comprehensive technical reference for the architecture and design of data-intensive applications. It provides a structured analysis of the fundamental principles required to build reliable, scalable, and maintainable software systems, covering the core trade-offs inherent in modern data infrastructure. The repository explores the mechanics of distributed data management, including strategies for replication, partitioning, and achieving consensus across multiple nodes. It details the design of storage engines, indexing techniques, and transaction management models, whi

RocketMQ is a distributed messaging and streaming platform designed for building event-driven applications. It serves as middleware to decouple services using publish-subscribe and request-reply patterns, and functions as a transactional messaging system that ensures atomicity by linking message delivery to local transaction outcomes. The platform includes specialized capabilities as a Kubernetes-native message broker for container orchestration environments and an MQTT broker for ingesting event data from mobile applications and hardware terminals. The system covers high-throughput data str

Vector is a high-performance observability data pipeline designed to collect, transform, and route logs, metrics, and traces across distributed infrastructure. It functions as a modular engine that decouples data ingestion from processing and transmission, utilizing a component-based architecture to connect diverse sources to multiple destinations. The project distinguishes itself through a focus on reliability and flow control. It implements backpressure-aware data movement to prevent data loss during traffic spikes and utilizes disk-backed event buffering to ensure durability during network

Scala is a statically typed programming language and compiler that combines object-oriented and functional programming paradigms. It serves as a cross-platform runtime language capable of targeting the Java Virtual Machine and JavaScript to share logic between backend servers and web frontends. The project provides a functional programming framework with immutable data structures and higher-order functions to build reliable concurrent and distributed applications. It distinguishes itself through deep interoperability with Java and JavaScript ecosystems and the ability to transform code into n

dtm is a distributed transaction framework and polyglot transaction coordinator designed to maintain data consistency across microservices. It functions as a Saga orchestration engine and a two-phase message coordinator, ensuring that multi-service operations either succeed completely or roll back to a consistent state. The project distinguishes itself by supporting multiple consistency patterns, including Saga, TCC, XA, and outbox patterns, allowing users to select the appropriate model for their specific application requirements. It provides a polyglot integration layer via HTTP and gRPC, e

CRMEB is a comprehensive e-commerce platform built on ThinkPHP 6, designed as a headless system that delivers standardized APIs to various frontend clients. It provides a unified backend to synchronize product catalogs, orders, and customer data across web browsers, mobile applications, and mini-programs. The platform supports diverse commerce models, including multi-vendor marketplaces where independent merchants manage their own stores, centralized chain store networks, and social commerce frameworks featuring affiliate distribution and community group buying. It also integrates specialized

This project is a comprehensive knowledge base and study resource designed for mastering technical interviews. It provides structured guides, roadmaps, and curricula focused on data structures, algorithms, system design, and frontend engineering to help candidates prepare for software engineering screenings. The repository distinguishes itself by offering a holistic approach to professional advancement. Beyond technical drills, it includes a career development handbook covering resume optimization, salary benchmarking, and strategic negotiation coaching. It also provides detailed methodologie

This project is a self-hosted email verification system and API designed to validate email existence and clean mailing lists on private infrastructure. It functions as a deliverability tool that confirms if email addresses are reachable by communicating with mail servers via the SMTP protocol without sending actual messages. The system is distinguished by its high-volume SMTP infrastructure, which utilizes a stateless worker architecture and message queue task distribution to scale validation tasks. It includes an SMTP proxy gateway that routes requests through SOCKS5 proxies to mask server i

This project is a comprehensive educational resource and curriculum focused on site reliability engineering, distributed systems, and infrastructure operations. It provides technical guides, a systems engineering course, and instructional manuals designed to teach the principles of managing large-scale computing environments. The curriculum covers high-level architectural design for scalability and resilience, including fault-tolerant infrastructure, high-availability patterns, and microservices decomposition. It emphasizes the practical application of site reliability engineering through the

Concourse is a container-based continuous integration and delivery platform that functions as a distributed build system. It operates as a declarative pipeline orchestrator, using a central controller and multiple worker nodes to execute concurrent tasks within isolated containers. The system distinguishes itself by executing every build step in a separate container to ensure environment consistency and by defining software delivery sequences through portable, versionable configuration files. It provides a web-based pipeline visualizer to display the real-time status and progress of automated

Swarms is a multi-agent orchestration framework and autonomous agent toolkit designed to coordinate large language model agents. It serves as a workflow engine for managing agent relationships, providing the infrastructure to build autonomous agents with integrated memory, tool-calling capabilities, and reasoning loops. The framework is distinguished by its multi-agent consensus systems, which utilize voting, adversarial debates, and judge agents to synthesize high-quality responses. It supports a variety of collaboration patterns, including director-worker hierarchies, expert synthesis, and

Faust is a Python library for building distributed stream processing applications that integrate with Kafka. It functions as an asynchronous stream processor designed to handle high-throughput event streams and real-time data analysis using asynchronous functions. The system operates as a distributed stream processor and state store, utilizing sharding and partitioned topics to scale processing workloads horizontally across multiple worker nodes. It maintains state through a replicated key-value storage system backed by local databases to ensure high availability and fast recovery. The frame

Kill Bill is a subscription billing platform and usage-based billing engine designed to manage recurring invoicing and automated payment collection. It functions as a multi-tenant billing infrastructure, providing isolated environments for different organizational entities through a dedicated API. The system is distinguished by a plugin-based extension framework that allows for the integration of third-party payment gateways and custom business logic. It includes a payment gateway orchestrator to handle transactions and refunds, as well as a revenue recognition system to allocate contract rev

yii2_fecshop هو منصة تجارة إلكترونية B2C شاملة ونظام إدارة محتوى (CMS) مبني على إطار عمل Yii2 PHP. يعمل كـ API تجارة بدون رأس وحل واجهة متجر دولية، ويدعم كلاً من المتاجر ذات البائع الواحد والأسواق متعددة البائعين حيث يمكن للتجار المستقلين إدارة منتجاتهم وطلباتهم الخاصة. يتميز النظام ببنية نقطة دخول متعددة تفصل الواجهات الإدارية، والويب، والجوال، و API مع مشاركة طبقة خدمة مشتركة. يوفر قابلية توسع عالية من خلال استبدال الخدمة القائم على التكوين، والامتدادات الوظيفية القائمة على الأحداث، ونظام قوالب ذي أولوية للتخصيص المرئي. تغطي المنصة مجموعة واسعة من قدرات التجارة، بما في ذلك التسعير متعدد العملات، وتكامل بوابة الدفع الدولية، وإدارة الشحن العالمي، وتوطين المحتوى متعدد اللغات. كما يتضمن أدوات لإدارة الكتالوج والمحتوى، وفهرسة البحث المؤتمتة مع تحسين SEO، والقدرة على مزامنة البيانات مع أنظمة ERP الخارجية. تشمل الميزات التشغيلية الإضافية التحكم في الوصول القائم على الأدوار، وتصفية IP الإقليمية، وأتمتة مهام الخلفية لصيانة المتجر، ونظام تتبع توزيع متعدد المستويات للحوافز على المبيعات.

Jocko عبارة عن منصة بث أحداث سحابية أصلية وسجل التزامات (Commit log) موزع تم تنفيذه بلغة Go. يعمل كوسيط رسائل موزع يضمن متانة البيانات وتوافرها العالي من خلال تكرار تسلسلات السجلات عبر مجموعة (Cluster). تم تصميم النظام كبث أحداث بدون Zookeeper، مستخدمًا تنسيق إجماع مدمج لإدارة حالة المجموعة وانتخاب القائد دون الحاجة إلى خدمات تنسيق خارجية. ينفذ بروتوكول Kafka، مما يسمح له بالتواصل مع عملاء وأدوات النظام البيئي الحالي. توفر المنصة قدرات لإدارة السجلات الموزعة، بما في ذلك استخدام سجل التزامات للإلحاق فقط (Append-only) لضمان استمرارية البيانات. كما تتضمن آليات تلقائية للاحتفاظ بالبيانات تقوم بمسح السجلات القديمة بناءً على حدود زمنية أو عتبات مساحة القرص. يتم توزيع المشروع كملف تشغيل ثنائي واحد.

هذا المشروع عبارة عن مكتبة مرجعية منسقة لأنماط الخوارزميات، وتطبيقات هياكل البيانات، وملاحظات تصميم النظام. يعمل كمجموعة مشاكل خوارزمية بلغة Java ودليل برمجة تنافسي، ويوفر مجموعة من الحلول لتحديات البرمجة من منصات مثل LeetCode وLintCode. تتميز المكتبة بمجموعتها الشاملة من تطبيقات Java لهياكل البيانات المتقدمة والاستراتيجيات الخوارزمية. تتضمن مراجع مفصلة لحل المشكلات المعقدة مع تحليل مصاحب لتعقيد الوقت والمساحة. يغطي المشروع مساحة واسعة من أساسيات علوم الكمبيوتر، بما في ذلك تصميم الخوارزميات، وتطبيق هياكل البيانات، وتصميم النظام. تمتد محتوياته إلى نظرية الرسم البياني، والبرمجة الديناميكية، والبحث والتحسين، وتقنيات معالجة البيانات الخطية. كما يتضمن ملاحظات حول قابلية توسيع البنية التحتية، والتخزين المؤقت للأداء، وأنماط هندسة البرمجيات.

Lilishop هي منصة تجارة إلكترونية B2B2C متعددة البائعين مصممة كسوق حيث يمكن للتجار الخارجيين إدارة واجهات متاجر مستقلة داخل نظام بيئي موحد. تم بناؤها على بنية الخدمات المصغرة Spring Cloud وواجهة خلفية Spring Boot، باستخدام Vue و Uniapp لتقديم واجهات مستخدم متجاوبة عبر منصات الويب والجوال والبرامج المصغرة. تتميز المنصة بأدوات أتمتة تسويقية متكاملة، بما في ذلك الشراء الجماعي، وحملات المساومة، وشبكة توزيع تابعة قائمة على العمولات. كما تتميز بنظام تسوية مالية شامل يتعامل مع معالجة الدفع متعدد القنوات عبر Alipay و WeChat Pay، إلى جانب توزيع العمولات المؤتمتة وفواتير التجار. يغطي النظام مجموعة واسعة من عمليات التجزئة، بما في ذلك إدارة الكتالوج الهرمي، وتوزيع القسائم المؤتمت، وتتبع دورة حياة الطلب متعدد القنوات. يتضمن أدوات مخصصة للتحكيم في النزاعات، وتدقيق خدمات ما بعد البيع، وفهرسة بحث المنتجات متعددة الأوجه. تتم إدارة النشر من خلال نصوص Docker و Compose لتسهيل تنسيق خدمات API الموزعة والمستهلكين.