Awesome GitHub RepositoriesTask Schedulers

هذا المشروع عبارة عن دليل منسق من قبل المجتمع للبرمجيات مفتوحة المصدر المصممة للنشر في بيئات الخوادم الخاصة والمختبرات المنزلية. يعمل كمورد شامل لاكتشاف بدائل مستقلة ذاتية الاستضافة لخدمات السحابة السائدة، مما يمكن المستخدمين من الحفاظ على ملكية كاملة للبيانات والتحكم في بنيتهم التحتية الرقمية. يتم تنظيم الدليل من خلال تصنيف هرمي ينظم مجموعة واسعة من التطبيقات في فئات منطقية، تتراوح من إدارة الوسائط وتحليل البيانات إلى التواصل الخاص وأدوات إنتاجية الفريق. يتميز بعملية مراجعة أقران تعاونية، حيث يقوم أعضاء المجتمع بالتحقق من جودة وملاءمة كل طلب لضمان بقاء الدليل دقيقاً وموثوقاً. يغطي المشروع نطاقاً واسعاً من القدرات، بما في ذلك أتمتة البنية التحتية، ونشر الخدمات القائمة على الحاويات، وإدارة التكوين التصريحي. تساعد هذه الأدوات المستخدمين في الحفاظ على بيئات خادم قابلة للتكرار وإدارة تبعيات الخدمات المعقدة عبر الأجهزة الخاصة. يتم الحفاظ على الدليل كمستودع خاضع للتحكم في الإصدار، مما يضمن تتبع جميع التحديثات والتغييرات التي يقودها المجتمع وأنها شفافة.

Airflow is a platform for programmatically authoring, scheduling, and monitoring complex data pipelines. It functions as a workflow automation engine that manages the lifecycle of recurring business processes by executing code-defined task dependencies. By representing workflows as directed acyclic graphs, the system ensures that task execution order and data flow are explicitly defined and reliably maintained across distributed computing environments. The platform distinguishes itself through a highly modular, provider-based architecture that decouples core orchestration logic from external

Payload is a headless content management system and application framework that uses a code-first approach to define data schemas and administrative interfaces. By utilizing a centralized, type-safe configuration object, it automatically generates database schemas, API endpoints, and a fully customizable admin panel. The system is built on a database-agnostic architecture, allowing it to interface with various storage engines while providing a unified, type-safe API for server-side operations, REST, and GraphQL. What distinguishes Payload is its deep extensibility and developer-centric design.

Ray is a distributed computing framework designed to scale Python and Java applications across clusters by abstracting task scheduling and resource management. It functions as a resource-aware execution engine that manages task dependencies, placement, and fault tolerance across networked compute nodes. At its core, the system provides a stateful actor model, allowing developers to define classes that run in dedicated processes to maintain and mutate internal state across remote method calls. The framework distinguishes itself through a robust cross-language interoperability layer, enabling f

Ente is a privacy-focused platform for end-to-end encrypted storage and two-factor authentication management. It functions as a zero-knowledge identity provider, ensuring that all cryptographic operations, key derivation, and data encryption occur locally on the user's device. By maintaining this architecture, the service provider remains unable to access or decrypt any stored personal information or authentication credentials. The platform distinguishes itself through a combination of on-device intelligence and resilient data distribution. It utilizes a local machine learning engine to perfo

Chaos Monkey is a chaos engineering tool designed to verify the resilience of distributed systems by intentionally terminating production instances. It functions as a fault injection service that identifies weaknesses in cloud-based architectures by simulating real-world hardware and software outages. The platform operates through a centralized orchestration engine that executes periodic disruption cycles based on predefined configuration rules. It employs a rule-based selection process that evaluates instance metadata against safety constraints to ensure that only eligible targets are disrup

Moto is a cloud service mockery framework and API mock server that simulates AWS infrastructure locally. It allows developers to test cloud-dependent code and verify infrastructure-as-code templates without deploying real resources or incurring costs. The project functions as an SDK interceptor that can patch existing service clients to redirect requests to a local mock environment. It can also be run as a standalone HTTP server, enabling any programming language to interact with the simulated endpoints. The framework covers a vast array of simulated capabilities, including data storage, com

Elsa Core is a workflow engine framework designed for defining, executing, and managing long-running business processes. It functions as a distributed workflow orchestrator and event-driven trigger system, capable of operating as a multi-tenant platform with secure data isolation. The project distinguishes itself through a flexible approach to workflow definitions, supporting a visual drag-and-drop designer, programmatic C# definitions, and portable JSON specifications. It provides a highly extensible architecture allowing for the development of custom activities and the use of a dynamic expr

Flower is a federated learning framework and distributed machine learning orchestrator designed to train models across decentralized devices. It functions as a privacy-preserving toolkit that enables model training and data analysis on local hardware, ensuring raw data remains on the device while contributing to a synchronized global model. The system employs an agnostic wrapper and integrator to connect diverse machine learning libraries, allowing different frameworks to operate within the same training loop. It uses a remote procedure call orchestrator to manage the exchange of model weight

Quartz is a Java job scheduling framework and task execution engine designed to manage and execute scheduled tasks within application environments. It functions as an enterprise job scheduler that persists job state and execution history to maintain reliability across system restarts. The system distinguishes itself through a decoupled architecture that separates the definition of a job's action from the trigger logic that determines when it runs. It supports distributed task coordination across multiple server nodes to provide high availability and load balancing. The framework covers a bro

A command line toolkit to generate maps, point clouds, 3D models and DEMs from drone, balloon or kite images. 📷

Reth is a modular, high-performance Ethereum execution layer client written in Rust. It serves as a full Ethereum node that syncs, validates, and serves blockchain data, functioning as an archive node implementation, a high-throughput RPC node server, and a snapshot sync tool. The project is built around a modular component architecture that allows assembling custom node behavior by swapping independent Rust crates for consensus, execution, mempool, and networking. The client distinguishes itself through a staged sync pipeline that downloads headers and bodies online before processing the res

SparkInternals is a technical reference and architecture guide detailing the internal design and implementation of the Apache Spark distributed computing engine. It serves as a study of big data engine analysis, focusing on how the system manages cluster execution and the interaction between driver nodes, executors, and workers. The project provides a detailed breakdown of how logical plans are converted into physical execution stages. It specifically analyzes the mechanics of data shuffle operations, memory management, and the coordination of distributed job scheduling. The documentation co

This project provides a comprehensive technical guide and framework for engineering large-scale machine learning systems. It covers the full lifecycle of model development, focusing on the infrastructure and computational principles required to build, train, and serve generative AI models across distributed GPU clusters. The repository distinguishes itself by offering deep-dive tutorials and implementation strategies for complex system challenges. It emphasizes high-performance architectural primitives, such as collective communication orchestration, distributed tensor sharding, and static gr

kcp-go هي مكتبة Go تنفذ بروتوكول KCP لنقل البيانات الموثوق عبر UDP. توفر طبقة نقل قائمة على التدفق وبروتوكول شبكة طلب تكرار تلقائي لضمان التسليم المرتب مع تقليل زمن انتقال الشبكة. يتميز المشروع باستخدام تصحيح الخطأ الأمامي عبر رموز Reed-Solomon لإعادة بناء الحزم المفقودة دون عمليات إعادة إرسال. كما ينفذ نفق UDP آمناً باستخدام تشفيرات الكتلة لتشفير كل من رؤوس الحزم والحمولات. تتضمن المكتبة قدرات للشبكات عالية الإنتاجية من خلال معالجة استدعاء النظام المجمعة وإعادة تدوير الذاكرة. كما تعمل على تحسين الأداء من خلال ضبط الازدحام لتحديد أولويات استعادة البيانات الفورية ونظام جدولة مهام من مرحلتين لتوزيع CPU المتوازي. يتضمن التنفيذ أدوات مراقبة لتتبع مقاييس الشبكة وتتبع أحداث البروتوكول.

tiny-llm is a large language model inference engine and transformer model implementation. It serves as a quantized model runtime and paged key-value cache manager, providing a specialized inference stack optimized for Apple Silicon. The system distinguishes itself through high-throughput execution techniques, including continuous batching and paged attention. It utilizes a paged memory system to eliminate fragmentation during token generation and employs on-the-fly dequantization of compressed weights to reduce the memory footprint during matrix multiplication. The project covers a broad ran

Failsafe is a JVM fault tolerance library and resilience pattern framework. It provides a toolkit for implementing circuit breakers, rate limiters, and other stability patterns within Java Virtual Machine applications to prevent cascading failures in distributed systems. The project is distinguished by its policy-based execution pipeline, which allows for the composition of multiple resilience patterns into a sequential flow. It features a state-machine circuit breaker to manage service recovery and a leaky-bucket rate limiter to control operation frequency. The library covers a broad range

LazyLLM is a multi-agent framework and orchestration engine designed for building complex AI applications. It provides a system for chaining large language models into sequential or parallel pipelines, utilizing a tool registry to convert standard functions into discoverable tools that models can invoke via reasoning. The project features an application deployment kit that enables hosting model workflows as web services with integrated chat interfaces and API gateways. It includes an infrastructure abstraction layer that allows users to switch between bare-metal servers, clusters, and public