Awesome GitHub RepositoriesDynamic Task Graphs

Airflow is a workflow orchestration platform for authoring, scheduling, and monitoring complex data pipelines as code using Python. It employs a DAG-based task scheduler to manage execution timing and dependencies via directed acyclic graphs, utilizing a distributed task execution engine to run workloads across a cluster of worker nodes. The platform provides a data pipeline monitor for tracking the health and execution history of programmatic workflows. This includes a web interface for workflow progress visualization and health monitoring to identify and troubleshoot pipeline failures. The

Conductor is a durable workflow engine designed to orchestrate complex, long-running business processes and autonomous agent loops. It functions as a stateful execution platform that persists the entire history of a process, ensuring that workflows remain reliable and recoverable across infrastructure failures, system restarts, and transient network errors. By managing task lifecycles, worker polling, and state transitions, it provides a centralized coordination layer for distributed systems. The platform distinguishes itself through its specialized support for AI agent orchestration, allowin

Luigi is a Python framework designed for building and managing complex batch data pipelines. It functions as a workflow orchestration engine that organizes tasks into directed acyclic graphs, ensuring that jobs execute in the correct logical order based on their dependencies. By utilizing a centralized scheduler, the system coordinates task execution across distributed environments, tracks global workflow state, and prevents redundant processing by verifying the existence of output targets before triggering any work. The project distinguishes itself through a robust state-tracking mechanism t

Dask ist ein Framework für paralleles Rechnen und ein verteilter Task-Scheduler, der darauf ausgelegt ist, Python-Data-Science-Workflows von einzelnen Maschinen auf große Cluster zu skalieren. Es fungiert als Cluster-Ressourcenmanager, der die Berechnungslogik orchestriert, indem Aufgaben und deren Abhängigkeiten als gerichtete azyklische Graphen dargestellt werden. Diese Architektur ermöglicht es dem System, die Verteilung von Workloads auf verfügbare Hardware zu automatisieren und gleichzeitig komplexe Ausführungsanforderungen zu verwalten. Das Projekt zeichnet sich durch eine Lazy-Evaluation-Engine aus, die Datenoperationen verzögert, bis sie explizit angefordert werden, was eine globale Graphoptimierung und effiziente Ressourcenzuweisung ermöglicht. Es integriert speicherbewusstes Data-Spilling, um Systemabstürze bei der Verarbeitung von Datensätzen zu verhindern, die den verfügbaren Speicher überschreiten, und nutzt Task-Graph-Fusion, um Sequenzen von Operationen in einzelne Ausführungsschritte zu kombinieren, wodurch Scheduling-Overhead und Inter-Node-Kommunikation minimiert werden. Die Plattform bietet eine umfassende Oberfläche für die Datenanalyse im großen Maßstab, einschließlich Unterstützung für verteiltes maschinelles Lernen, Integration in das Hochleistungsrechnen und parallele Datenverarbeitung. Sie bietet umfangreiche Werkzeuge für das Cluster-Lebenszyklusmanagement, Performance-Profiling und die Echtzeitüberwachung der Aufgabenausführung. Benutzer können diese Umgebungen über verschiedene Infrastrukturen hinweg bereitstellen, einschließlich lokaler Hardware, Cloud-Anbietern, containerisierten Systemen und Hochleistungsrechner-Clustern.

This project is a framework for building AI coding agents that automate software development tasks using large language models. It includes a task lifecycle manager that tracks complex development goals through a persistent graph of dependent tasks and a system for multi-agent orchestration to delegate tasks to specialized sub-agents. The framework implements a Model Context Protocol client to discover and execute tools from external servers and provides a remote development bridge to synchronize local command line interfaces with remote containers or desktop environments. The system covers

Cpp-taskflow is a C++ task-parallelism framework and task graph scheduler designed to manage and execute complex dependency graphs of parallel tasks across CPU and GPU hardware. It provides a parallel algorithm library for high-performance implementations of reductions, sorts, pipelines, and iterations. The framework distinguishes itself through its ability to offload heavy computational workloads from a task graph to graphics processors for acceleration. It also includes a task profiling tool and a performance analysis interface for visualizing task execution flow and dependency structures t

Taskflow is a C++ task-parallel framework designed to build high-performance parallel workflows and complex dependency graphs. It provides a programming model that organizes computational work into directed acyclic graphs, enabling developers to manage concurrency, resource scheduling, and task dependencies across multi-core CPUs and GPU accelerators. The framework distinguishes itself through its ability to orchestrate heterogeneous systems, allowing for the integration of hardware-accelerated kernels and memory operations into unified execution pipelines. It supports dynamic runtime subflow

UFO is a multi-device task orchestrator and LLM agent orchestration framework designed to decompose natural language requests into executable task graphs. It functions as a cross-platform UI automation tool capable of performing interactions on Windows and mobile devices while routing tasks to distributed agents based on their hardware and software capabilities. The system is distinguished by its RAG-enhanced agent architecture, which integrates external documentation and previous execution traces to improve decision-making. It employs a hybrid UI detection approach that combines computer vis

This project is a comprehensive collection of educational examples and reference implementations for building vision and language models using PyTorch. It serves as a deep learning tutorial covering the end-to-end process of developing neural networks, from initial architecture definition to final production deployment. The repository provides detailed guides on implementing a wide range of domain-specific models, including convolutional neural networks for object detection and segmentation, as well as transformer and recurrent architectures for natural language processing. It emphasizes gene

Open Source Javascript Gantt

This is a documentation and sample project for learning Unreal Engine's Gameplay Ability System (GAS), focused on building networked gameplay abilities that synchronize correctly in a multiplayer environment. The project provides a step-by-step guide to implementing GAS, covering the core concepts of defining gameplay mechanics through a data-driven attribute system that separates ability logic from character state management. The documentation explores key architectural patterns for multiplayer ability design, including a predicted-action network model that uses client-side prediction and se

Butterfly ist eine interaktive Bibliothek für Graph-Visualisierung und eine SVG-basierte Diagramm-Engine, die zum Aufbau von Node-basierten Workflow-Designern und visuellen Flow-Editoren verwendet wird. Sie bietet ein Framework zum Rendern miteinander verbundener Knoten, Kanten und Gruppen, um technische Architekturen, Geschäftsprozesse und datengesteuerte Workflows zu modellieren. Das Projekt zeichnet sich durch die Fähigkeit aus, benutzerdefinierte UI-Komponenten als Graph-Knoten zu integrieren, sowie durch spezialisierte Unterstützung für die Visualisierung von Datenherkunft (Data Lineage). Es ermöglicht die Abbildung von Beziehungen zwischen Datenfeldern, Tabellen und Geschäftsentitäten, um die Provenienz nachzuverfolgen und den Workflow-Status zu überwachen. Das System deckt ein breites Spektrum an Funktionen ab, darunter automatisierte Knoten-Layout-Algorithmen, hierarchische Gruppennestung und Validierung der Verbindungslogik. Es enthält Navigationsdienstprogramme wie eine Minimap, Zoom- und Pan-Steuerungen sowie Marquee-Auswahl, neben Tools für die Traversierung der Graphentopologie und den Bildexport in Vektor- und Rasterformaten. Die Bibliothek ist in JavaScript implementiert.