Awesome GitHub RepositoriesCloud Workflow Runners

Prefect is a workflow orchestration platform designed to define, schedule, and monitor complex data pipelines as Python code. It functions as a container-native engine that wraps individual tasks in isolated environments, ensuring consistent dependencies and resource allocation across diverse infrastructure. By utilizing a state-machine-based orchestration model, the system tracks execution progress through discrete transitions and persistent event logs to maintain reliable and observable task processing. The platform distinguishes itself through a decoupled worker-API architecture, which sep

Osmedeus is a security workflow orchestration engine that coordinates AI agents, shell commands, and scanning tools through declarative YAML pipelines. It functions as a distributed security scanner, a declarative workflow automator, and an AI agent framework for security, enabling automated multi-step security analysis with conditional branching, parallel execution, and distributed workers. The engine distinguishes itself through a hybrid runner model that executes workflow steps on the local host, inside Docker containers, or over SSH to remote machines, selected per step or module. It supp

CML 是一个用于训练和评估机器学习模型的管道自动化工具，作为机器学习的 CI/CD 系统运行。它作为一个云计算编排器和基于 Git 的工作流管理器，通过分支管理、自动提交和集成报告来自动化模型训练周期。 该项目通过配置临时云实例或 Kubernetes 节点来提供计算密集型任务所需的专用硬件，从而脱颖而出。它还管理远程计算运行器，允许连接自托管 GPU 集群或本地机器来执行容器化机器学习工作流。 该系统涵盖了广泛的功能，包括 ML 实验跟踪（性能指标和可视化直接发布到版本控制 Pull Request 中）。它处理从初始数据导入和版本控制到生成格式化工作流报告和外部可视化链接的 ML 管道自动化。 该工具通过基于 SSH 的远程调试和恢复中断作业的能力，为基础设施管理提供了额外的实用性。