Pythonic tool for orchestrating machine-learning/high performance/quantum-computing workflows in heterogeneous compute environments.
Airflow is a workflow orchestration platform for authoring, scheduling, and monitoring complex data pipelines as code using Python. It employs a DAG-based task scheduler to manage execution timing and dependencies via directed acyclic graphs, utilizing a distributed task execution engine to run workloads across a cluster of worker nodes. The platform provides a data pipeline monitor for tracking the health and execution history of programmatic workflows. This includes a web interface for workflow progress visualization and health monitoring to identify and troubleshoot pipeline failures. The
The main features of adaptivegenome/openge are: Workflow Frameworks.
Open-source alternatives to adaptivegenome/openge include: agnostiqhq/covalent — Pythonic tool for orchestrating machine-learning/high performance/quantum-computing workflows in heterogeneous compute… aiidateam/aiida-core — The official repository for the AiiDA code. airbnb/airflow — Airflow is a workflow orchestration platform for authoring, scheduling, and monitoring complex data pipelines as code… alpha-unito/streamflow. argonne-lcf/balsam — High throughput workflows and automation for HPC. 4dn-dcic/tibanna.
