A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow
Die Hauptfunktionen von mara/data-integration sind: Service Programming, Workflow Frameworks.
Open-Source-Alternativen zu mara/data-integration sind unter anderem: spotify/luigi — Luigi is a Python framework designed for building and managing complex batch data pipelines. It functions as a… agnostiqhq/covalent — Pythonic tool for orchestrating machine-learning/high performance/quantum-computing workflows in heterogeneous compute… aiidateam/aiida-core — The official repository for the AiiDA code. airbnb/airflow — Airflow is a workflow orchestration platform for authoring, scheduling, and monitoring complex data pipelines as code… alpha-unito/streamflow. adaptivegenome/openge.
Luigi is a Python framework designed for building and managing complex batch data pipelines. It functions as a workflow orchestration engine that organizes tasks into directed acyclic graphs, ensuring that jobs execute in the correct logical order based on their dependencies. By utilizing a centralized scheduler, the system coordinates task execution across distributed environments, tracks global workflow state, and prevents redundant processing by verifying the existence of output targets before triggering any work. The project distinguishes itself through a robust state-tracking mechanism t
Pythonic tool for orchestrating machine-learning/high performance/quantum-computing workflows in heterogeneous compute environments.
