awesome-repositories.com
Blog
MCP
awesome-repositories.com

Discover the best open-source repositories with AI-powered search.

ExploreCurated searchesOpen-source alternativesSelf-hosted softwareBlogSitemap
ProjectMCP serverAboutHow we rankPress
LegalPrivacyTerms
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
orchest avatar

orchest/orchestArchived

0
View on GitHub↗

Orchest

Orchest is a data pipeline orchestrator and containerized workflow manager. It provides a platform for designing, scheduling, and executing complex data processing sequences through a combination of a graphical interface and scripting.

The platform distinguishes itself by using containers to manage software dependencies, ensuring consistent execution across different environments. It features a polyglot task scheduler capable of triggering jobs written in multiple programming languages and includes a version control system that tracks historical snapshots of project configurations and code.

The system covers visual workflow design and graph-based dependency mapping, alongside time-triggered task scheduling for recurring or immediate execution. It also supports the deployment of persistent background services that remain active for the duration of a pipeline run.

AI search

Explore more awesome repositories

Describe what you need in plain English — the AI ranks thousands of curated open-source projects by relevance.

Start searching with AI
orchest.readthedocs.io/en/stable
↗

Features

  • Data Pipeline Orchestration - Provides a platform for designing, scheduling, and monitoring complex sequences of data processing tasks.
  • Data Pipeline Orchestrators - Provides a professional platform for automating complex sequences of data processing tasks.
  • Nodal Workflow Designers - Provides a nodal interface for constructing and managing modular data processing pipelines through visual connections.
  • Polyglot Execution Orchestrators - Provides a polyglot execution runtime that manages the sequential execution of code across diverse language runtimes.
  • Scripted Workflow Automations - Implements a scripting language to define the order and flow of data processing tasks, replacing manual sequences.
  • Scripted Pipeline Logic - Uses a domain-specific scripting language to define the logic and order of data processing sequences.
  • Workflow Managers - Acts as a platform that uses containers to manage software dependencies and ensure consistent workflow execution.
  • Containerized - Uses containers to isolate software requirements for each task, preventing version conflicts across different computing environments.
  • Execution Environment Configurations - Defines software dependencies and parameters to ensure workflows execute consistently across different machines.
  • Polyglot Task Processors - Triggers data processing jobs written in multiple programming languages using a polyglot task processor.
  • Polyglot Workflow Orchestration - Provides a managed environment that coordinates business logic and state across multiple different programming language runtimes in a single workflow.
  • Graph-Based Workflow Models - Implements graph-based workflow models to map task dependencies and execution sequences visually.
  • Recurring Task Schedulers - Ships utilities for executing specific workflow subsets immediately or repeatedly at fixed time intervals.
  • Container-Based Isolation - Provides container-based isolation to ensure task-specific software dependencies remain consistent across different hardware architectures.
  • Visual Workflow Orchestrators - Provides a graphical user interface for managing task dependencies and execution order through visual representations.
  • Data Pipeline Flow Designers - Provides a graphical user interface for mapping task dependencies and designing data processing flows visually.
  • Batch Processing Schedulers - Automates and manages the execution of recurring data processing jobs on a scheduled basis.
  • Version Control Snapshots - Tracks historical changes to project configurations and code through a system of versioned immutable snapshots.
  • Time-Based Triggers - Executes workflow subsets based on recurring calendar intervals or immediate manual trigger requests.
  • Persistent Background Services - Launches persistent background services that remain active for the entire duration of a pipeline execution.
  • Configuration Version Control - Integrates version control to track and audit historical changes to project configurations and code.
  • Workflow Orchestration - Visual pipeline editor and workflow orchestrator for Kubernetes.
  • Data Pipelines - No-code builder for data pipelines.
  • Data Orchestration - Visual interface for building and managing data pipelines.
  • GitOps And Continuous Delivery - Tool for building data pipelines with Jupyter notebook support.
  • Workflow Platforms - Integrated development environment for data science workflows.
4,138 stars·261 forks·TypeScript·Apache-2.0·22 views

Star history

Star history chart for orchest/orchestStar history chart for orchest/orchest

How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.

Frequently asked questions

What does orchest/orchest do?

Orchest is a data pipeline orchestrator and containerized workflow manager. It provides a platform for designing, scheduling, and executing complex data processing sequences through a combination of a graphical interface and scripting.

What are the main features of orchest/orchest?

The main features of orchest/orchest are: Data Pipeline Orchestration, Data Pipeline Orchestrators, Nodal Workflow Designers, Polyglot Execution Orchestrators, Scripted Workflow Automations, Scripted Pipeline Logic, Workflow Managers, Containerized.

Which projects share features with orchest/orchest?

Projects with overlapping indexed features include: prefecthq/prefect — Prefect is a workflow orchestration platform designed to define, schedule, and monitor complex data pipelines as… apache/nifi — Apache NiFi is a flow-based programming platform that enables the visual design, monitoring, and management of data… zenml-io/zenml — ZenML is an orchestration platform designed for building, deploying, and monitoring reproducible machine learning… spotify/luigi — Luigi is a Python framework designed for building and managing complex batch data pipelines. It functions as a… flyteorg/flyte — Flyte is a Kubernetes-based machine learning orchestrator and containerized pipeline manager designed for coordinating… ploomber/ploomber — The fastest ⚡️ way to build data pipelines. Develop iteratively, deploy anywhere. ☁️.

Projects sharing features with Orchest

These projects share indexed features with Orchest. Shared tags can include platform or build tooling; verify the primary use case before treating a result as a replacement.
  • prefecthq/prefectPrefectHQ avatar

    PrefectHQ/prefect

    21,640View on GitHub↗

    Prefect is a workflow orchestration platform designed to define, schedule, and monitor complex data pipelines as Python code. It functions as a container-native engine that wraps individual tasks in isolated environments, ensuring consistent dependencies and resource allocation across diverse infrastructure. By utilizing a state-machine-based orchestration model, the system tracks execution progress through discrete transitions and persistent event logs to maintain reliable and observable task processing. The platform distinguishes itself through a decoupled worker-API architecture, which sep

    Pythonautomationdatadata-engineering
    View on GitHub↗21,640
  • apache/nifiapache avatar

    apache/nifi

    5,976View on GitHub↗

    Apache NiFi is a flow-based programming platform that enables the visual design, monitoring, and management of data pipelines. At its core, it provides a web-based visual dataflow designer where users build directed graphs of processors to route, transform, and mediate data movement between any source and destination without writing custom code. The system records fine-grained data provenance for every data item from ingestion to delivery, supporting audit, debugging, and replay of data lineage. The platform distinguishes itself through a zero-master cluster architecture that distributes proc

    Javaapachehacktoberfestjava
    View on GitHub↗5,976
  • zenml-io/zenmlzenml-io avatar

    zenml-io/zenml

    5,451View on GitHub↗

    ZenML is an orchestration platform designed for building, deploying, and monitoring reproducible machine learning pipelines and agentic workflows. It provides a unified framework that manages the entire lifecycle of machine learning assets, from data processing and model training to the deployment of persistent inference services. By decoupling pipeline logic from underlying compute and storage, the platform enables teams to transition workflows seamlessly from local development environments to production-grade cloud infrastructure. The platform distinguishes itself through a service-oriented

    Pythonagentopsagentsai
    View on GitHub↗5,451
  • spotify/luigispotify avatar

    spotify/luigi

    18,676View on GitHub↗

    Luigi is a Python framework designed for building and managing complex batch data pipelines. It functions as a workflow orchestration engine that organizes tasks into directed acyclic graphs, ensuring that jobs execute in the correct logical order based on their dependencies. By utilizing a centralized scheduler, the system coordinates task execution across distributed environments, tracks global workflow state, and prevents redundant processing by verifying the existence of output targets before triggering any work. The project distinguishes itself through a robust state-tracking mechanism t

    Pythonhadoopluigiorchestration-framework
    View on GitHub↗18,676
Compare all 30 related projects→