# kestra-io/kestra **Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/kestra-io-kestra).** 26,419 stars · 2,497 forks · Java · apache-2.0 ## Links - GitHub: https://github.com/kestra-io/kestra - Homepage: https://kestra.io - awesome-repositories: https://awesome-repositories.com/repository/kestra-io-kestra.md ## Topics `automation` `data-orchestration` `devops` `hacktoberfest` `high-availability` `infrastructure-as-code` `java` `low-code` `lowcode` `orchestration` `pipeline` `pipeline-as-code` `workflow` ## Description Kestra is a declarative workflow orchestrator designed to manage complex task dependencies and automated processes through versioned configuration files. It functions as a distributed platform that decouples task scheduling from execution by offloading computational workloads to a fleet of worker nodes. The system uses a reactive, event-driven engine to initiate workflows automatically in response to external signals, webhooks, schedules, or file system changes. The platform distinguishes itself through a modular plugin architecture that allows for the integration of custom tasks and external services. It provides an AI-native development environment that incorporates language models to generate, refine, and execute automation logic using natural language prompts. To support diverse operational needs, Kestra implements a multi-tenant execution model that isolates resources, data, and access controls for different teams within a single shared instance. The system covers a broad range of operational capabilities, including robust state management, granular role-based access control, and comprehensive system auditing. It offers extensive tools for workflow logic, such as conditional branching, parallel task execution, and iterative processing, alongside built-in resilience features like automated retries and failure policies. Users can manage these configurations through a centralized interface that supports visual editing and real-time monitoring of execution status. ## Tags ### Data & Databases - [Data Pipeline Orchestrators](https://awesome-repositories.com/f/data-databases/data-pipeline-orchestrators.md) — Automates complex sequences of data processing tasks, including scheduling and dependency management. - [Data Processing Tasks](https://awesome-repositories.com/f/data-databases/data-processing-tasks.md) — Offloads compute-intensive data tasks like file operations and API requests to dedicated worker components. ([source](https://kestra.io/docs/workflow-components/tasks.md)) - [State Management](https://awesome-repositories.com/f/data-databases/state-management.md) — Maintains workflow execution status and metadata in a centralized database for auditability and reliability. - [Task Result Storage](https://awesome-repositories.com/f/data-databases/task-result-storage.md) — Saves task results in internal storage to generate a file URI that other tasks can reference. ([source](https://kestra.io/docs/workflow-components/outputs.md)) - [Collection Iterators](https://awesome-repositories.com/f/data-databases/collection-iterators.md) — Processes lists of values by executing task groups for each item with support for concurrency limits. ([source](https://kestra.io/docs/workflow-components/tasks/flowable-tasks.md)) - [Inter-flow Data Sharing](https://awesome-repositories.com/f/data-databases/inter-flow-data-sharing.md) — Shares typed data between parent and sub-flows, allowing downstream tasks to access results from child executions. ([source](https://kestra.io/docs/workflow-components/outputs.md)) - [Dynamic Data Accessors](https://awesome-repositories.com/f/data-databases/dynamic-data-accessors.md) — References dynamic iteration values and task outputs within loops to process list elements or results. ([source](https://kestra.io/docs/workflow-components/outputs.md)) ### Development Tools & Productivity - [Workflow Orchestrators](https://awesome-repositories.com/f/development-tools-productivity/workflow-orchestrators.md) — Manages complex task dependencies and execution logic through versioned configuration files. - [Event-Driven Automation Engines](https://awesome-repositories.com/f/development-tools-productivity/event-driven-automation-engines.md) — Triggers and executes automated processes based on schedules, webhooks, or external system events. - [Workflow Parameter Schemas](https://awesome-repositories.com/f/development-tools-productivity/workflow-parameter-schemas.md) — Defines strongly-typed inputs with validation and default values to ensure parameters are provided before execution. ([source](https://kestra.io/docs/workflow-components/inputs.md)) - [Plugin Architectures](https://awesome-repositories.com/f/development-tools-productivity/plugin-architectures.md) — Enables integration of custom tasks and external services through a modular plugin interface. - [Execution Context Management](https://awesome-repositories.com/f/development-tools-productivity/execution-context-management.md) — Updates execution-scoped variables during a workflow run to inject new data into the context. ([source](https://kestra.io/docs/workflow-components/variables.md)) - [Template Rendering Engines](https://awesome-repositories.com/f/development-tools-productivity/template-rendering-engines.md) — Generates dynamic values and controls template flow using expression and logic delimiters. ([source](https://kestra.io/docs/expressions/syntax.md)) ### DevOps & Infrastructure - [Workflow Orchestration](https://awesome-repositories.com/f/devops-infrastructure/workflow-orchestration.md) — Kestra groups tasks to run one after another in a defined sequence, allowing downstream tasks to access outputs from preceding steps. ([source](https://kestra.io/docs/workflow-components/tasks/flowable-tasks.md)) - [Workflow Event Triggers](https://awesome-repositories.com/f/devops-infrastructure/workflow-event-triggers.md) — Sets up automated execution using schedules, webhooks, or event-based triggers to initiate processes. ([source](https://kestra.io/docs/workflow-components/triggers.md)) - [Distributed Task Queues](https://awesome-repositories.com/f/devops-infrastructure/distributed-task-queues.md) — Offloads computational workloads to a fleet of worker nodes via a centralized message bus. - [Event-Driven Triggers](https://awesome-repositories.com/f/devops-infrastructure/event-driven-triggers.md) — Initiates workflow execution automatically by monitoring external signals, webhooks, and schedules. - [Infrastructure Orchestration](https://awesome-repositories.com/f/devops-infrastructure/infrastructure-orchestration.md) — Coordinates microservices and infrastructure operations through versioned, declarative configuration files. - [Task Scheduling Policies](https://awesome-repositories.com/f/devops-infrastructure/task-scheduling-policies.md) — Assigns specific tasks to designated worker groups to isolate resource-intensive workloads. ([source](https://kestra.io/docs/enterprise/scalability/worker-group.md)) - [Execution Rate Limiters](https://awesome-repositories.com/f/devops-infrastructure/execution-rate-limiters.md) — Restricts the number of simultaneous executions for a specific flow to protect downstream systems. ([source](https://kestra.io/docs/workflow-components/concurrency.md)) - [Concurrency Policies](https://awesome-repositories.com/f/devops-infrastructure/concurrency-policies.md) — Defines how the system handles new executions when limits are reached by queuing, canceling, or failing requests. ([source](https://kestra.io/docs/workflow-components/concurrency.md)) - [Incident Containment](https://awesome-repositories.com/f/devops-infrastructure/incident-containment.md) — Stops problematic executions across specific tenants to contain incidents without pausing the entire platform. ([source](https://kestra.io/docs/enterprise/instance/kill-switch.md)) - [Trigger Condition Filters](https://awesome-repositories.com/f/devops-infrastructure/trigger-condition-filters.md) — Restricts workflow execution based on criteria like status, labels, or custom expressions. ([source](https://kestra.io/docs/workflow-components/triggers.md)) ### Artificial Intelligence & ML - [AI-Native Development Environments](https://awesome-repositories.com/f/artificial-intelligence-ml/ai-native-development-environments.md) — Enables users to generate, refine, and execute workflow code using natural language prompts. - [AI-Native Development](https://awesome-repositories.com/f/artificial-intelligence-ml/ai-native-development.md) — Uses natural language prompts to generate, refine, and execute complex automation logic. ### Business & Productivity Software - [Workflow Automation](https://awesome-repositories.com/f/business-productivity-software/workflow-automation.md) — Triggers business processes automatically in response to external events and scheduled intervals. ### Software Engineering & Architecture - [Declarative Configuration](https://awesome-repositories.com/f/software-engineering-architecture/declarative-configuration.md) — Defines complex workflow logic and task dependencies using versioned, human-readable configuration files. - [Workflow Definitions](https://awesome-repositories.com/f/software-engineering-architecture/workflow-definitions.md) — Defines orchestration units using configuration files to manage tasks, inputs, and execution logic. ([source](https://kestra.io/docs/workflow-components/flow.md)) - [Orchestration Logic](https://awesome-repositories.com/f/software-engineering-architecture/orchestration-logic.md) — Manages workflow execution using control-logic tasks that handle parallel execution, conditional branching, and iterative processing. ([source](https://kestra.io/docs/workflow-components/tasks.md)) - [Workflow Triggers](https://awesome-repositories.com/f/software-engineering-architecture/workflow-triggers.md) — Initiates workflow runs automatically based on external events, scheduled times, webhooks, or file system changes. ([source](https://kestra.io/docs/workflow-components/flow.md)) - [Conditional Branching](https://awesome-repositories.com/f/software-engineering-architecture/conditional-branching.md) — Routes workflow logic to specific tasks based on the evaluation of contextual variables. ([source](https://kestra.io/docs/workflow-components/tasks/flowable-tasks.md)) - [Task Execution Engines](https://awesome-repositories.com/f/software-engineering-architecture/task-execution-engines.md) — Runs individual actions within a flow, choosing between computational tasks for data work or flowable tasks for controlling execution logic. ([source](https://kestra.io/docs/workflow-components/flow.md)) - [Workflow Replay Systems](https://awesome-repositories.com/f/software-engineering-architecture/workflow-replay-systems.md) — Manages failed workflows by restarting specific tasks or replaying from a point of failure. ([source](https://kestra.io/docs/workflow-components/retries.md)) - [Parallel Execution Engines](https://awesome-repositories.com/f/software-engineering-architecture/parallel-execution-engines.md) — Runs multiple tasks simultaneously to improve processing efficiency in distributed environments. ([source](https://kestra.io/docs/workflow-components/tasks/flowable-tasks.md)) - [Task Retry Policies](https://awesome-repositories.com/f/software-engineering-architecture/task-retry-policies.md) — Defines automatic retry policies for failed tasks, including maximum attempts and interval strategies. ([source](https://kestra.io/docs/workflow-components/retries.md)) - [Error Handling Workflows](https://awesome-repositories.com/f/software-engineering-architecture/error-handling-workflows.md) — Defines a list of tasks to execute sequentially when an error occurs to perform automated alerts or cleanup. ([source](https://kestra.io/docs/workflow-components/errors.md)) - [Execution Failure Policies](https://awesome-repositories.com/f/software-engineering-architecture/execution-failure-policies.md) — Defines task execution policies to allow downstream tasks to continue running despite failures. ([source](https://kestra.io/docs/workflow-components/errors.md)) - [Subflow Orchestrators](https://awesome-repositories.com/f/software-engineering-architecture/subflow-orchestrators.md) — Executes child flows from parent workflows with support for completion waiting and state propagation. ([source](https://kestra.io/docs/workflow-components/subflows.md)) - [Task Property Templating](https://awesome-repositories.com/f/software-engineering-architecture/task-property-templating.md) — Sets task properties using static values or dynamic expressions while respecting data type constraints. ([source](https://kestra.io/docs/workflow-components/tasks.md)) - [Workflow Input Schemas](https://awesome-repositories.com/f/software-engineering-architecture/workflow-input-schemas.md) — Provides strongly typed parameters at execution time to customize flow behavior. ([source](https://kestra.io/docs/workflow-components/flow.md)) - [Configuration Variables](https://awesome-repositories.com/f/software-engineering-architecture/configuration-variables.md) — Enables dynamic injection of static and computed values into workflow configurations using templating syntax. ([source](https://kestra.io/docs/workflow-components/variables.md)) - [Event-Driven Triggers](https://awesome-repositories.com/f/software-engineering-architecture/event-driven-triggers.md) — Initiates separate, decoupled flows from current executions to enable modular process design. ([source](https://kestra.io/docs/workflow-components/tasks/flowable-tasks.md)) - [Global Retry Strategies](https://awesome-repositories.com/f/software-engineering-architecture/global-retry-strategies.md) — Defines system-wide retry policies to automatically apply consistent error recovery behavior across all tasks. ([source](https://kestra.io/docs/workflow-components/retries.md)) - [Namespace Management](https://awesome-repositories.com/f/software-engineering-architecture/namespace-management.md) — Organizes secrets, variables, and plugin defaults within logical namespaces for fine-grained access control. ([source](https://kestra.io/docs/enterprise/governance/namespace-management.md)) - [Subflow Data Interfaces](https://awesome-repositories.com/f/software-engineering-architecture/subflow-data-interfaces.md) — Passes input arguments to child flows and retrieves output values for use in downstream tasks. ([source](https://kestra.io/docs/workflow-components/subflows.md)) - [Task Identity Management](https://awesome-repositories.com/f/software-engineering-architecture/task-identity-management.md) — Assigns core properties to tasks to uniquely identify each step and specify its implementation class. ([source](https://kestra.io/docs/workflow-components/tasks.md)) ### Security & Cryptography - [Execution Isolation](https://awesome-repositories.com/f/security-cryptography/execution-isolation.md) — Provides secure task execution isolation to prevent cross-tenant interference. ([source](https://kestra.io/docs/enterprise/governance/worker-isolation.md)) - [Multi-Tenancy Management](https://awesome-repositories.com/f/security-cryptography/multi-tenancy-management.md) — Isolates resources, data, and access controls for different teams within a single shared instance. - [Role-Based Access Control](https://awesome-repositories.com/f/security-cryptography/role-based-access-control.md) — Assigns specific permissions to users and service accounts to control access to resources. ([source](https://kestra.io/docs/enterprise/auth/rbac.md)) - [Multi-Tenancy Governance](https://awesome-repositories.com/f/security-cryptography/multi-tenancy-governance.md) — Manages isolated workflows and resources for different teams while maintaining strict security boundaries. - [Secret Management](https://awesome-repositories.com/f/security-cryptography/secret-management.md) — Integrates external services to store and retrieve sensitive data securely during task execution. ([source](https://kestra.io/docs/enterprise/governance/secrets-manager.md)) - [Environment Isolation](https://awesome-repositories.com/f/security-cryptography/environment-isolation.md) — Separates resources, data, and access controls for different teams using isolated administrative environments. ([source](https://kestra.io/docs/enterprise/governance/tenants.md)) - [Execution Sandboxes](https://awesome-repositories.com/f/security-cryptography/execution-sandboxes.md) — Ensures security by running individual tasks within isolated, containerized environments. - [Single Sign-On](https://awesome-repositories.com/f/security-cryptography/single-sign-on.md) — Authenticates users through external identity providers using the OpenID Connect protocol. ([source](https://kestra.io/docs/enterprise/auth/sso.md)) - [Data Encryption](https://awesome-repositories.com/f/security-cryptography/data-encryption.md) — Kestra protects sensitive script task outputs by encrypting them, ensuring they remain hidden in the interface while remaining accessible to subsequent tasks. ([source](https://kestra.io/docs/workflow-components/outputs.md)) - [Service Account Management](https://awesome-repositories.com/f/security-cryptography/service-account-management.md) — Provides programmatic API access for external applications using assigned roles and tokens. ([source](https://kestra.io/docs/enterprise/auth/service-accounts.md)) ### System Administration & Monitoring - [Audit Logging](https://awesome-repositories.com/f/system-administration-monitoring/audit-logging.md) — Tracks all actions performed by users and service accounts to ensure accountability and compliance. ([source](https://kestra.io/docs/enterprise/governance/audit-logs.md))