Frameworks für verteiltes Rechnen

Light Task Scheduler is a distributed job scheduling and workflow orchestration platform designed for managing background processing across scalable computing environments. It functions as a cluster management system that coordinates stateless nodes to execute recurring, cron-based, or one-time tasks with centralized control and high availability. The platform distinguishes itself through a leader-based coordination model that automatically elects a primary controller to manage task distribution and system state. It supports complex workflow dependencies, ensuring that prerequisite tasks comp

Akka is an actor model framework and distributed systems platform used to build concurrent and distributed applications. It provides a toolkit for managing multi-threaded state and behavior through asynchronous message passing, allowing developers to create concurrent applications without manual locks or synchronization. The system functions as a cluster management and event sourcing framework, automating the scaling and coordination of high-availability clusters. It enables the deployment of elastic services that coordinate workloads across multiple network nodes and ensures fault tolerance

Celery is an asynchronous job processor and distributed task queue designed to offload time-consuming operations to background worker nodes. By utilizing a message-passing architecture, it decouples task producers from consumers, allowing applications to maintain responsiveness while scaling workloads across multiple isolated environments. The system functions as a distributed workload orchestrator that manages the lifecycle of deferred operations through persistent queues. It distinguishes itself by providing a pluggable transport abstraction, which allows the core task logic to remain indep

Machinery is a distributed task queue and asynchronous workflow engine. It provides a system for processing heavy workloads outside the main request flow using a network of distributed background workers and a message-based job orchestrator. The project manages complex task lifecycles through sequential chaining, where results are passed between tasks, and parallel coordination, which can trigger callback tasks upon the completion of a group. It supports periodic workflow scheduling for recurring jobs and delayed execution via specific timestamps. The system includes capabilities for result

OTP is a concurrent programming framework and distributed computing platform that serves as the Erlang runtime environment. It provides a fault-tolerant operating environment designed for building scalable, real-time systems that manage massive amounts of simultaneous tasks through asynchronous messaging. The environment is distinguished by its use of an actor-based concurrency model and hierarchical supervision trees that automatically restart failed processes. It supports hot code loading to allow system updates without downtime and utilizes a preemptive user-space scheduler to manage light

Ray is a distributed computing framework designed to scale Python and Java applications across clusters by abstracting task scheduling and resource management. It functions as a resource-aware execution engine that manages task dependencies, placement, and fault tolerance across networked compute nodes. At its core, the system provides a stateful actor model, allowing developers to define classes that run in dedicated processes to maintain and mutate internal state across remote method calls. The framework distinguishes itself through a robust cross-language interoperability layer, enabling f

Faktory is an open-source work server that queues, dispatches, and manages background jobs across multiple programming languages. It stores job payloads as JSON hashes in a Redis-backed queue and provides language-specific client and worker libraries that enable any language to push jobs to the server or fetch and execute them. The server includes a batch workflow orchestrator that groups jobs into batches with completion tracking for coordinating multi-step asynchronous workflows. It features a configurable job uniqueness filter that prevents duplicate enqueues within a time window, an expon

protoactor-go is a framework for building concurrent and distributed systems in Go using the actor model. It provides a distributed actor system that enables isolated entities to communicate via asynchronous messaging and share state across a cluster. The framework implements a multi-language actor protocol, allowing interoperability between actors written in Go, C#, and Java. It further supports a virtual actor implementation, where actors are automatically instantiated across a network based on a unique identity. The system includes a supervision model for managing actor lifecycles and fau

Dramatiq is a distributed task queue and workload manager used to offload function execution to background workers. It functions as an asynchronous task orchestrator that enables the distribution of computational tasks across a cluster using a pluggable transport layer supporting RabbitMQ and Redis. The framework provides specialized tools for complex task orchestration, including the ability to link background jobs into sequences, pipelines, and barriers. It further manages distributed concurrency through the use of shared mutexes, rate limiters, and exponential backoff retries to prevent re

Apache Mesos is a distributed systems kernel and cluster resource manager that abstracts CPU, memory, and storage across a pool of nodes. It functions as a distributed infrastructure orchestrator, providing a layer to run multiple orchestration frameworks on a shared set of physical or virtual machines. The system acts as a resource isolation engine, dividing a shared cluster into isolated containers to run diverse workloads concurrently. It enables multi-framework orchestration, allowing different distributed application frameworks to share a single infrastructure to maximize hardware utiliz

Temporal is a distributed workflow orchestration engine designed to manage fault-tolerant, stateful, and long-running background processes. It functions as a platform for coordinating complex cross-service operations, ensuring consistency and reliability in distributed environments by decoupling workflow orchestration from task execution. The platform distinguishes itself through a deterministic, event-sourced execution model that reconstructs workflow state by re-executing code from an immutable event log. This approach isolates non-deterministic side effects into managed activities, allowin

Hazelcast is a distributed data platform that combines an in-memory data grid with a stream processing engine to support real-time analytics and event-driven applications. It functions as a partitioned, distributed key-value store that replicates data across cluster nodes to provide low-latency access and high availability. The platform also serves as a distributed SQL query engine, allowing users to execute standard SQL statements against both in-memory datasets and external data sources. What distinguishes Hazelcast is its use of a distributed consensus subsystem to maintain strongly consis

Prefect is a workflow orchestration platform designed to define, schedule, and monitor complex data pipelines as Python code. It functions as a container-native engine that wraps individual tasks in isolated environments, ensuring consistent dependencies and resource allocation across diverse infrastructure. By utilizing a state-machine-based orchestration model, the system tracks execution progress through discrete transitions and persistent event logs to maintain reliable and observable task processing. The platform distinguishes itself through a decoupled worker-API architecture, which sep

Dask is a parallel computing framework and distributed task scheduler designed to scale Python data science workflows from single machines to large clusters. It functions as a cluster resource manager that orchestrates computational logic by representing tasks and their dependencies as directed acyclic graphs. This architecture allows the system to automate the distribution of workloads across available hardware while managing complex execution requirements. The project distinguishes itself through a lazy evaluation engine that defers data operations until they are explicitly requested, enabl

Dapr is a distributed application runtime that provides a sidecar-based infrastructure layer for building resilient microservices and event-driven applications. By utilizing a sidecar proxy pattern, it abstracts complex infrastructure tasks into standardized, network-accessible APIs, allowing developers to focus on application logic while the runtime handles service discovery, state management, and secure communication. The platform distinguishes itself through a pluggable component architecture and language-agnostic design, enabling services written in any programming language to interact wi

Spring Cloud Alibaba is a microservices orchestration framework that provides a standardized programming model for building distributed systems. It functions as a cloud-native integration layer, bridging enterprise application frameworks with distributed infrastructure to manage service discovery, traffic control, and state consistency across complex, multi-part application environments. The framework distinguishes itself through specialized components for managing distributed operations, including aspect-oriented traffic control that enforces flow rules, circuit breaking, and rate limiting a

Metaflow is a Python machine learning framework and MLOps workflow orchestrator designed to manage the lifecycle of data pipelines from local prototyping to production. It serves as a distributed compute manager and an experiment tracking system, enabling the creation of reproducible pipelines that transition between development and high-availability production environments. The framework distinguishes itself through an integrated checkpointing system that automatically persists intermediate data artifacts to remote storage, allowing failed runs to be resumed from the last successful step. It

Chronos is a distributed, fault-tolerant job scheduler designed for managing containerized workloads within a cluster. It functions as a task orchestrator that automates the execution of recurring background jobs and complex, multi-step workflows across distributed computing resources. The system distinguishes itself through its ability to manage directed acyclic graph dependencies, ensuring that tasks are triggered only upon the successful completion of prerequisite jobs. It utilizes a leader-follower consensus architecture to maintain high availability and state persistence, while relying o