Open-source alternatives to Celery

Dramatiq is a distributed task queue and workload manager used to offload function execution to background workers. It functions as an asynchronous task orchestrator that enables the distribution of computational tasks across a cluster using a pluggable transport layer supporting RabbitMQ and Redis. The framework provides specialized tools for complex task orchestration, including the ability to link background jobs into sequences, pipelines, and barriers. It further manages distributed concurrency through the use of shared mutexes, rate limiters, and exponential backoff retries to prevent re

rq is a distributed task queue and background worker system for Python that uses a Redis backend to decouple task submission from execution. It functions as a reliable message queue and task scheduler, allowing Python functions or asyncio coroutines to be processed asynchronously across multiple worker processes. The project distinguishes itself through reliable queuing mechanisms that prevent job loss during worker crashes using atomic operations. It provides specialized orchestration capabilities, including the prevention of duplicate jobs, job execution prioritization, and the ability to m

RoadRunner is a high-performance application server and process manager designed to serve PHP applications using a persistent worker model. It eliminates bootload overhead and initialization time by keeping application processes alive between requests, acting as a protocol-agnostic proxy that routes traffic to a pool of supervised workers. The server is built with a plugin-based modular architecture, allowing it to be extended with custom Go plugins and compiled into tailored binaries. It distinguishes itself by providing a unified execution model for a wide array of communication protocols,

Machinery is a distributed task queue and asynchronous workflow engine. It provides a system for processing heavy workloads outside the main request flow using a network of distributed background workers and a message-based job orchestrator. The project manages complex task lifecycles through sequential chaining, where results are passed between tasks, and parallel coordination, which can trigger callback tasks upon the completion of a group. It supports periodic workflow scheduling for recurring jobs and delayed execution via specific timestamps. The system includes capabilities for result

.. image:: https://media.charlesleifer.com/blog/photos/huey3-logo.png

Flower is a monitoring and administration tool for Celery task queues. It provides a real-time web dashboard and a REST API to monitor distributed task clusters, manage worker instances, and observe message broker health. The project distinguishes itself by offering centralized control over the task lifecycle, allowing users to trigger, revoke, or terminate tasks and apply execution rate limits. It also includes a Prometheus metrics exporter to surface internal performance and status data for external monitoring and alerting systems. The tool covers a broad range of observability and managem

Hyperf is a high-performance PHP coroutine framework designed for building microservices and middleware. It utilizes non-blocking coroutines to handle high concurrency and low-latency request processing, providing a foundation for scalable distributed systems. The framework is distinguished by an aspect-oriented programming based dependency injector that enables pluggable components and meta-programming. It includes a coroutine-optimized object-relational mapper with integrated model caching and an orchestration toolkit for microservice governance, featuring service discovery, circuit breaker

This project is a self-hosted email verification system and API designed to validate email existence and clean mailing lists on private infrastructure. It functions as a deliverability tool that confirms if email addresses are reachable by communicating with mail servers via the SMTP protocol without sending actual messages. The system is distinguished by its high-volume SMTP infrastructure, which utilizes a stateless worker architecture and message queue task distribution to scale validation tasks. It includes an SMTP proxy gateway that routes requests through SOCKS5 proxies to mask server i

NATS Server is a high-performance, lightweight messaging system designed for cloud-native applications, edge computing, and distributed microservices. It functions as a distributed publish-subscribe broker that routes messages using hierarchical, dot-separated subject strings, enabling decoupled communication between services without requiring centralized broker lookups. The system supports core messaging patterns including asynchronous publish-subscribe, request-reply, and load-balanced queue processing. The platform distinguishes itself through a decentralized architecture that eliminates t

This project is a comprehensive educational resource and study guide focused on distributed systems architecture and backend infrastructure design. It provides a structured curriculum for mastering the principles of scalability, reliability, and performance required to design complex software systems. The repository distinguishes itself by offering a methodical approach to technical interview preparation, incorporating design patterns, architectural trade-offs, and spaced repetition tools to help users retain complex concepts. It emphasizes constraint-driven analysis, teaching users how to ev

Faktory is an open-source work server that queues, dispatches, and manages background jobs across multiple programming languages. It stores job payloads as JSON hashes in a Redis-backed queue and provides language-specific client and worker libraries that enable any language to push jobs to the server or fetch and execute them. The server includes a batch workflow orchestrator that groups jobs into batches with completion tracking for coordinating multi-step asynchronous workflows. It features a configurable job uniqueness filter that prevents duplicate enqueues within a time window, an expon

This project provides a framework for AI agent orchestration and context management, enabling the deployment of specialized AI personas and subagents to solve multi-step technical goals. It centers on managing specialized agents with isolated contexts and role-based prompts to handle domain-specific tasks. The system differentiates itself through a hierarchical project memory using markdown files to maintain coding standards and a secure execution model that utilizes sandboxed environments and git worktree isolation. It also features a Model Context Protocol integration for external tool conn

This project provides a structured curriculum and visual guide for mastering web development within the ASP.NET Core ecosystem. It serves as a comprehensive roadmap that maps out the essential technologies, milestones, and proficiency sequences required for developers to progress from beginner to advanced levels. The repository distinguishes itself by curating high-quality learning resources and technical documentation into a logical progression. It visualizes complex development paths through structured diagrams, helping users navigate the technical requirements of building and maintaining m

Beanstalkd is a distributed work queue server designed for asynchronous job processing. It functions as a standalone server that distributes background tasks between producers and consumers to improve application responsiveness and throughput. The system organizes tasks using numeric priority levels to ensure critical work is processed first. It manages the job lifecycle through discrete states and uses a simple line-based text protocol over TCP for communication. To ensure reliability, the server persists job data to a sequential disk log, allowing the queue state to be recovered after a sy

Resque is a Ruby library for enqueueing and processing asynchronous tasks using Redis as a data store. It functions as a distributed task processor and queue manager, allowing long-running work to be moved out of the main request cycle. The system executes background jobs in isolated child processes to prevent memory leaks and provides a web-based dashboard for monitoring queue depths, worker activity, and failed job statistics. Capability areas include distributed worker coordination via signals, error handling with job retry mechanisms, and priority-ordered queue management. It also suppor

Temporal is a distributed workflow orchestration engine designed to manage fault-tolerant, stateful, and long-running background processes. It functions as a platform for coordinating complex cross-service operations, ensuring consistency and reliability in distributed environments by decoupling workflow orchestration from task execution. The platform distinguishes itself through a deterministic, event-sourced execution model that reconstructs workflow state by re-executing code from an immutable event log. This approach isolates non-deterministic side effects into managed activities, allowin

Olmocr is a distributed document processing framework designed to convert PDF and image files into structured markdown. It functions as a vision-based document parser that utilizes multimodal neural networks to interpret complex visual layouts and translate them into standardized text representations. The system operates as a remote inference orchestrator, offloading heavy document analysis tasks to external servers or cloud APIs to minimize local computational requirements. By employing a stateless worker architecture, it decouples document ingestion from inference, allowing for the distribu

ClusterFuzz is an automated platform that runs coverage-guided fuzzers at scale to find security and stability bugs in software. It orchestrates libFuzzer and AFL++ across distributed clusters of worker bots, collecting coverage feedback to guide input mutation and discover crashes. The platform provides a web-based dashboard for configuring fuzzing jobs, monitoring progress, and inspecting crash reports, with role-based access control to restrict sensitive features. The system automates the full fuzzing lifecycle, from build pipeline integration and corpus management to crash triage and bug

cookiecutter-django is a template-based project generator and production-ready scaffold for Django web applications. It functions as a boilerplate that injects user-defined variables into predefined file templates to automate the creation of a standardized directory structure and initial project configuration. The project provides a production blueprint that integrates a customizable user authentication system, environment-variable configuration, and a containerized development environment. It bundles Django with databases and task queues to ensure consistency across local and production work

Redisson is a Java library and Redis client that functions as a distributed Java object mapper, caching provider, and locking framework. It maps Java collections and concurrency primitives to distributed implementations backed by Redis and Valkey, providing synchronous, asynchronous, and reactive APIs for interacting with these data stores. The project distinguishes itself by providing a comprehensive suite of distributed coordination tools, including a locking framework for managing semaphores and countdown latches across multiple application nodes. It also serves as a distributed messaging

pg-boss is a background task scheduler and distributed task queue that uses PostgreSQL as a reliable message broker for asynchronous job processing. It provides a system for distributing work across multiple application instances, ensuring exactly-once delivery through atomic database transactions. The project includes a cron job scheduler for automating recurring tasks and a PostgreSQL pub-sub system for fan-out event distribution. It also features a web-based management dashboard for monitoring queue statistics and controlling job lifecycles, including manual retries and cancellations. Cap

RocketMQ is a cloud-native distributed messaging platform and streaming engine. It functions as a distributed transactional queue that ensures atomicity between local transactions and message delivery, and serves as an MQTT IoT message broker to bridge lightweight device traffic into high-performance data streams. The system is distinguished by a Kubernetes-native architecture that decouples compute from storage to allow independent scaling of traffic and data retention. It utilizes a tiered storage model to offload older data to remote storage and employs quorum-based replication and automat

Cadence is a distributed workflow orchestration engine designed to execute long-running, asynchronous business logic with built-in durability and resilience across distributed systems. It functions as a stateful process manager that ensures processes resume from their last known state following system crashes or network outages. The platform utilizes a distributed task queue to manage work across independent worker nodes and supports persistence via SQL or Cassandra backend storage. It includes a workflow visualization dashboard for inspecting execution histories and state traces, alongside a

This is a Golang client library for interacting with a cloud native distributed messaging system. It provides the necessary tools for Go applications to exchange messages using publish-subscribe and request-reply patterns, as well as specialized clients for managing persistent streams and distributed storage. The library includes a JetStream client for durable message streaming and replay, a Key-Value store client for managing distributed state with versioning and watchers, and an Object Store client for the storage and retrieval of large binary files via chunked delivery. The implementation

Sidekiq is a Ruby background processing framework and asynchronous task runner. It functions as a Redis-backed background job processor that offloads heavy or time-consuming work from web requests to separate worker processes to ensure the main application remains responsive. The system operates as a Redis task queue, storing pending jobs in Redis to be processed concurrently by multiple threads. It provides a framework for distributed task queueing and asynchronous job scheduling to coordinate work across multiple server instances. The project covers Ruby application scaling by executing ba

This is a Go client library and protocol implementation for interacting with message brokers using the AMQP 0.9.1 standard. It serves as an asynchronous messaging toolkit and message queue client for implementing producer and consumer patterns. The library provides a programmatic interface for managing the full message lifecycle, including publishing, consumption, and delivery acknowledgments. It supports reliable delivery mechanisms such as publishing confirmations and atomic transactions. The project covers the administration of routing topologies through the declaration and binding of que

Light Task Scheduler is a distributed job scheduling and workflow orchestration platform designed for managing background processing across scalable computing environments. It functions as a cluster management system that coordinates stateless nodes to execute recurring, cron-based, or one-time tasks with centralized control and high availability. The platform distinguishes itself through a leader-based coordination model that automatically elects a primary controller to manage task distribution and system state. It supports complex workflow dependencies, ensuring that prerequisite tasks comp