# celery/celery

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/celery-celery).**

28,115 stars · 4,952 forks · Python · other

## Links

- GitHub: https://github.com/celery/celery
- Homepage: https://docs.celeryq.dev
- awesome-repositories: https://awesome-repositories.com/repository/celery-celery.md

## Topics

`amqp` `python` `python-library` `python3` `queue-tasks` `queue-workers` `queued-jobs` `redis` `redis-queue` `sqs` `sqs-queue` `task-manager` `task-runner` `task-scheduler`

## Description

Celery is an asynchronous job processor and distributed task queue designed to offload time-consuming operations to background worker nodes. By utilizing a message-passing architecture, it decouples task producers from consumers, allowing applications to maintain responsiveness while scaling workloads across multiple isolated environments.

The system functions as a distributed workload orchestrator that manages the lifecycle of deferred operations through persistent queues. It distinguishes itself by providing a pluggable transport abstraction, which allows the core task logic to remain independent of specific messaging protocols. Furthermore, the framework includes built-in support for scheduled job execution, enabling the automation of recurring or delayed tasks without manual intervention.

The platform also incorporates an event-driven monitoring framework that broadcasts internal system signals to provide real-time visibility into task lifecycles and worker node health. This diagnostic layer, combined with result-backend persistence and serialization-based payload management, ensures reliable task completion and consistent data transmission across distributed systems.

## Tags

### Software Engineering & Architecture

- [Distributed Task Queues](https://awesome-repositories.com/f/software-engineering-architecture/distributed-task-queues.md) — Scales application workloads by distributing high volumes of concurrent jobs across multiple worker nodes using a message broker architecture.
- [Task Queues](https://awesome-repositories.com/f/software-engineering-architecture/task-queues.md) — A message-passing architecture that offloads time-consuming operations to background worker nodes to maintain responsive application performance.
- [Distributed Task Processors](https://awesome-repositories.com/f/software-engineering-architecture/distributed-task-processors.md) — Processes high volumes of messages in real-time by scheduling background operations and maintaining system health. ([source](https://docs.celeryq.dev))
- [Job Processors](https://awesome-repositories.com/f/software-engineering-architecture/job-processors.md) — A background execution engine that manages the lifecycle of deferred operations through persistent queues and pluggable message brokers.
- [Message Queuing Architectures](https://awesome-repositories.com/f/software-engineering-architecture/message-queuing-architectures.md) — Decouples task producers from consumers by routing serialized job payloads through an intermediary message queue for asynchronous processing.
- [Asynchronous Background Processors](https://awesome-repositories.com/f/software-engineering-architecture/asynchronous-background-processors.md) — Offloads time-consuming operations to background workers to keep user-facing applications responsive.
- [Worker Pool Models](https://awesome-repositories.com/f/software-engineering-architecture/worker-pool-models.md) — Maintains persistent background processes that pull tasks from queues to execute concurrent operations within isolated runtime environments.
- [Workload Orchestrators](https://awesome-repositories.com/f/software-engineering-architecture/workload-orchestrators.md) — A coordination layer that distributes concurrent tasks across multiple isolated environments while tracking execution status and metadata outcomes.
- [Transport Abstractions](https://awesome-repositories.com/f/software-engineering-architecture/transport-abstractions.md) — Decouples the core task logic from specific messaging protocols by using a unified interface to communicate with various external brokers.

### Development Tools & Productivity

- [Job Schedulers](https://awesome-repositories.com/f/development-tools-productivity/job-schedulers.md) — Automates recurring tasks or delays specific operations to run at precise future times.

### System Administration & Monitoring

- [Distributed Monitoring Tools](https://awesome-repositories.com/f/system-administration-monitoring/distributed-monitoring-tools.md) — Tracks the health, performance, and execution status of background processes across multiple servers.
- [Event Monitoring Systems](https://awesome-repositories.com/f/system-administration-monitoring/event-monitoring-systems.md) — Broadcasts internal system signals to external observers to provide real-time visibility into task lifecycles and worker node health.
- [Monitoring Frameworks](https://awesome-repositories.com/f/system-administration-monitoring/monitoring-frameworks.md) — A diagnostic layer that broadcasts internal state changes and performance metrics to provide real-time visibility into distributed system health.