agenda/agenda
Agenda
Agenda is a persistent background job scheduler and distributed task runner for Node.js applications. It functions as a cron job manager and task queue that ensures background processes survive application restarts by storing job state and metadata in a database.
The system coordinates execution across multiple worker instances using distributed locking mechanisms to prevent duplicate processing. It supports flexible scheduling via cron expressions or specific dates and includes a pluggable storage interface for backends such as MongoDB, PostgreSQL, and Redis.
The platform provides controls for job behavior, including priority and concurrency limits, job debouncing, and automated retry strategies with exponential backoff. Operational visibility is provided through a system dashboard, structured event logging, and administrative tools for managing the job lifecycle. Long-running tasks can be offloaded to separate child processes to maintain main thread responsiveness.
The project includes a REST API and a management interface protected by API key authentication and access control.
Features
- Cron Scheduling - Core engine that translates cron expressions into a persistent database-driven queue for recurring task execution.
- Job Schedulers - Provides flexible scheduling for tasks to run once or on a recurring basis using dates or cron expressions.
- Atomic State Updates - Implements atomic database queries to ensure consistency and prevent race conditions across distributed workers.
- Pluggable Storage Drivers - Offers a pluggable storage interface allowing the use of MongoDB, PostgreSQL, or Redis for job persistence.
- State Storage Implementations - Provides a standard repository interface allowing the integration of any database to store, query, and lock jobs.
- Distributed Locking - Prevents multiple worker instances from processing the same job simultaneously using distributed locking mechanisms.
- Distributed Locks - Implements a distributed locking mechanism to prevent multiple worker instances from processing the same job simultaneously.
- Distributed Task Workers - Coordinates heavy workload execution across multiple worker processes using a shared job queue.
- Job Concurrency Controllers - Manages and limits the simultaneous execution of background tasks based on shared resource keys to optimize system usage.
- Job Behavior Configurations - Provides a system for specifying job logic, concurrency limits, priority, and custom retry strategies.
- Node.js Job Schedulers - Provides a comprehensive library for scheduling and executing persistent background tasks in Node.js.
- Asynchronous Task Managers - Provides a framework for coordinating concurrent background operations and non-blocking task execution.
- Task Coordinations - Synchronizes job execution and tracks completion across distributed nodes using shared locks.
- Automated Retry Strategies - Implements automated retry strategies with constant, linear, or exponential backoff to handle transient errors.
- Background Job Schedulers - A persistent background job scheduler for Node.js that executes tasks independently of the main process.
- Persistent Background Workers - Ensures background tasks survive application restarts by persisting job state in a database.
- Change Notification Streams - Utilizes database change streams to trigger immediate job execution without relying on periodic polling.
- Atomic Duplicate Prevention - Prevents duplicate job processing by utilizing atomic locks and data property checks within the shared data store.
- Database Job Lifecycle Management - Enables the ability to enable, disable, cancel, or remove jobs directly from the database using query criteria.
- Process-Based Isolation - Offloads heavy or long-running tasks to separate child processes to isolate crashes and prevent main thread blocking.
- Job Debouncing - Combines multiple rapid job submissions into a single execution to prevent redundant processing of frequent updates.
- Job Event Callbacks - Triggers custom logic or captures error context, such as stack traces, via callbacks when jobs fail or complete.
- Job Workflow Administrators - Provides a graphical interface for the full lifecycle management of jobs, including requeueing and bulk modification.
- Manual Job Triggering - Allows triggering the immediate execution of a defined job task without altering its future cron schedule.
- Real-Time Triggers - Uses database change streams or signals to notify workers to process queued tasks instantly, eliminating polling delays.
- MongoDB Persistence Providers - Leverages MongoDB to persist job metadata and manage distributed locking for task execution.
- Worker Notification Channels - Implements a pluggable notification system to trigger immediate job processing across multiple worker processes.
- Background Task Retry Policies - Implements background task retry policies including configurable backoff intervals for failing jobs.
- Job Process Sandboxing - Isolates job execution in separate OS processes to protect the main application from crashes and event-loop blocking.
- Retry Policies - Provides configurable retry policies with exponential backoff to handle transient failures in background jobs.
- Job Monitoring Tools - Ships a dashboard and utilities for inspecting the status, successes, and failures of background jobs.
- Real-Time Monitoring Dashboards - Provides a visual dashboard for real-time monitoring of job counts, current states, and system configuration.
- Structured Logging - Stores structured execution events in a database to provide a machine-readable audit trail of job successes and failures.
- Worker Lifecycle Management - Provides controls for starting, stopping, and draining workers to ensure graceful shutdowns.
- Job Queues - MongoDB-backed job scheduling library.
- Scheduled Tasks - Lightweight job scheduling library.
