OptimalBits/bull

Features

• Distributed Task Queues - Implements a distributed task queue to process background work across multiple nodes reliably and at scale.
• Redis-Backed Queues - Provides a distributed task queue that uses Redis as the primary data store for persistent state and message brokering.
• Cron Scheduling - Supports automated job execution based on cron expressions for recurring background tasks.
• Job Schedulers - Acts as a job scheduler for executing asynchronous functions with support for delayed and cron-based execution.
• Background Task Runners - Orchestrates the lifecycle of heavy background tasks with custom priority, retry policies, and concurrency limits.
• Task Schedulers - Automates periodic maintenance and updates using cron specifications to trigger recurring background jobs.
• Distributed Processing - Distributes asynchronous task processing across multiple Node.js worker processes using a shared Redis backend.
• Distributed Task Workers - Functions as a distributed task worker that processes heavy workloads across multiple threads or processes.
• Job Priority Management - Manages job execution order based on priority levels, concurrency limits, and automatic retry policies.
• Message Queues - Implements a persistent message queue for asynchronous task buffering and reliable event-driven workflows.
• Heartbeat Monitors - Tracks worker liveness through periodic heartbeats to automatically detect and re-queue stalled jobs.
• Job Queues - Provides infrastructure for managing asynchronous task execution by enqueueing serializable data objects.
• Job Processors - Provides an engine to define job processors with associated concurrency limits and routing rules.
• Message Queuing Architectures - Implements a message queuing architecture to decouple servers via asynchronous, persistent communication channels.
• Persistent Operation Queues - Persists queue metadata and job payloads in Redis to ensure reliability across process restarts.
• Reliable Task Queues - Ensures distributed system reliability by recovering from process crashes and automatically handling stalled tasks.
• Task Retry Policies - Implements automatic job retries based on defined policies to recover from transient execution errors.
• Lua Scripting - Utilizes server-side Lua scripts to perform atomic state transitions and prevent race conditions between distributed workers.
• State Querying - Allows querying of specific job instances and aggregate counts of jobs in various execution states.
• Slot-Based Routing - Uses hash-slot key prefixing to ensure all keys for a queue map to the same Redis cluster shard for atomicity.
• Crash Recovery Systems - Automatically recovers from process crashes by detecting and restarting interrupted jobs.
• Delayed Executions - Provides the capability to postpone the execution of background tasks until a specified future time.
• Process Sandboxing - Executes CPU-intensive jobs in separate processes to prevent main-thread crashes and event-loop blocking.
• Lifecycle Management - Enables manual control over job lifecycles, including retrying failed tasks and promoting delayed ones.
• Hanging Job Alerts - Detects hanging processes through heartbeat monitoring and automatically triggers alerts or retries.
• Polling-Based Worker Execution - Employs a polling-based execution model where workers retrieve tasks using Redis lists and sorted sets.
• Process Scaling - Scales throughput by running multiple jobs simultaneously across independent worker processes.
• Worker Throttling - Implements processing rate controls to prevent background jobs from overloading downstream dependencies.
• Message Brokers - Functions as a Redis-based message broker to decouple servers through the exchange of serializable data objects.
• Execution Control - Offers mechanisms to pause or resume job processing globally across all workers or locally.
• Job Process Sandboxing - Provides child-process sandboxing to execute CPU-intensive or unstable jobs without blocking the main Node.js event loop.
• Rate Limiting - Caps the number of jobs processed within a specific timeframe across all distributed workers to maintain system stability.
• Task Sequencing Strategies - Supports flexible job processing sequences including first-in-first-out, last-in-first-out, and priority-based ordering.
• Event Monitoring - Provides event monitoring to notify the system when jobs change state, such as completing or failing.
• Job Monitoring Tools - Provides tools to track job progress via completion percentages and custom log history.
• Job Queues - Persistent job and message queue system.
• Messaging and Event Streaming - Fast and reliable Redis-based queue for Node.js applications.
• Task Queues - Persistent job and message queue.

Star history