oban-bg/oban

Features

• Background Job Queues - Provides a distributed background job queue using PostgreSQL for transactional storage and reliable task execution.
• Database-Backed Deferred Queues - Provides a persistent background job queue utilizing PostgreSQL to ensure transactional guarantees and reliable delivery.
• Hierarchical Workflows - Coordinates complex sequences of distributed jobs with cumulative context and hierarchical sub-workflows.
• Database Job Lifecycle Management - Allows enabling, disabling, retrying, or cancelling jobs directly through database operations without code deployment.
• Workflow Scheduling And Orchestration - Provides a distributed framework for managing complex multi-step job pipelines and data process orchestration.
• Background Task Schedulers - Inserts tasks into a SQL database for asynchronous execution with transactional reliability guarantees.
• Workflow Orchestration - Coordinates complex sequences of dependent background jobs that share context and execute in a specific order.
• Cron Scheduling - Implements a distributed cron-like timer for triggering recurring background jobs across a cluster.
• Job Schedulers - Automates recurring tasks on a cron-like schedule across multiple nodes while preventing duplicate insertions.
• Schedule Management - Provides the capability to modify scheduled jobs, timezones, and parameters at runtime across the cluster.
• Distributed Concurrency Controls - Limits total job execution rates across all nodes using a shared distributed quota system.
• Background Job Processing - Provides a distributed system for scheduling and executing asynchronous tasks using a SQL database.
• Concurrency Control - Provides mechanisms to limit parallel job execution through isolated queues.
• Distributed Job Execution - Distributes background tasks across multiple cluster nodes and tracks completion results.
• Distributed Recurring Task Scheduling - Manages cron-like jobs that run on a set schedule across multiple nodes without triggering duplicate executions.
• Distributed Task Schedulers - Provides a cluster-aware scheduler for managing asynchronous tasks, cron jobs, and complex workflow dependencies.
• Execution Rate Limiters - Enforces global concurrency and rate limits across a cluster to prevent system overload.
• Job Dependency Coordination - Manages complex asynchronous workflows through parent-child job hierarchies and shared context.
• Transactional Enqueueing - Wraps job enqueueing within database transactions to ensure tasks are only scheduled if the parent business logic succeeds.
• Distributed Rate Limiting - Synchronizes request and task limits across a distributed cluster to prevent downstream system overload.
• Database-Backed Workers - Implements a worker system that leverages PostgreSQL to coordinate job concurrency and rate limiting across nodes.
• Distributed Cluster Coordination - Synchronizes job state and scheduling across cluster nodes using a shared PostgreSQL database as the source of truth.
• Job Continuation Chaining - Implements mechanisms for linking dependent asynchronous tasks where child jobs trigger upon parent success.
• Partitioned - Ships the ability to split queues into isolated partitions with independent concurrency limits.
• Global Job Concurrency Limits - Implements cluster-wide concurrency limits to prevent downstream system overload.
• Chunked Execution - Allows executing jobs atomically in groups based on size or timeouts with granular error handling.
• Runtime Queue Management - Allows starting, stopping, pausing, or reconfiguring queues dynamically with settings that persist across restarts.
• Job Priority Management - Provides logic to adjust job priority over time to prevent task starvation.
• Job Deduplication - Prevents duplicate background tasks from being enqueued based on arguments or queues.
• Job Progress Grouping - Tracks collective progress of job sets and triggers callbacks when individual jobs or entire groups complete.
• Job Cancellation - Provides the capability to stop active jobs mid-execution across any node in the cluster.
• Stalled Job Recovery - Identifies and rescues orphaned jobs or those that stalled due to worker crashes.
• Worker Scaling - Adjusts the number of active worker nodes based on real-time queue depth to handle traffic fluctuations.
• Job Batching - Groups multiple background tasks together to track collective progress and trigger callbacks upon completion.
• Programmatic Quota Reservations - Allows programmatic checking and reservation of capacity to manage background task volume.
• Synchronous Result Retrieval - Inserts jobs from any node and receives results synchronously via automatic forwarding across the cluster.
• Background Job Monitoring - Tracks throughput, queue depth, and execution time through built-in instrumentation and metrics.
• Historical Job Analytics - Retains processed job data in the database for historic analytics and health monitoring.
• Queue Administration Interfaces - Provides tools for pausing queues, adjusting priorities, and pruning tasks at runtime.

Star history