Open-source alternatives to P Queue

Tunny is a worker pool library and concurrency manager for Go. It functions as a concurrent task scheduler that limits the number of parallel workers processing jobs to prevent system resource exhaustion. The project enables the maintenance of persistent state within individual worker routines to avoid repeated initialization costs. It also supports dynamic worker scaling, allowing the number of active background workers to be adjusted in real time without interrupting tasks in progress. The library provides mechanisms for task timeout enforcement and worker lifecycle management, including t

Dramatiq is a distributed task queue and workload manager used to offload function execution to background workers. It functions as an asynchronous task orchestrator that enables the distribution of computational tasks across a cluster using a pluggable transport layer supporting RabbitMQ and Redis. The framework provides specialized tools for complex task orchestration, including the ability to link background jobs into sequences, pipelines, and barriers. It further manages distributed concurrency through the use of shared mutexes, rate limiters, and exponential backoff retries to prevent re

Bluebird is a JavaScript promise library designed for managing asynchronous operations with a custom promise implementation. It provides a framework for controlling promise lifecycles and orchestrating non-blocking programming patterns. The library distinguishes itself with an asynchronous debugging toolkit that captures long stack traces across asynchronous boundaries and a global handler for unhandled promise rejections. It includes a predicate-based error filtering system to target specific exception types and a mechanism for the deterministic cleanup of system resources. It covers a broa

Gevent is a Python coroutine concurrency library and asynchronous task manager designed for high-concurrency I/O tasks. It provides a cooperative networking framework for building asynchronous TCP, UDP, and HTTP servers, as well as a WSGI web server implementation for hosting web applications. The project is distinguished by its standard library monkey-patching tool, which replaces blocking synchronous functions with cooperative versions to enable asynchronous behavior in third-party code. This allows for a cooperative multitasking workflow where the system yields execution during I/O waits t

Inngest is a durable execution framework and event-driven automation engine designed to orchestrate background workflows. It enables developers to build resilient, stateful processes by memoizing function steps, ensuring that long-running tasks can automatically resume from the last successful operation after failures, timeouts, or infrastructure restarts. The platform distinguishes itself through its event-driven architecture, which uses a schema-validated bus to trigger functions and coordinate complex, multi-step logic. It employs an onion-model middleware approach for cross-cutting concer

Crawlee-python is a web crawling framework for building scalable scrapers using Python. It serves as a comprehensive tool for web scraping automation, providing a system to extract structured data from websites using both lightweight HTTP requests and headless browser automation. The framework is distinguished by its anti-bot evasion capabilities, which include browser fingerprint impersonation and tiered proxy rotation to bypass detection systems and solve challenges such as Cloudflare. It also incorporates artificial intelligence for autonomous website navigation and schema-based data extra

Promisify a callback-style function

Run promise-returning & async functions a specific number of times concurrently

Run multiple promise-returning & async functions with limited concurrency

Map over promises concurrently

Run promise-returning & async functions concurrently with optional limited concurrency

Delay a promise a specified amount of time

Promisify an event by waiting for it to be emitted

Kue is a Redis-backed job queue library for Node.js that provides a complete system for defining, scheduling, and processing background work. It stores job metadata and state in Redis lists and sorted sets, enabling persistent, in-memory operations with configurable concurrency control and priority-sorted processing. The library includes a RESTful HTTP API for managing jobs and a web-based monitoring dashboard for inspecting job status, progress, and logs. The system distinguishes itself through its event-driven worker model, where workers listen for job events via Redis pub/sub and process j

This is an open-source, crowd-sourced wiki textbook that teaches Linux system programming in C. It covers the core operating system concepts of process management through the fork-exec-wait model, dynamic memory allocation using implicit free list heap allocators, inode-based file systems, inter-process communication via pipes and shared memory, POSIX threads with synchronization primitives, signal-based asynchronous notification, virtual memory with page table translation, and runtime diagnostics using Valgrind and GDB. The textbook distinguishes itself by providing practical, implementation

conc is a Go concurrency library and structured concurrency framework providing primitives for managing parallel tasks, mapping slices, and collecting results. It implements a system for spawning scoped tasks to ensure all child processes complete before their parent exits. The library includes a goroutine pool manager to limit active concurrent processes and a panic-safe task runner that catches panics in goroutines and propagates stack traces to the parent. It also provides a concurrent map-reduce tool for transforming data slices and processing streams in parallel while maintaining the ori

LiteOS is a lightweight real-time operating system kernel designed for constrained devices with limited memory and processing power. It provides a foundation for embedded IoT development, focusing on stable and predictable system resource management. The system implements a preemptive task scheduler and interrupt-driven resource management to ensure real-time responsiveness. It incorporates a tickless idle mechanism and optimized sleep-wake cycles to reduce power consumption in battery-powered hardware. The kernel includes capabilities for task synchronization via semaphores and static memor

threads.js is a JavaScript worker thread library and asynchronous task orchestrator designed to offload CPU-intensive tasks to background threads. It functions as a cross-platform worker pool and a TypeScript worker runner that automatically transpiles and executes TypeScript files within separate threads. The project distinguishes itself by providing a unified worker API that allows for cross-platform execution across different JavaScript environments. It includes a specialized worker data serialization tool to pass complex class instances and binary data between threads, and supports inlini

RxDownload is a multi-threaded download manager built with Kotlin and RxJava. It functions as a concurrent file transfer client and batch download controller, allowing for the retrieval of remote files using multiple threads to increase transfer speeds. The system provides a dedicated record management database for download tasks, enabling the retrieval of history through filtered lists and paginated results. It includes a centralized controller for executing bulk actions to start, stop, or delete groups of tracked download tasks simultaneously. The project covers real-time progress monitori

Async is a JavaScript asynchronous flow library designed to manage the execution and coordination of asynchronous tasks in Node.js and the browser. It provides functional utilities to wrap, process, and orchestrate complex asynchronous workflows. The library distinguishes itself through a comprehensive task orchestrator that handles dependency graphs to resolve circular references and manages concurrent task queues. It includes a unification bridge that allows callback-style and promise-based functions to operate within the same execution interface. The project covers several primary capabil

Failsafe is a JVM fault tolerance library and resilience pattern framework. It provides a toolkit for implementing circuit breakers, rate limiters, and other stability patterns within Java Virtual Machine applications to prevent cascading failures in distributed systems. The project is distinguished by its policy-based execution pipeline, which allows for the composition of multiple resilience patterns into a sequential flow. It features a state-machine circuit breaker to manage service recovery and a leaky-bucket rate limiter to control operation frequency. The library covers a broad range

This project is a comprehensive performance programming guide and reference for the Go language, focusing on runtime efficiency and memory optimization. It provides a collection of patterns and techniques designed to increase execution speed by reducing garbage collection overhead and optimizing memory usage. The resource distinguishes itself through detailed reference implementations for memory optimization, such as escape analysis, object pooling, and structure memory alignment. It offers specific strategies for reducing binary size and improving CPU cache efficiency through structure memor

Slurm is a cluster workload manager and job scheduler designed for high-performance computing environments. It functions as a distributed compute orchestrator that queues and executes large-scale computational tasks across multiple compute nodes in a cluster. The system acts as a resource arbitrator, distributing hardware nodes and processors among concurrent users to prevent resource conflicts and maximize efficiency. It coordinates the simultaneous launch of multiple processes across different physical servers to execute parallel jobs and scientific workloads. The platform covers broad cap

Pueue is a task queue manager for shell commands, built as a daemon and command-line interface. It accepts shell commands into a managed queue and executes them with configurable parallel execution limits, supporting both global and per-group concurrency controls. The daemon persists its entire state—task queue, logs, and configuration—to disk, ensuring survival across crashes and system restarts. The project distinguishes itself through a dependency graph that lets tasks declare prerequisites, forming a directed acyclic graph that controls execution order. Tasks can be organized into named g

Hatchet is an open-source durable workflow engine and task orchestration platform. It provides a framework for building and executing fault-tolerant, multi-step pipelines as directed acyclic graphs (DAGs), with automatic retries, scheduling, and real-time observability. The system is built around durable task checkpointing, which persists execution state after each step so work can resume from the last checkpoint after a worker crash or restart, and it supports event-driven task resumption that pauses a task until a matching external event arrives. The platform distinguishes itself through it

Ants is a goroutine pool library for Go that functions as a concurrent task orchestrator and resource consumption limiter. Its primary purpose is to manage and reuse a pool of goroutines to limit concurrency and reduce the memory allocation overhead associated with frequent thread creation. The system provides a runtime worker manager capable of adjusting pool capacity dynamically to respond to fluctuating workloads. To maintain stability, it includes panic recovery mechanisms that intercept runtime failures within worker threads to prevent a single failing task from crashing the entire appli

This project is an instructional repository and learning resource providing a comprehensive collection of study notes, programming guides, and runnable source code examples. It serves as a guide for Java SE and Java web development, featuring a library of code samples that demonstrate common algorithms and technical implementations. The repository focuses on language core and runtime analysis, with detailed tutorials on multithreading and concurrency. It provides practical implementations of software architecture and design patterns, such as the singleton and strategy patterns, as well as exp