Open-source alternatives to Pgmq

Queue Classic is a background processing framework for Ruby applications that manages asynchronous tasks by utilizing relational database tables for job persistence. By storing tasks directly within the database, the system ensures that job creation remains coupled with application transactions, guaranteeing that tasks are only queued when associated data changes are successfully committed. The framework coordinates concurrent worker processes through database-level locking mechanisms, which prevent redundant execution and allow for distributed task processing without the need for an external

Dramatiq is a distributed task queue and workload manager used to offload function execution to background workers. It functions as an asynchronous task orchestrator that enables the distribution of computational tasks across a cluster using a pluggable transport layer supporting RabbitMQ and Redis. The framework provides specialized tools for complex task orchestration, including the ability to link background jobs into sequences, pipelines, and barriers. It further manages distributed concurrency through the use of shared mutexes, rate limiters, and exponential backoff retries to prevent re

Agenda is a persistent background job scheduler and distributed task runner for Node.js applications. It functions as a cron job manager and task queue that ensures background processes survive application restarts by storing job state and metadata in a database. The system coordinates execution across multiple worker instances using distributed locking mechanisms to prevent duplicate processing. It supports flexible scheduling via cron expressions or specific dates and includes a pluggable storage interface for backends such as MongoDB, PostgreSQL, and Redis. The platform provides controls

Bull is a Node.js library for managing distributed jobs and message queues using Redis as the primary data store. It functions as a distributed task worker, job scheduler, and priority queue manager designed to handle asynchronous workloads across multiple processes. The project distinguishes itself by providing a persistent communication channel that decouples servers through the exchange of serializable data objects. It ensures distributed system reliability by detecting stalled tasks and recovering from process crashes to ensure every queued job is completed. The system covers a broad ran

River is a transactional job queue and distributed job scheduler for Go that uses PostgreSQL for persistence and state management. It functions as a resumable task framework, allowing long-running background work to be broken into persisted steps that can resume from the last saved checkpoint after a failure. The system ensures strict data consistency by allowing background tasks to be enqueued and completed within the same database transaction as the primary application data. It distinguishes itself through a coordinator model that employs leader election to manage periodic and delayed tasks

This is a Go client library and protocol implementation for interacting with message brokers using the AMQP 0.9.1 standard. It serves as an asynchronous messaging toolkit and message queue client for implementing producer and consumer patterns. The library provides a programmatic interface for managing the full message lifecycle, including publishing, consumption, and delivery acknowledgments. It supports reliable delivery mechanisms such as publishing confirmations and atomic transactions. The project covers the administration of routing topologies through the declaration and binding of que

SpringCloud-Learning is an educational project that demonstrates how to build microservices using Spring Cloud, covering the core patterns of service discovery, centralized configuration management, and API gateway routing. The project provides hands-on examples for registering and discovering microservice instances with Nacos, Eureka, or Consul, and for routing external API requests through Spring Cloud Gateway with support for filters and load balancing. The tutorials explore service resilience through circuit breakers and rate limiting with Sentinel and Hystrix, including custom fallback l

This project is a Model Context Protocol server and AI agent database connector. It provides a standardized communication layer that allows language models to interact with relational data stores, read database schemas, and manage PostgreSQL database resources. The implementation acts as a serverless host for the Model Context Protocol, deploying on distributed edge functions to connect AI assistants to a project. This enables AI agents to perform database administration, execute SQL queries, and handle schema migrations through an AI-compatible interface. The system covers broader capabilit

Osmedeus is a security workflow orchestration engine that coordinates AI agents, shell commands, and scanning tools through declarative YAML pipelines. It functions as a distributed security scanner, a declarative workflow automator, and an AI agent framework for security, enabling automated multi-step security analysis with conditional branching, parallel execution, and distributed workers. The engine distinguishes itself through a hybrid runner model that executes workflow steps on the local host, inside Docker containers, or over SSH to remote machines, selected per step or module. It supp

libzmq is a C++ based asynchronous messaging engine and networking core designed for routing non-blocking messages between distributed nodes. It functions as a distributed message queue that implements the ZMTP wire-format framing protocol to standardize how data moves across different network transport layers. The library provides a multi-transport abstraction that allows a single interface to route data across TCP, IPC, and in-process memory. It incorporates a cryptographic layer to encrypt and authenticate transmissions between nodes and employs topology-based messaging patterns, such as p

Resque is a Ruby library for enqueueing and processing asynchronous tasks using Redis as a data store. It functions as a distributed task processor and queue manager, allowing long-running work to be moved out of the main request cycle. The system executes background jobs in isolated child processes to prevent memory leaks and provides a web-based dashboard for monitoring queue depths, worker activity, and failed job statistics. Capability areas include distributed worker coordination via signals, error handling with job retry mechanisms, and priority-ordered queue management. It also suppor

Iggy is a distributed message streaming platform and multi-protocol message broker that functions as a persistent distributed log store. It provides infrastructure for publishing and consuming binary messages using an append-only log, ensuring high availability and data consistency across nodes through Viewstamped Replication. The platform is distinguished by its specialized LLM streaming infrastructure, which uses a server protocol to connect large language models to streaming data and system controls. This includes standardized protocols for context management and data bridging via HTTP or

Machinery is a distributed task queue and asynchronous workflow engine. It provides a system for processing heavy workloads outside the main request flow using a network of distributed background workers and a message-based job orchestrator. The project manages complex task lifecycles through sequential chaining, where results are passed between tasks, and parallel coordination, which can trigger callback tasks upon the completion of a group. It supports periodic workflow scheduling for recurring jobs and delayed execution via specific timestamps. The system includes capabilities for result

Airflow is a workflow orchestration platform for authoring, scheduling, and monitoring complex data pipelines as code using Python. It employs a DAG-based task scheduler to manage execution timing and dependencies via directed acyclic graphs, utilizing a distributed task execution engine to run workloads across a cluster of worker nodes. The platform provides a data pipeline monitor for tracking the health and execution history of programmatic workflows. This includes a web interface for workflow progress visualization and health monitoring to identify and troubleshoot pipeline failures. The

Minigo is a TensorFlow-based reinforcement learning engine designed to master the game of Go. It functions as a comprehensive system for training neural networks to predict board policies and game outcomes, utilizing a model trainer to generate self-play data and optimize weights. The project is distinguished by its ability to perform large-scale game simulations using Kubernetes to distribute worker nodes across CPU, GPU, and TPU hardware. It employs a Monte Carlo Tree Search implementation to identify optimal moves and supports specialized hardware acceleration, including inference on Edge

Gorush is a dedicated notification dispatch server used to send real-time alerts to iOS and Android mobile devices. It functions as a centralized infrastructure for hosting a notification API and managing the delivery of push messages through high-throughput queuing. The system supports both gRPC streams and HTTP endpoints to receive notification triggers and includes a command-line interface for triggering alerts without a running server process. It manages network traffic through proxy routing via HTTP or SOCKS5 and secures server communication using automated TLS certificate management. O

This is a distributed voting application designed to demonstrate a multi-service architecture. It uses stateless web frontends for submitting votes and viewing live results, with a Redis-backed queue to buffer incoming votes and a PostgreSQL database for persistent tallying. The application is built around asynchronous message queue processing, decoupling the vote submission from the tallying workflow. The project showcases how to deploy a multi-service application using container orchestration tools. It provides YAML-driven declarative deployment manifests for Docker Compose, Docker Swarm, a

This project is a set of hands-on labs for practicing cloud development, focusing on implementing web apps, functions, storage solutions, and containerized workloads. It provides a practical framework for developing solutions within the Azure ecosystem. The content covers a wide range of specialized cloud capabilities, including serverless development with HTTP and timer triggers, container orchestration using apps and instances, and API management for routing and transforming traffic. It also emphasizes identity and access management through OpenID Connect and managed identities. Additional

Cellnet is a Go networking framework designed for building distributed systems and peer-to-peer networks. It provides a multi-protocol transport layer that standardizes message exchange across TCP, UDP, and WebSockets, utilizing a binary message codec system to map message types to unique identifiers for network transmission. The framework is distinguished by its remote procedure call library, which enables the execution of functions on remote peers using synchronous or asynchronous request patterns. It further differentiates itself with an asynchronous event dispatcher that processes network

SwiftMessages is an iOS notification bar library and in-app message queue manager. It provides a system for displaying customizable alerts and status messages at the top, bottom, or center of the screen. The library features a hosting bridge that allows declarative SwiftUI views to be rendered within imperative UIKit containers. It includes a queue management system that sequences chronological notifications and filters duplicate alerts to prevent interface overlap. The system handles dynamic layout management to avoid occlusion by the system keyboard and provides an accessibility interface

KafkaJS is a pure JavaScript client for Apache Kafka, providing the necessary tools to produce and consume messages from a Kafka cluster without requiring native dependencies or external addons. It functions as a comprehensive integration library for Node.js applications to engage in distributed message processing and real-time event streaming. The project is distinguished by its native implementation of the Kafka wire protocol, avoiding C++ dependencies. It features a security client supporting SSL, TLS, and SASL authentication, alongside transactional capabilities that allow for atomic mess

This platform serves as a centralized dashboard for collecting, visualizing, and analyzing personal activity logs and timestamped life events. It functions as a unified repository that aggregates disparate digital records into a single store, enabling long-term historical tracking and personal data analysis. The system distinguishes itself through a modular report composition engine that groups specific event datasets and visual elements into reusable structures. It incorporates an automated alerting engine that monitors incoming data streams against predefined thresholds, triggering notifica

Sidekiq is a background job processor and queue manager for Ruby that uses Redis to manage asynchronous tasks. It functions as a distributed task scheduler capable of handling periodic, delayed, and recurring jobs across a cluster of worker processes. The project features a job monitoring dashboard and administrative web interface for visualizing system state, tracking worker performance, and managing failed or dead jobs. It provides a distributed rate limiter to control execution frequency across multiple processes. The framework covers a broad range of operational capabilities, including j

Hyperopt is a Python library for hyperparameter optimization designed to minimize scalar-valued objective functions. It operates as a stochastic search space engine that finds optimal input parameters by searching through real-valued, discrete, and conditional spaces. The framework distinguishes itself through its support for complex search space configurations, allowing for conditional parameter hierarchies where specific hyperparameters are sampled only if their parent parameters meet certain criteria. It is built as an asynchronous optimization framework, decoupling the generation of searc

This project is an edge computing development toolkit and serverless command line interface used to develop, test, and deploy serverless functions to a global edge network. It serves as an edge runtime bundler and resource orchestrator, managing the entire lifecycle of edge projects from local development to worldwide distribution. The toolkit distinguishes itself through distributed workflow management, coordinating stateful instances and the durable execution of long-running processes across the edge. It also provides specialized integrations for edge AI, including the management of vector

OpenCTI is a cyber threat intelligence platform and knowledge base used to store, manage, and analyze technical security data. It functions as a threat intelligence visualization tool and an enterprise security data orchestrator that maps relationships between threat actors, malware, and vulnerabilities. The platform utilizes the STIX and TAXII standards for data representation and exchange, allowing for the sharing and receiving of standardized intelligence bundles. It distinguishes itself by converting complex security information into visual relationship diagrams and geographic maps to ide

This project is a comprehensive knowledge base and study resource designed for mastering technical interviews. It provides structured guides, roadmaps, and curricula focused on data structures, algorithms, system design, and frontend engineering to help candidates prepare for software engineering screenings. The repository distinguishes itself by offering a holistic approach to professional advancement. Beyond technical drills, it includes a career development handbook covering resume optimization, salary benchmarking, and strategic negotiation coaching. It also provides detailed methodologie