pgmq/pgmq

Features

• Postgres Message Queueing - Implements a full asynchronous task processing system as a PostgreSQL extension using the database as the broker.
• Database-Backed Brokers - A broker that leverages PostgreSQL for persistent storage, atomic operations, and notification-driven message delivery.
• PostgreSQL Task Queues - Provides a lightweight message queuing system implemented as a PostgreSQL extension to manage asynchronous tasks.
• Distributed Task Workers - Manages the delivery and consumption of tasks across multiple distributed worker processes.
• Atomic Queue Transitions - Provides atomic operations to retrieve and delete messages in a single step to ensure at-most-once delivery.
• Message Queues - Provides the primary mechanism for adding data payloads to a queue with optional delivery delays.
• Message Consumption - Implements message retrieval mechanisms including long-polling and visibility timeouts to manage consumer access.
• Message Deletion Operations - Provides capabilities to remove single or batch messages from the queue after successful processing.
• Table-Based Storage - Uses dedicated PostgreSQL tables to store message payloads and metadata for persistent queue management.
• FIFO - Provides a queueing system that ensures strict first-in-first-out ordering of messages using group keys.
• Message Processing Timeouts - Implements visibility timeouts to prevent premature message redelivery while a task is being processed.
• Visibility Timeout Controls - Provides granular control over message visibility timeouts to manage processing attempts and redelivery.
• Atomic Pop Operations - Implements atomic pop operations that combine message retrieval and deletion in one transaction for at-most-once delivery.
• Skip Locked Concurrency - Employs the FOR UPDATE SKIP LOCKED clause to allow multiple consumers to process messages without blocking each other.
• FIFO Group Processing - Supports strict first-in-first-out ordering for related tasks using group keys to prevent race conditions.
• FIFO Task Sequencing - Ensures strict order of operations for related tasks using group identifiers to prevent race conditions.
• Group-Keyed FIFO Ordering - Maintains strict processing sequences for related messages using group keys and index-based sorting.
• Database Insert Notifications - Employs Postgres NOTIFY events to alert consumers of new messages immediately without requiring constant polling.
• Queue Purging - Ships a dedicated operation to clear all pending messages from a queue without deleting the queue itself.
• Dead Letter Queues - Moves repeatedly failing messages to a separate dead-letter queue to prevent processing blocks.
• Queue-Specific Metrics - Provides detailed operational statistics including queue length, message age, and overall throughput.
• Message Requeuing - Implements the ability to re-inject failed messages into the queue by resetting their visibility timeouts.
• Queue Lifecycle Management - Implements administrative tools for creating and deleting the database tables used for queue storage.
• Topic Routing Patterns - Implements flexible message distribution using topics and wildcard patterns for complex data delivery.
• SQS-Compatible Queues - Offers an SQS-compatible queue model featuring visibility timeouts and delayed delivery.
• Database Trigger Notifications - Uses Postgres NOTIFY events to alert consumers of new message inserts without requiring constant polling.
• Queue Throughput Analytics - Provides tools for tracking queue length and message throughput to monitor background processing health.
• Job Queues - Lightweight message queue built on PostgreSQL.

Star history