awesome-repositories.com
Blog
MCP
awesome-repositories.com

Discover the best open-source repositories with AI-powered search.

ExploreCurated searchesOpen-source alternativesSelf-hosted softwareBlogSitemap
ProjectMCP serverAboutHow we rankPress
LegalPrivacyTerms
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
beanstalkd avatar

beanstalkd/beanstalkd

0
View on GitHub↗
beanstalkd.github.io↗

Beanstalkd

Beanstalkd is a distributed work queue server designed for asynchronous job processing. It functions as a standalone server that distributes background tasks between producers and consumers to improve application responsiveness and throughput.

The system organizes tasks using numeric priority levels to ensure critical work is processed first. It manages the job lifecycle through discrete states and uses a simple line-based text protocol over TCP for communication.

To ensure reliability, the server persists job data to a sequential disk log, allowing the queue state to be recovered after a system crash or restart. It also employs timeout-based job reservations to prevent multiple workers from processing the same task simultaneously.

AI search

Explore more awesome repositories

Describe what you need in plain English — the AI ranks thousands of curated open-source projects by relevance.

Start searching with AI

Features

  • Background Job Queues - Provides a fast distributed work queue for asynchronous task processing between producers and consumers.
  • Distributed Task Queues - Provides a distributed system for coordinating background tasks between producers and consumers.
  • In-Memory Queues - Maintains the primary work queue in RAM for high-performance job retrieval and updates.
  • In-Memory Databases with Persistence - Prioritizes in-memory processing for speed while utilizing a disk-based log for durability.
  • Asynchronous Task Processing - Enables offloading of heavy work to background workers to keep the main application responsive.
  • Custom TCP Protocols - Uses a custom TCP-based request protocol for efficient communication between clients and the server.
  • Priority-Based Job Schedulers - Provides a scheduling system that orders tasks by numeric priority to ensure critical work is processed first.
  • Filesystem-Backed Queues - Implements a task queue that uses a sequential disk log for persistence and crash recovery.
  • Standalone Servers - Functions as a standalone server that manages job priorities and state for distributed environments.
  • Text-Based Communication Protocols - Implements a simple line-based text communication format over TCP for low-overhead interactions.
  • Background Job Processors - Functions as a distributed background job processor that offloads heavy tasks to maintain responsiveness.
  • Priority Queues - Uses priority queues to organize tasks, ensuring higher-priority jobs are dequeued before lower-priority ones.
  • Task Ordering - Implements numeric priority levels to ensure high-priority jobs are processed before lower-priority ones.
  • Background Job Schedulers - Manages the organization and execution of background tasks independently of the main application process.
  • Append-Only Persistence - Employs append-only sequential logging to disk for durable recovery of the queue state.
  • Disk-Backed Data Structures - Implements a disk-backed storage mechanism for the work queue to ensure state recovery after system crashes.
  • Message Queue Integrations - Acts as a message broker to decouple system components via a durable task queue.
  • Job Queue Persistence - Persists job data to disk to ensure work is not lost during server restarts or system crashes.
  • Job State Tracking - Tracks the lifecycle of jobs through distinct states such as ready, reserved, and buried.
  • Lifecycle State Machines - Implements a formal state machine to manage transitions between job states for reliable workflow and retry logic.
  • Text-Based TCP Protocols - Uses a simple line-based text protocol over TCP for efficient communication between clients and the server.
  • Distributed Job Locking - Prevents duplicate processing by locking jobs for specific consumers for a set duration.
  • Message Queuing - Simple and fast work queue for background jobs.
6,697 stars·867 forks·C·20 views

Star history

Star history chart for beanstalkd/beanstalkdStar history chart for beanstalkd/beanstalkd

How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.

Projects sharing features with Beanstalkd

These projects share indexed features with Beanstalkd. Shared tags can include platform or build tooling; verify the primary use case before treating a result as a replacement.
  • rq/rqrq avatar

    rq/rq

    10,653View on GitHub↗

    rq is a distributed task queue and background worker system for Python that uses a Redis backend to decouple task submission from execution. It functions as a reliable message queue and task scheduler, allowing Python functions or asyncio coroutines to be processed asynchronously across multiple worker processes. The project distinguishes itself through reliable queuing mechanisms that prevent job loss during worker crashes using atomic operations. It provides specialized orchestration capabilities, including the prevention of duplicate jobs, job execution prioritization, and the ability to m

    Pythonasyncbackground-jobsdelayed-jobs
    View on GitHub↗10,653
  • bogdanp/dramatiqBogdanp avatar

    Bogdanp/dramatiq

    5,136View on GitHub↗

    Dramatiq is a distributed task queue and workload manager used to offload function execution to background workers. It functions as an asynchronous task orchestrator that enables the distribution of computational tasks across a cluster using a pluggable transport layer supporting RabbitMQ and Redis. The framework provides specialized tools for complex task orchestration, including the ability to link background jobs into sequences, pipelines, and barriers. It further manages distributed concurrency through the use of shared mutexes, rate limiters, and exponential backoff retries to prevent re

    Pythondistributed-lockpythonrabbit
    View on GitHub↗5,136
  • richardknop/machineryRichardKnop avatar

    RichardKnop/machinery

    7,956View on GitHub↗

    Machinery is a distributed task queue and asynchronous workflow engine. It provides a system for processing heavy workloads outside the main request flow using a network of distributed background workers and a message-based job orchestrator. The project manages complex task lifecycles through sequential chaining, where results are passed between tasks, and parallel coordination, which can trigger callback tasks upon the completion of a group. It supports periodic workflow scheduling for recurring jobs and delayed execution via specific timestamps. The system includes capabilities for result

    Goamqpaws-sqsgo
    View on GitHub↗7,956
  • timgit/pg-bosstimgit avatar

    timgit/pg-boss

    3,218View on GitHub↗

    pg-boss is a background task scheduler and distributed task queue that uses PostgreSQL as a reliable message broker for asynchronous job processing. It provides a system for distributing work across multiple application instances, ensuring exactly-once delivery through atomic database transactions. The project includes a cron job scheduler for automating recurring tasks and a PostgreSQL pub-sub system for fan-out event distribution. It also features a web-based management dashboard for monitoring queue statistics and controlling job lifecycles, including manual retries and cancellations. Cap

    TypeScriptjobnodenodejs
    View on GitHub↗3,218
Compare all 30 related projects→

Frequently asked questions

What does beanstalkd/beanstalkd do?

Beanstalkd is a distributed work queue server designed for asynchronous job processing. It functions as a standalone server that distributes background tasks between producers and consumers to improve application responsiveness and throughput.

What are the main features of beanstalkd/beanstalkd?

The main features of beanstalkd/beanstalkd are: Background Job Queues, Distributed Task Queues, In-Memory Queues, In-Memory Databases with Persistence, Asynchronous Task Processing, Custom TCP Protocols, Priority-Based Job Schedulers, Filesystem-Backed Queues.

Which projects share features with beanstalkd/beanstalkd?

Projects with overlapping indexed features include: rq/rq — rq is a distributed task queue and background worker system for Python that uses a Redis backend to decouple task… bogdanp/dramatiq — Dramatiq is a distributed task queue and workload manager used to offload function execution to background workers. It… richardknop/machinery — Machinery is a distributed task queue and asynchronous workflow engine. It provides a system for processing heavy… timgit/pg-boss — pg-boss is a background task scheduler and distributed task queue that uses PostgreSQL as a reliable message broker… resque/resque — Resque is a Ruby library for enqueueing and processing asynchronous tasks using Redis as a data store. It functions as… coleifer/huey — .. image:: https://media.charlesleifer.com/blog/photos/huey3-logo.png.