11 مستودعات
Architectures for decoupling data capture from intensive processing via queues.
Distinguishing note: Focuses on queue-based task management, not general message brokers.
Explore 11 awesome GitHub repositories matching development tools & productivity · Task Queuing Systems. Refine with filters or upvote what's useful.
Claude-mem is an agentic memory persistence system designed to provide AI assistants with long-term context across multiple development sessions. It functions as a background orchestrator that captures, summarizes, and indexes interaction history, allowing models to maintain continuity and recall technical decisions from past tasks. By utilizing a vector-augmented context engine, the system injects relevant historical observations into active sessions, ensuring that AI agents remain informed without exceeding finite token budgets. The project distinguishes itself through an endless memory arc
Decouples data capture from intensive processing using a queue-based architecture to handle parallel events and retries.
Ultimate Vocal Remover is a desktop application designed for AI-driven audio source separation. It utilizes deep learning models to isolate vocals, drums, and other individual instruments from mixed audio files, providing a utility for professional production and creative editing workflows. The software distinguishes itself by leveraging GPU-accelerated tensor computation to perform complex signal processing tasks, significantly reducing the time required for high-fidelity audio extraction. It incorporates a modular plugin architecture that integrates external utilities to support a wide rang
Manages sequential execution of audio processing tasks through a queue to maintain interface responsiveness.
Olmocr is a distributed document processing framework designed to convert PDF and image files into structured markdown. It functions as a vision-based document parser that utilizes multimodal neural networks to interpret complex visual layouts and translate them into standardized text representations. The system operates as a remote inference orchestrator, offloading heavy document analysis tasks to external servers or cloud APIs to minimize local computational requirements. By employing a stateless worker architecture, it decouples document ingestion from inference, allowing for the distribu
Coordinates distributed processing by using cloud storage buckets as a shared message bus for task assignment.
Asynq is a distributed background job processing framework for Go applications. It manages asynchronous task queues by offloading heavy operations to persistent storage, allowing the main application to remain responsive while background workers handle workloads. The system utilizes Redis to manage task state, concurrency, and message distribution across multiple worker instances. It employs atomic Lua scripting and sorted sets to ensure reliable job acquisition, precise scheduling of delayed tasks, and fault-tolerant processing through a two-stage acknowledgement flow. The framework support
Manages asynchronous work queues using Redis as a persistent storage layer to ensure reliable task delivery.
AutoMQ is a cloud-native streaming platform and Apache Kafka distribution that implements a decoupled compute and storage architecture. It functions as an S3-backed message queue, using object storage as the primary log repository to eliminate dependencies on local disks. The platform utilizes a stateless broker architecture to enable dynamic compute scaling and automated partition balancing. This design allows the system to adjust the number of brokers in seconds and distribute network traffic without requiring manual data migration or partition reassignment. The system provides multi-avail
Implements a distributed message queue that uses S3 object storage as the primary log repository to eliminate local disk dependencies.
Gyroflow is a gyroscope video stabilization software and IMU telemetry processor designed to remove camera shake from video files. It functions as a hardware-accelerated video renderer and lens calibration tool, utilizing embedded or external gyroscope and accelerometer data to perform pixel-level stabilization. The system is distinguished by its ability to integrate with professional non-linear video editing software via plugins, allowing stabilization to be applied directly to timelines without transcoding original footage. It supports diverse telemetry ingestion from camera brands, flight
Implements a task queuing system to process multiple stabilization export jobs sequentially.
Osmedeus is a security workflow orchestration engine that coordinates AI agents, shell commands, and scanning tools through declarative YAML pipelines. It functions as a distributed security scanner, a declarative workflow automator, and an AI agent framework for security, enabling automated multi-step security analysis with conditional branching, parallel execution, and distributed workers. The engine distinguishes itself through a hybrid runner model that executes workflow steps on the local host, inside Docker containers, or over SSH to remote machines, selected per step or module. It supp
Provides CLI subcommands to list and control deferred workflow tasks in a background queue.
Pueue is a task queue manager for shell commands, built as a daemon and command-line interface. It accepts shell commands into a managed queue and executes them with configurable parallel execution limits, supporting both global and per-group concurrency controls. The daemon persists its entire state—task queue, logs, and configuration—to disk, ensuring survival across crashes and system restarts. The project distinguishes itself through a dependency graph that lets tasks declare prerequisites, forming a directed acyclic graph that controls execution order. Tasks can be organized into named g
Modifies the command, working directory, or label of a task before it starts.
PraisonAI is an autonomous AI agent platform that coordinates multiple LLM-powered agents for research, planning, and execution of complex workflows. It functions as a multi-agent orchestration framework, a workflow builder, and a Model Context Protocol server, while also providing retrieval-augmented generation through vector knowledge bases. Agents can interact via CLI, web, or standardized protocols with sandboxed code execution. The platform distinguishes itself with a rich set of agent communication protocols, including A2A, REST, WebSocket, voice and telephony integration, and MCP, allo
Queues and processes prompts sequentially when the agent is busy.
TacticalRMM is a remote monitoring and management platform designed for overseeing endpoints and automating IT administration. It functions as an endpoint management tool and IT automation framework, providing a centralized dashboard for executing scripts, monitoring system health, and managing remote devices across multiple tenants. The platform distinguishes itself through a comprehensive remote administration suite that includes real-time shell access, remote file management, and registry editing. It integrates with third-party remote desktop software and provides a hierarchical policy inh
Queues scripts and tasks on the server to be executed by endpoints immediately upon their next connection.
This project is a web scraping and automation tool designed to collect public data from Instagram and perform automated social media interactions. It functions by gathering profile details, captions, media files, and engagement metrics directly from web pages, bypassing the need for official developer interfaces or platform-specific credentials. The tool distinguishes itself by combining data extraction with automated engagement capabilities. It allows users to programmatically interact with content by liking posts that match specific search criteria or hashtags, aiming to increase account vi
Manages concurrent scraping operations by distributing work across a pool of workers to improve throughput and handle rate limits.