Awesome GitHub RepositoriesJob Queues

This project is a community-driven directory that aggregates essential software projects and educational content for the Node.js ecosystem. It functions as a centralized knowledge base and discovery index, designed to simplify the navigation of a fragmented technical landscape by providing a structured collection of high-quality links, tools, and learning materials. The repository distinguishes itself through a decentralized, peer-reviewed curation model. By utilizing standard version control workflows and pull requests, the community ensures that all listed resources undergo human verificati

Bull is a Node.js library for managing distributed jobs and message queues using Redis as the primary data store. It functions as a distributed task worker, job scheduler, and priority queue manager designed to handle asynchronous workloads across multiple processes. The project distinguishes itself by providing a persistent communication channel that decouples servers through the exchange of serializable data objects. It ensures distributed system reliability by detecting stalled tasks and recovering from process crashes to ensure every queued job is completed. The system covers a broad ran

This tool is a command-line utility designed for automated web resource discovery, fuzzing, and application structure mapping. It functions as a security-focused scanner that identifies hidden files, directories, parameters, and virtual hosts by injecting payloads into HTTP requests. By systematically testing how servers handle various inputs, it assists in mapping the architecture of web applications and uncovering potential security vulnerabilities. The tool distinguishes itself through a highly concurrent engine that manages asynchronous request execution and recursive job orchestration. I

algorithm-base is an educational library and study guide designed for simulating algorithms and studying data structures. It functions as an execution visualizer that renders step-by-step state changes and pointer updates through animated simulations to illustrate how data movement works. The project distinguishes itself by mapping conceptual logic directly to multi-language source code implementations. It utilizes a comparative analysis framework to evaluate different algorithmic strategies based on stability, time complexity, and space complexity, while organizing problems by underlying mec

Inbox Zero is an AI-powered email automation platform and inbox organizer. It uses large language models to automatically categorize, label, and archive emails, while providing a conversational interface for managing workflows and drafting responses through natural language. The project distinguishes itself by integrating real-time calendar availability into its drafting process and generating AI-summarized meeting briefings. It supports a pluggable AI provider interface with model fallback chains, allowing it to connect to various cloud or local LLM providers. Users can also control their in

Hangfire is a background job scheduler and distributed task queue for .NET applications. It serves as a job orchestration framework that offloads heavy processing to background workers using a SQL-backed processor to manage job state across multiple servers. The framework distinguishes itself through reliable task scheduling, where job metadata and arguments are persisted in an external database to ensure tasks survive application restarts. It supports advanced orchestration patterns, including the ability to chain dependent tasks so that a child job triggers automatically upon the successful

Redis is a high-performance in-memory key-value store that functions as a distributed cache, message broker, and NoSQL database. It provides sub-millisecond read and write access to data stored in RAM and can operate as a vector database for indexing high-dimensional embeddings. The system supports a wide range of data storage and synchronization primitives, including the management of strings, hashes, lists, sets, and JSON documents. It enables real-time data operations through atomic transactions, hybrid persistence using snapshots and append-only logs, and high-availability configurations

Romm is a self-hosted game library manager and ROM management web interface. It serves as a central server for storing and categorizing game files and emulator firmware, providing a web-based browser to organize collections through automated library scanning and metadata retrieval. The project distinguishes itself by integrating a web-based emulator frontend that uses WebAssembly to play games directly in the browser. It further provides a game save synchronization server that uses SSH-based synchronization to transfer save states and progress between the server and registered handheld device

ytdlnis is a mobile application that serves as a graphical client for the yt-dlp engine on Android. It functions as a media downloader and manager, providing a user interface to retrieve video and audio from websites. The project distinguishes itself by integrating directly with the Android system share menu and intents to trigger background downloads from external apps. It includes a dedicated authentication cookie manager to import and sync browser session data, enabling the retrieval of private, age-restricted, or premium content. The application covers broad capability areas including au

Databasus is a self-hosted backup platform that automates PostgreSQL backups, verifies their restorability, and stores them across multiple destinations while managing team access with role-based permissions. It combines on-the-fly AES-256-GCM encryption, cron-driven scheduling, job-queue-based verification, multi-destination storage, WAL streaming, throwaway container restore testing, and workspace-based role access control into a unified backup system. The platform distinguishes itself through automatic backup verification that restores each backup into a temporary database container for in

Persepolis is a Python-based download manager and graphical user interface for the aria2 engine. It functions as a concurrent file downloader that splits files into multiple chunks to be retrieved simultaneously, maximizing available network bandwidth. The application integrates a media extraction engine to resolve direct URLs, enabling the retrieval of video and audio content from various online streaming platforms and hosting services. It provides tools for download queue management and automation, including the ability to schedule specific start times or define time windows for downloads

Scraperr एक सेल्फ-होस्टेड वेब स्क्रैपिंग और क्रॉलिंग प्लेटफॉर्म है जिसे XPath सिलेक्टर्स का उपयोग करके वेबसाइटों से स्ट्रक्चर्ड डेटा निकालने के लिए डिज़ाइन किया गया है। यह एक कतार (queue) के माध्यम से स्क्रैपिंग जॉब्स को प्रबंधित करने और आर्टिफिशियल इंटेलिजेंस का उपयोग करके परिणामी कंटेंट का विश्लेषण करने के लिए एक कंटेनराइज्ड सिस्टम के रूप में कार्य करता है। यह प्रोजेक्ट अपने Kubernetes-नेटिव आर्किटेक्चर के माध्यम से खुद को अलग करता है, जो पैकेज मैनेजर के माध्यम से स्केलेबल डिप्लॉयमेंट और प्रबंधन की अनुमति देता है। इसमें लिंक्ड पेजों को खोजने के लिए डोमेन-लेवल स्पाइडरिंग में सक्षम एक क्रॉलिंग इंजन और निकाले गए वेब कंटेंट को क्वेरी करने के लिए आर्टिफिशियल इंटेलिजेंस का उपयोग करने वाला एक डेटा एनालाइजर शामिल है। यह प्लेटफॉर्म ऑटोमेटेड डेटा एक्सट्रैक्शन, बल्क वेब क्रॉलिंग और मीडिया फाइल डाउनलोडिंग सहित क्षमताओं की एक विस्तृत श्रृंखला को कवर करता है। यह स्क्रैप किए गए डेटा को टेबल में विज़ुअलाइज़ करने, ब्राउज़र पहचान की नकल करने के लिए कस्टम रिक्वेस्ट हेडर कॉन्फ़िगर करने और परिणामों को CSV या Markdown प्रारूपों में निर्यात करने के लिए टूल्स प्रदान करता है। एप्लिकेशन Kubernetes डिप्लॉयमेंट कॉन्फ़िगरेशन के माध्यम से अनुकूलन योग्य इंस्टॉलेशन पैरामीटर और वर्जन अपडेट का समर्थन करता है।

ace-step-ui is an AI music production workspace and interface for generating, editing, and organizing synthetic audio tracks and vocals. It provides a technical control panel for managing prompts, seeds, and style parameters to produce high-quality audio. The project includes a digital audio workstation interface for trimming and fading files, alongside an audio stem separation tool that splits mixed tracks into individual components such as drums, bass, and vocals. It also features a music video creator for generating visual content and procedural album art to accompany generated music. The

Bee-queue एक Node.js बैकग्राउंड प्रोसेसिंग सिस्टम है जो जॉब क्यूइंग और पर्सिस्टेंस के लिए Redis का उपयोग करता है। इसे एप्लिकेशन रिस्पॉन्सिवनेस बनाए रखने के लिए मुख्य निष्पादन थ्रेड से भारी कार्यों को बैकग्राउंड वर्कर्स पर ऑफलोड करने के लिए डिज़ाइन किया गया है। यह प्रोजेक्ट डिस्ट्रीब्यूटेड जॉब प्रोसेसिंग प्रदान करता है, जिससे वर्कर नोड्स को एक साथ बड़ी मात्रा में कार्यों को संभालने के लिए मल्टीपल प्रोसेसेस में चलने की अनुमति मिलती है। यह ऑटोमैटिक रिट्रायल्स और स्टॉल्ड प्रोसेसेस की रिकवरी के माध्यम से विश्वसनीय कार्य निष्पादन सुनिश्चित करता है। इसकी क्षमता सतह में विलंबित जॉब्स के लिए एसिंक्रोनस टास्क शेड्यूलिंग, वर्कर नोड्स के लिए कॉनकरेंसी कंट्रोल और जॉब लाइफसाइकिल मैनेजमेंट शामिल है। इसमें क्यू हेल्थ की निगरानी, जॉब प्रोग्रेस को ट्रैक करने और जॉब स्टेट के आधार पर परिणाम प्राप्त करने के लिए टूल्स शामिल हैं। यह सिस्टम नेटवर्क ओवरहेड को कम करने के लिए बल्क जॉब एनक्यूइंग को सपोर्ट करता है और कस्टम जॉब आइडेंटिफायर्स व विफल कार्यों के लिए कॉन्फ़िगर करने योग्य बैकऑफ रणनीतियों की अनुमति देता है।

pg-boss is a background task scheduler and distributed task queue that uses PostgreSQL as a reliable message broker for asynchronous job processing. It provides a system for distributing work across multiple application instances, ensuring exactly-once delivery through atomic database transactions. The project includes a cron job scheduler for automating recurring tasks and a PostgreSQL pub-sub system for fan-out event distribution. It also features a web-based management dashboard for monitoring queue statistics and controlling job lifecycles, including manual retries and cancellations. Cap

Bull-board is a web-based administrative interface and dashboard for visualizing, monitoring, and managing asynchronous job queues in Node.js. It serves as a control panel for background job administrators to oversee queue processing and manage the lifecycle of tasks powered by BullMQ, Bull, and Redis. The project distinguishes itself by providing a pluggable interface that integrates with various web server frameworks as middleware. It utilizes an adapter-based system to provide unified administrative controls across different queue backends and includes authentication guards to secure the d

AnyCrawl is an AI-powered data extractor, automated web crawler, and headless browser orchestrator. It serves as a web content extraction API and a gateway that connects crawling and scraping tools to language models using a standardized API protocol. The project specializes in converting unstructured website content into structured JSON or markdown optimized for AI assistants. It utilizes language models and JSON schemas to pull specific information into validated formats and provides capabilities for AI page summarization and LLM-optimized content extraction. The system manages comprehensi