awesome-repositories.com
Blog
awesome-repositories.com

Entdecke die besten Open-Source-Repositories mit KI-gestützter Suche.

EntdeckenKuratierte SuchenOpen-source alternativesSelf-hosted softwareBlogSitemap
ProjektÜber unsHow we rankPresseMCP-Server
RechtlichesDatenschutzAGB
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
·

79 Repos

Awesome GitHub RepositoriesAsynchronous Task Queues

Frameworks and utilities for managing background job execution and offloading long-running processes.

Distinguishing note: Focuses on the architectural pattern of background task offloading rather than specific message broker implementations.

Explore 79 awesome GitHub repositories matching software engineering & architecture · Asynchronous Task Queues. Refine with filters or upvote what's useful.

Awesome Asynchronous Task Queues GitHub Repositories

Finde die besten Repos mit KI.Wir suchen mit KI nach den am besten passenden Repositories.
  • donnemartin/system-design-primerAvatar von donnemartin

    donnemartin/system-design-primer

    353,387Auf GitHub ansehen↗

    Dieses Projekt ist eine umfassende Bildungsressource und ein Studienleitfaden, der sich auf die Architektur verteilter Systeme und das Design von Backend-Infrastrukturen konzentriert. Es bietet einen strukturierten Lehrplan zur Beherrschung der Prinzipien von Skalierbarkeit, Zuverlässigkeit und Leistung, die für den Entwurf komplexer Softwaresysteme erforderlich sind. Das Repository zeichnet sich durch einen methodischen Ansatz zur Vorbereitung auf technische Vorstellungsgespräche aus, der Entwurfsmuster, architektonische Kompromisse und Tools für räumliche Wiederholungen integriert, um Nutzern das Behalten komplexer Konzepte zu erleichtern. Es betont die einschränkungsgesteuerte Analyse und lehrt Nutzer, wie sie konkurrierende Anforderungen wie Latenz, Konsistenz und Verfügbarkeit beim Entwurf von Architekturen bewerten können. Der Inhalt deckt ein breites Spektrum an Systemdesign-Fähigkeiten ab, einschließlich Strategien für die Datenbankskalierung, Verkehrsmanagement und Infrastrukturoptimierung. Es werden Techniken für horizontale Skalierung, mehrschichtiges Caching, asynchrone Kommunikation und Service-Discovery detailliert beschrieben, während gleichzeitig Frameworks für die Durchführung von Ressourcenschätzungen und Kapazitätsplanungen bereitgestellt werden. Die Dokumentation ist als Studienleitfaden organisiert und bietet einen systematischen Pfad durch die Grundlagen des Backend-Engineerings und des großskaligen Systemdesigns.

    Covers the architectural pattern of using message queues to decouple system components and handle background tasks.

    Pythondesigndesign-patternsdesign-system
    Auf GitHub ansehen↗353,387
  • gradio-app/gradioAvatar von gradio-app

    gradio-app/gradio

    42,931Auf GitHub ansehen↗

    Gradio is a Python library that enables the creation of interactive web applications by converting functions into browser-based interfaces. It functions as a declarative framework where developers define input and output components to automatically generate web forms, visualizations, and data-driven dashboards. By abstracting away manual web markup, the library allows for the rapid construction of interfaces for machine learning models, research demonstrations, and analytical workflows within a single environment. The platform distinguishes itself by automatically exposing internal applicatio

    Offloads long-running tasks and generator functions to background workers to keep the user interface responsive during heavy data processing.

    Pythondata-analysisdata-sciencedata-visualization
    Auf GitHub ansehen↗42,931
  • quantumnous/new-apiAvatar von QuantumNous

    QuantumNous/new-api

    39,722Auf GitHub ansehen↗

    This project is an AI model API gateway and proxy server designed to provide a unified interface for interacting with diverse artificial intelligence service providers. It functions as a centralized middleware platform that routes, load balances, and translates API requests across multiple models, enabling developers to access text, image, audio, and video generation capabilities through a single, standardized integration. The gateway distinguishes itself through comprehensive administrative and financial controls, including event-driven usage accounting, real-time token consumption tracking,

    Manages the lifecycle of long-running operations like video generation through background task queues.

    Goai-gatewayclaudedeepseek
    Auf GitHub ansehen↗39,722
  • openbmb/voxcpmAvatar von OpenBMB

    OpenBMB/VoxCPM

    29,985Auf GitHub ansehen↗

    VoxCPM is a multilingual speech synthesis system and text-to-speech inference server. It functions as an AI voice cloning tool and a synthetic voice designer, capable of generating natural speech across global languages and regional dialects using a GPU-accelerated audio generator. The project features a speech model fine-tuning framework that supports both full parameter updates and low-rank adaptation for customizing voice characteristics. It enables high-fidelity voice cloning from reference audio, including cross-lingual voice transfer and acoustic environment mimicry, as well as the crea

    Manages concurrent synthesis tasks using an asynchronous server that handles task priority and GPU memory allocation.

    Pythonaudiodeeplearningminicpm
    Auf GitHub ansehen↗29,985
  • pirate/archiveboxAvatar von pirate

    pirate/ArchiveBox

    27,721Auf GitHub ansehen↗

    ArchiveBox is a self-hosted web archiving system designed to capture and preserve permanent static copies of webpages, media, and PDFs on personal infrastructure. It functions as a digital content curator and personal web archive manager, allowing users to import URLs from bookmarks, RSS feeds, and browser history to create a centralized, searchable knowledge base. The project is distinguished by its ability to archive private, paywalled, or login-protected content using browser cookies and authenticated session persistence. It ensures long-term availability by saving pages in multiple concur

    Processes archiving and import jobs through an asynchronous queue to prevent UI blocking.

    Python
    Auf GitHub ansehen↗27,721
  • automattic/mongooseAvatar von Automattic

    Automattic/mongoose

    27,479Auf GitHub ansehen↗

    Mongoose is an object data modeling library and framework for Node.js that maps application objects to MongoDB documents. It functions as a document mapper and schema validator, ensuring consistent data types and validation rules for records stored in MongoDB. The project provides a system for defining structured schemas to model application data, including the ability to create hierarchical data structures through nested schema composition. It implements a middleware-based hook system that allows for the interception and modification of data states during the lifecycle of database operations

    Implements an asynchronous model for database interactions to ensure the main execution thread remains non-blocking during I/O.

    JavaScript
    Auf GitHub ansehen↗27,479
  • fincept-corporation/finceptterminalAvatar von Fincept-Corporation

    Fincept-Corporation/FinceptTerminal

    26,900Auf GitHub ansehen↗

    FinceptTerminal is a quantitative finance platform and financial engineering library designed for asset valuation, risk management, and fixed-income analytics. It provides a comprehensive suite for algorithmic trading and investment strategy automation, integrating specialized language model agents and node-based workflows to automate market research and alpha generation. The project distinguishes itself with a dedicated game theory analysis engine for calculating Nash equilibria and simulating strategic interactions in competitive markets. It also features a specialized credit risk modeling

    Offloads long-running computations to background workers to maintain application responsiveness.

    C++bloomberg-terminalcontributions-welcomefinance
    Auf GitHub ansehen↗26,900
  • alexellis/faasAvatar von alexellis

    alexellis/faas

    26,192Auf GitHub ansehen↗

    This project is a Kubernetes serverless framework and OCI container function platform. It provides a system for deploying event-driven functions and microservices as compatible container images onto a Kubernetes cluster. The platform includes an event-driven function orchestrator that triggers executions via HTTP requests or message streams. It features an auto-scaling function manager that adjusts the number of active instances based on real-time demand and scales down to zero during inactivity. A background queuing system is included to process asynchronous tasks and maintain application re

    Implements a background queuing system to offload long-running tasks and maintain application responsiveness.

    Go
    Auf GitHub ansehen↗26,192
  • sdwebimage/sdwebimageAvatar von SDWebImage

    SDWebImage/SDWebImage

    25,644Auf GitHub ansehen↗

    SDWebImage is an asynchronous image downloader and caching library for iOS and macOS applications. It provides a core identity centered on a network utility for fetching images from URLs, a tiered memory and disk caching engine, and a processing framework for decoding and encoding media. The library features a specialized rendering engine for animated formats such as GIF and WebP, including support for progressive animation rendering. It distinguishes itself through a plugin system that allows for extended image format support and the ability to replace default loading or storage logic with c

    Implements an asynchronous task queue to manage image downloads and processing without blocking the main UI thread.

    Objective-Ccachecarthagecocoapods
    Auf GitHub ansehen↗25,644
  • aidc-ai/pixelle-videoAvatar von AIDC-AI

    AIDC-AI/Pixelle-Video

    23,403Auf GitHub ansehen↗

    Pixelle-Video is a text-to-video automation platform and generation engine that converts text topics into complete videos with synchronized narration, images, and music. It functions as a modular system for producing short-form content, utilizing large language models to automate script composition, visual asset generation, and voiceover production. The platform features a node-based workflow orchestrator that allows the composition of custom generation pipelines by linking different AI models. It includes a dynamic video layout designer that uses HTML templates to define aspect ratios and vi

    Implements background job execution to offload long-running video generation processes.

    Pythonaigccomfyuiimage-generation
    Auf GitHub ansehen↗23,403
  • goauthentik/authentikAvatar von goauthentik

    goauthentik/authentik

    22,035Auf GitHub ansehen↗

    Authentik is a centralized identity and access management platform designed to serve as a unified authentication authority. It enables enterprise single sign-on across diverse applications and services, providing a cloud-native identity provider that manages user sessions and security protocols from a single location. The platform distinguishes itself through a policy-driven flow engine and a visual orchestration interface. This allows administrators to design complex, custom authentication workflows by chaining modular verification stages and conditional logic. These workflows can be further

    Offloads long-running background operations to a decoupled worker model for improved system maintenance and execution tracking.

    Pythonauthenticationauthentikauthorization
    Auf GitHub ansehen↗22,035
  • pedroslopez/whatsapp-web.jsAvatar von pedroslopez

    pedroslopez/whatsapp-web.js

    22,024Auf GitHub ansehen↗

    This project is a Node.js framework designed for headless browser automation, enabling the creation of automated messaging clients. It functions by controlling a headless browser instance to programmatically interact with the messaging interface, allowing developers to simulate user sessions and manage complex chat workflows. The library distinguishes itself through its comprehensive session management and event-driven architecture. It supports persistent authentication by serializing session data to local or remote storage, ensuring that automated clients can maintain continuous connectivity

    Manages the flow of outgoing media and text packets to ensure reliable delivery while maintaining non-blocking execution of tasks.

    JavaScriptapibotbot-api
    Auf GitHub ansehen↗22,024
  • slopus/happyAvatar von slopus

    slopus/happy

    21,984Auf GitHub ansehen↗

    Happy is a remote management platform designed to orchestrate and monitor multiple concurrent artificial intelligence coding agents across self-hosted development environments. It functions as a secure gateway, allowing users to control local programming assistants and manage development tasks from any device. The platform distinguishes itself by providing a hands-free interaction layer that enables users to execute commands through voice input. It also integrates a notification workflow that delivers real-time alerts to mobile or desktop devices, ensuring users remain informed when agents re

    Manages concurrent development workloads by scheduling and tracking independent agent processes through a centralized coordinator.

    TypeScriptclaude-codeclaude-desktopclaude-mobile
    Auf GitHub ansehen↗21,984
  • hkuds/rag-anythingAvatar von HKUDS

    HKUDS/RAG-Anything

    21,372Auf GitHub ansehen↗

    RAG-Anything is a retrieval-augmented generation framework designed to index diverse document formats and perform semantic search using local machine learning models. It functions as a local multimodal data processor, extracting and organizing information from various file types into a unified knowledge base to facilitate private document analysis. The system distinguishes itself through its high-throughput ingestion engine, which processes large batches of documents into searchable vector embeddings. By executing machine learning models directly on local hardware, the framework ensures that

    Manages concurrent document ingestion tasks using background job execution to accelerate indexing.

    Pythonmulti-modal-ragretrieval-augmented-generation
    Auf GitHub ansehen↗21,372
  • anil-matcha/open-higgsfield-aiAvatar von Anil-matcha

    Anil-matcha/Open-Higgsfield-AI

    20,529Auf GitHub ansehen↗

    Open-Higgsfield-AI is a generative AI content studio and visual workflow orchestrator. It provides a unified interface for creating photorealistic images and videos, utilizing a node-based editor to chain multiple image, video, and audio models into automated content pipelines. The system functions as an AI video animation tool and local GPU inference engine, allowing users to run generative models on local hardware or remote servers. It includes specialized capabilities for audio-driven lip synchronization and cinematic camera controls to adjust virtual lens and focal settings. The platform

    Implements an asynchronous task queue to manage long-running video and image synthesis jobs.

    JavaScriptai-art-generatorai-image-generationai-video-generation
    Auf GitHub ansehen↗20,529
  • langchain-ai/langchainjsAvatar von langchain-ai

    langchain-ai/langchainjs

    17,818Auf GitHub ansehen↗

    LangChain.js is a framework for building, executing, and monitoring stateful agentic applications. It provides an orchestration engine that models workflows as directed graphs, allowing developers to connect language models, data sources, and external tools into modular, multi-step processes. The platform distinguishes itself through its focus on stateful execution and human-in-the-loop control. It manages agent lifecycles by persisting execution state across threads, enabling fault tolerance and the ability to pause workflows at designated breakpoints for manual review or modification. This

    Orchestrates asynchronous job execution through task queues to enable concurrent processing and streaming.

    TypeScript
    Auf GitHub ansehen↗17,818
  • stan-smith/fossflowAvatar von stan-smith

    stan-smith/FossFLOW

    17,487Auf GitHub ansehen↗

    FossFLOW is an open source metadata search engine and data platform designed to aggregate and normalize repository information from multiple code hosting services. It functions as a developer productivity utility, enabling users to discover software projects and analyze contributor networks through a unified, searchable index. The platform distinguishes itself by utilizing vector-based semantic search, which converts project descriptions and code metadata into numerical embeddings to facilitate discovery based on conceptual relevance. To maintain a consistent view of disparate data, the syste

    Offloads resource-intensive indexing and data retrieval tasks to background workers to maintain system responsiveness.

    TypeScriptdevopsinfrainfrastructure
    Auf GitHub ansehen↗17,487
  • simplifyjobs/new-grad-positionsAvatar von SimplifyJobs

    SimplifyJobs/New-Grad-Positions

    17,201Auf GitHub ansehen↗

    New-Grad-Positions is a centralized job aggregation platform designed to track and filter entry-level career opportunities for recent graduates across technical industries. The system functions as an automated career search tool, utilizing a relational database schema to organize job listings and user profiles for efficient querying. The platform distinguishes itself through integrated browser-based automation that populates online job application fields to reduce manual data entry. It further supports career search automation by monitoring new listings and triggering email alerts based on sp

    Distributes heavy scraping and form-filling operations to background workers to maintain system responsiveness.

    applicationscoderquadcollege
    Auf GitHub ansehen↗17,201
  • nodebb/nodebbAvatar von NodeBB

    NodeBB/NodeBB

    15,144Auf GitHub ansehen↗

    NodeBB is a real-time, self-hosted community forum platform built on Node.js. It is designed to support scalable discussion environments by utilizing a document-oriented database for content storage and an in-memory engine for high-speed data retrieval and session management. The platform provides a comprehensive administrative interface for managing user groups, forum settings, and system health. What distinguishes the platform is its native support for federated social networking via the ActivityPub protocol, allowing forums to exchange content, synchronize discussions, and interact with de

    Implements an asynchronous background task queue to offload heavy operations and maintain system responsiveness.

    JavaScriptcommunityforumjavascript
    Auf GitHub ansehen↗15,144
  • mindverse/second-meAvatar von mindverse

    mindverse/Second-Me

    15,123Auf GitHub ansehen↗

    Second-Me is a framework for orchestrating local agent tasks and fine-tuning personal language models. It provides a system for training specialized assistants on local datasets to support custom knowledge retrieval and task execution requirements. The project distinguishes itself through a modular architecture that manages the lifecycle of machine learning tasks. It includes a state manager that persists intermediate training progress to local storage, allowing for the interruption and resumption of long-running configuration processes. Furthermore, the system utilizes standardized protocols

    Manages background execution of resource-intensive model training tasks to maintain system responsiveness.

    Python
    Auf GitHub ansehen↗15,123
Vorherige123…4Nächste
  1. Home
  2. Software Engineering & Architecture
  3. Asynchronous Task Queues

Unter-Tags erkunden

  • Async Task WrappersUtilities that wrap blocks of work to resolve with a value or error on a specific queue. **Distinct from Asynchronous Task Queues:** Focuses on the wrapping of a work block into a promise rather than a full job queue management system.
  • Database I/O QueuingMechanisms to manage database read and write operations asynchronously to prevent event-loop blocking. **Distinct from Asynchronous Task Queues:** Specifically addresses database I/O rather than general background job task queues.
  • Kernel Request Deferred Response QueuesQueues that buffer filesystem operation results for asynchronous delivery back to the kernel. **Distinct from Asynchronous Task Queues:** Distinct from Asynchronous Task Queues: focuses on deferring kernel request completion via a dedicated channel, not general background job offloading.