41 dépôts
Offloading long-running tasks to background workers.
Explore 41 awesome GitHub repositories matching software engineering & architecture · Asynchronous Task Queueing. Refine with filters or upvote what's useful.
Superset is a web-based business intelligence platform designed for data exploration, visualization, and interactive dashboarding. It functions as a query-driven analytics engine that connects to various SQL databases, allowing users to perform ad-hoc analysis, define virtual metrics, and build complex data visualizations through a centralized interface. The platform distinguishes itself through a robust semantic layer that transforms raw database schemas into calculated columns and virtual metrics, enabling consistent business logic across an organization. It features a plugin-based visualiz
Delegates resource-intensive processing to background workers to ensure web interface responsiveness.
This project is a business intelligence suite and SQL data visualization platform used for data analysis, reporting, and monitoring. It provides a web application for exploring datasets and building interactive dashboards, complemented by a web-based SQL query editor for analyzing raw data from connected stores. The platform features a semantic data layer to define standardized metrics and dimensions, ensuring consistent data interpretation across reports. It includes a security framework with role-based access control to manage user permissions and authentication across shared dashboards. T
Offloads long-running queries and heavy data processing to background workers using Celery to maintain UI responsiveness.
This project is a community-driven directory that aggregates essential software projects and educational content for the Node.js ecosystem. It functions as a centralized knowledge base and discovery index, designed to simplify the navigation of a fragmented technical landscape by providing a structured collection of high-quality links, tools, and learning materials. The repository distinguishes itself through a decentralized, peer-reviewed curation model. By utilizing standard version control workflows and pull requests, the community ensures that all listed resources undergo human verificati
Features robust infrastructure solutions for managing asynchronous task execution and message processing.
Async is a JavaScript asynchronous flow library designed to manage the execution and coordination of asynchronous tasks in Node.js and the browser. It provides functional utilities to wrap, process, and orchestrate complex asynchronous workflows. The library distinguishes itself through a comprehensive task orchestrator that handles dependency graphs to resolve circular references and manages concurrent task queues. It includes a unification bridge that allows callback-style and promise-based functions to operate within the same execution interface. The project covers several primary capabil
Controls the flow of asynchronous tasks using concurrency limits and payload constraints.
UniGetUI is a graphical interface designed to manage software installations on the Windows operating system. It functions as a centralized dashboard that provides a unified view for installing, updating, and removing applications across multiple command-line package managers. The application distinguishes itself by normalizing the disparate inputs and outputs of various package management utilities into a single, consistent schema. It supports batch operations, allowing users to perform maintenance tasks on multiple packages simultaneously, and includes functionality to automate update proces
Executes multiple installation and update operations concurrently using background worker threads to maintain interface responsiveness.
Cube is a semantic data layer that provides a unified framework for defining business metrics, dimensions, and relationships across diverse data sources. By acting as a headless business intelligence engine, it transforms raw data into a governed model that can be queried via SQL, REST, and GraphQL interfaces. This architecture ensures consistent data definitions and logic across all downstream analytical applications and reporting tools. The platform distinguishes itself through its integrated conversational AI capabilities, which allow users to explore data using natural language. It orches
Manages incoming data requests through a distributed task system to deduplicate operations and protect source databases.
SpiderFoot is an open-source reconnaissance and intelligence automation framework designed to streamline the collection and correlation of data for security investigations. It functions as a comprehensive platform that automates the querying of hundreds of public data sources to map digital footprints, identify exposed assets, and uncover potential security threats across an organization's external perimeter. The platform distinguishes itself through a modular, plugin-based architecture that executes data gathering tasks in parallel, supported by a directed graph data model that tracks relati
Background workers process long-running data collection jobs to maintain system responsiveness during intensive scanning operations.
Bull is a Node.js library for managing distributed jobs and message queues using Redis as the primary data store. It functions as a distributed task worker, job scheduler, and priority queue manager designed to handle asynchronous workloads across multiple processes. The project distinguishes itself by providing a persistent communication channel that decouples servers through the exchange of serializable data objects. It ensures distributed system reliability by detecting stalled tasks and recovering from process crashes to ensure every queued job is completed. The system covers a broad ran
Provides infrastructure for managing asynchronous task execution by enqueueing serializable data objects.
ConvertX is a web-based file conversion management platform designed to transform documents, images, and video files between various formats. It utilizes system-level binary orchestration to execute conversion tasks, leveraging background worker threads to handle concurrent, high-volume bulk processing without blocking the user interface. The platform distinguishes itself through a comprehensive security and access control framework, which includes multi-user account management, session-based token authentication, and role-based permissions. Users can secure their output files with passwords
Manages concurrent file processing jobs by distributing workloads across background workers to ensure non-blocking performance.
This tool is a command-line utility designed for automated web resource discovery, fuzzing, and application structure mapping. It functions as a security-focused scanner that identifies hidden files, directories, parameters, and virtual hosts by injecting payloads into HTTP requests. By systematically testing how servers handle various inputs, it assists in mapping the architecture of web applications and uncovering potential security vulnerabilities. The tool distinguishes itself through a highly concurrent engine that manages asynchronous request execution and recursive job orchestration. I
Manages a dynamic queue of discovery tasks that automatically spawns new scan passes when specific directory structures are identified.
Ants is a goroutine pool library for Go that functions as a concurrent task orchestrator and resource consumption limiter. Its primary purpose is to manage and reuse a pool of goroutines to limit concurrency and reduce the memory allocation overhead associated with frequent thread creation. The system provides a runtime worker manager capable of adjusting pool capacity dynamically to respond to fluctuating workloads. To maintain stability, it includes panic recovery mechanisms that intercept runtime failures within worker threads to prevent a single failing task from crashing the entire appli
Submits work to a pool of workers to control execution flow and prevent system exhaustion.
Cookiecutter Django is a command-line utility designed to generate production-ready web application structures based on the Django framework. It functions as a standardized boilerplate generator that automates the initial setup of a project, providing a consistent foundation that includes pre-configured settings, security defaults, and integrated infrastructure services. The project distinguishes itself by providing a fully containerized development and deployment environment. It orchestrates multi-container stacks that manage databases, background workers, and web servers, ensuring that the
Offloads long-running operations to background workers to maintain application responsiveness.
t3code is a platform for operating AI coding agents through a web interface to develop and maintain software projects. It functions as an orchestrator that manages AI agent lifecycles and communication, providing a system for provisioning isolated remote development environments and managing secure connections to cloud-hosted resources. The project utilizes an event-sourced backend architecture that processes typed commands into persisted domain events to maintain system state. It employs a communication system based on JSON-RPC over WebSockets to synchronize real-time state and events betwee
Executes long-running asynchronous tasks and side effects using queue-backed workers to ensure deterministic order.
Banana-slides is a generative AI workflow engine designed to automate the creation and refinement of professional slide decks. By leveraging large language models, the platform transforms raw text, structured outlines, and existing documents into visual presentations. It functions as an automated tool that orchestrates the entire lifecycle of a presentation, from initial content generation and layout design to final export. The system distinguishes itself through a modular provider abstraction that allows users to integrate various artificial intelligence services for content and image synthe
Offloads resource-intensive generation and export tasks to background workers to maintain system responsiveness.
FileCentipede is a comprehensive file management and transfer application designed to handle diverse network protocols and data operations. It functions as a multi-protocol download manager, a full-featured BitTorrent client, and a remote filesystem manager, providing a unified interface for moving and organizing data across local and remote environments. The application distinguishes itself through deep browser integration, which allows for the direct capture of media streams, video, and bulk download links from web pages. It also includes a modular utility suite that enables users to perfor
Manages concurrent file operations and network requests using a background scheduler to optimize bandwidth and ensure reliable task completion.
Paper2gui is a multi-modal AI toolkit and model GUI wrapper designed to deploy and run various artificial intelligence models through a visual interface. Its primary purpose is to provide a way to execute complex AI research papers and models without requiring manual software installation or coding. The project distinguishes itself by using a wrapper-based model interface that abstracts command line arguments into visual input fields, utilizing template-driven UI generation to create parameter sliders and forms based on the specific requirements of the underlying model. It includes a centrali
Implements asynchronous task queueing to handle long-running model inferences in the background.
algorithm-base is an educational library and study guide designed for simulating algorithms and studying data structures. It functions as an execution visualizer that renders step-by-step state changes and pointer updates through animated simulations to illustrate how data movement works. The project distinguishes itself by mapping conceptual logic directly to multi-language source code implementations. It utilizes a comparative analysis framework to evaluate different algorithmic strategies based on stability, time complexity, and space complexity, while organizing problems by underlying mec
Demonstrates enqueueing and dequeueing elements using animations to simulate the first-in-first-out mechanism.
Inbox Zero is an AI-powered email automation platform and inbox organizer. It uses large language models to automatically categorize, label, and archive emails, while providing a conversational interface for managing workflows and drafting responses through natural language. The project distinguishes itself by integrating real-time calendar availability into its drafting process and generating AI-summarized meeting briefings. It supports a pluggable AI provider interface with model fallback chains, allowing it to connect to various cloud or local LLM providers. Users can also control their in
Executes high-volume background tasks like bulk cleaning and AI categorization using a durable job queue.
Hangfire is a background job scheduler and distributed task queue for .NET applications. It serves as a job orchestration framework that offloads heavy processing to background workers using a SQL-backed processor to manage job state across multiple servers. The framework distinguishes itself through reliable task scheduling, where job metadata and arguments are persisted in an external database to ensure tasks survive application restarts. It supports advanced orchestration patterns, including the ability to chain dependent tasks so that a child job triggers automatically upon the successful
Provides a robust system for enqueueing method calls and arguments to be processed asynchronously.
Redis is a high-performance in-memory key-value store that functions as a distributed cache, message broker, and NoSQL database. It provides sub-millisecond read and write access to data stored in RAM and can operate as a vector database for indexing high-dimensional embeddings. The system supports a wide range of data storage and synchronization primitives, including the management of strings, hashes, lists, sets, and JSON documents. It enables real-time data operations through atomic transactions, hybrid persistence using snapshots and append-only logs, and high-availability configurations
Implements distributed task queues using streams and consumer groups to ensure single-processing of jobs.