68 个仓库
Tools for monitoring and prioritizing background task execution.
Distinguishing note: Focuses on worker queue assignment and parallel execution.
Explore 68 awesome GitHub repositories matching devops & infrastructure · Queue Management. Refine with filters or upvote what's useful.
Mastodon is a self-hosted, decentralized social networking platform that functions as a microblogging application. It enables independent server instances to communicate and exchange social data through the standardized ActivityPub protocol, allowing users to participate in a global, interoperable network. The platform distinguishes itself through its federated architecture, which grants administrators full control over their community instances. This includes comprehensive tools for user moderation, account management, and the enforcement of community guidelines. The system is designed to ha
Assigns specific queues to background worker processes to ensure efficient execution of critical tasks.
SMSBoom is an automated SMS bomber and bulk messaging tool designed to send a rapid succession of text messages to a target phone number or a list of multiple recipients. It functions as a bulk SMS sender that automates the dispatch of large volumes of text messages. The project manages mass communication workflows through automated text dispatch and bulk messaging capabilities. It utilizes a system for sending messages to multiple recipients simultaneously via automated messaging systems. The tool handles message dispatching via HTTP requests to third party gateways, employing concurrent re
Organizes target phone numbers into a sequential queue to manage the flow of bulk messages.
RabbitMQ is a multi-protocol messaging broker that functions as an AMQP message broker, a clustered message queue, and a distributed message stream. It provides a server for translating and bridging communication between diverse messaging standards to connect heterogeneous systems. The system distinguishes itself through distributed broker clustering and federation, using shoveling mechanisms to synchronize data across geographically separate sites. It supports high-throughput, append-only logs for persisting and reading large sequences of messages for real-time processing. The broker covers
Manages the routing logic and relationships between specialized exchanges and queues, including delayed delivery.
kafka-ui is a web interface and centralized control plane for administering Apache Kafka clusters, topics, and brokers. It functions as a distributed message queue dashboard and orchestrator, allowing for the oversight of multiple distributed Kafka environments from a single management interface. The project provides dedicated tools for producing and inspecting messages within topics using various serialization and encoding formats. It includes a schema registry client for defining and versioning data schemas and a consumer monitoring dashboard to track offsets and calculate partition lag. T
Functions as a centralized monitoring system for tracking cluster health and consumer lag across environments.
Webmagic is a Java web crawling framework designed for building scalable automated crawlers to download and process large volumes of web pages. It functions as a distributed web crawler and dynamic content crawler, utilizing an XPath HTML parser to locate and extract specific data points from page structures. The framework distinguishes itself through its ability to handle dynamic content by rendering JavaScript and executing asynchronous requests to extract data from non-static pages. It also allows users to define and execute crawler logic via scripting languages, enabling the update of col
Implements a pipeline for tracking discovered links and scheduling page downloads to ensure complete traversal of target websites.
EOS is a Layer 1 blockchain infrastructure and high-throughput transaction engine. It serves as a WebAssembly smart contract platform that manages state transitions and network consensus across a peer-to-peer network. The system utilizes a sandboxed virtual machine for executing smart contract logic and employs a Byzantine Fault Tolerant delegated proof-of-stake consensus mechanism to finalize the global ledger state. It features a resource-based stake model for CPU and memory allocation and an asynchronous messaging system for inter-contract communication to prevent recursive call overflows.
Triggers inter-contract actions using an asynchronous queue to prevent recursive call overflows.
libzmq is a C++ based asynchronous messaging engine and networking core designed for routing non-blocking messages between distributed nodes. It functions as a distributed message queue that implements the ZMTP wire-format framing protocol to standardize how data moves across different network transport layers. The library provides a multi-transport abstraction that allows a single interface to route data across TCP, IPC, and in-process memory. It incorporates a cryptographic layer to encrypt and authenticate transmissions between nodes and employs topology-based messaging patterns, such as p
Implements internal buffers to decouple senders and receivers for non-blocking communication.
dtm is a distributed transaction framework and polyglot transaction coordinator designed to maintain data consistency across microservices. It functions as a Saga orchestration engine and a two-phase message coordinator, ensuring that multi-service operations either succeed completely or roll back to a consistent state. The project distinguishes itself by supporting multiple consistency patterns, including Saga, TCC, XA, and outbox patterns, allowing users to select the appropriate model for their specific application requirements. It provides a polyglot integration layer via HTTP and gRPC, e
Triggers remote service calls asynchronously without requiring dedicated message queue infrastructure.
EverydayWechat is a WeChat automation bot designed to automate messages, scheduled reminders, and automatic replies within the WeChat messaging ecosystem. It functions as a multi-purpose system that combines the roles of a scheduled message sender, an auto-reply bot, and a chatbot assistant. The project enables the delivery of customized recurring messages to specific users and group chats on a fixed timetable. It also provides automated individual replies based on preconfigured rules and group chat assistance that fetches real-time data for weather, logistics, and calendars. The system inco
Implements a message queue to sequence outgoing messages and avoid platform rate limits.
Redis is a high-performance in-memory key-value store that functions as a distributed cache, message broker, and NoSQL database. It provides sub-millisecond read and write access to data stored in RAM and can operate as a vector database for indexing high-dimensional embeddings. The system supports a wide range of data storage and synchronization primitives, including the management of strings, hashes, lists, sets, and JSON documents. It enables real-time data operations through atomic transactions, hybrid persistence using snapshots and append-only logs, and high-availability configurations
Implements asynchronous message queues using lists and sorted sets for task sequencing.
JsBridge is a bidirectional communication layer for Android that synchronizes data and triggers actions between a native Java application and an embedded web view. It serves as an interface for calling Java methods from JavaScript and executing JavaScript functions from native Android code. The project features an asynchronous bridge queue that stores pending function calls in a readiness buffer, ensuring requests are executed only after the connection between the native and web environments is stable. It utilizes callback identifier mapping to route asynchronous native responses back to the
Implements a message-queueing buffer that stores calls until the native and web environments are stable.
Resque is a Ruby library for enqueueing and processing asynchronous tasks using Redis as a data store. It functions as a distributed task processor and queue manager, allowing long-running work to be moved out of the main request cycle. The system executes background jobs in isolated child processes to prevent memory leaks and provides a web-based dashboard for monitoring queue depths, worker activity, and failed job statistics. Capability areas include distributed worker coordination via signals, error handling with job retry mechanisms, and priority-ordered queue management. It also suppor
Provides a visual dashboard for tracking queue depths and the overall health of the task system.
This project is a comprehensive knowledge base and study resource designed for mastering technical interviews. It provides structured guides, roadmaps, and curricula focused on data structures, algorithms, system design, and frontend engineering to help candidates prepare for software engineering screenings. The repository distinguishes itself by offering a holistic approach to professional advancement. Beyond technical drills, it includes a career development handbook covering resume optimization, salary benchmarking, and strategic negotiation coaching. It also provides detailed methodologie
Uses message queues to decouple producers and consumers for scalable asynchronous request processing.
Gorush is a dedicated notification dispatch server used to send real-time alerts to iOS and Android mobile devices. It functions as a centralized infrastructure for hosting a notification API and managing the delivery of push messages through high-throughput queuing. The system supports both gRPC streams and HTTP endpoints to receive notification triggers and includes a command-line interface for triggering alerts without a running server process. It manages network traffic through proxy routing via HTTP or SOCKS5 and secures server communication using automated TLS certificate management. O
Uses a configurable pool of concurrent workers and queues to handle high-volume notification throughput.
BullMQ is a Redis-backed message queue library and background processor designed for distributed task queueing. It functions as a distributed queue manager and task scheduler, utilizing Redis to manage asynchronous job processing and persistence. The system distinguishes itself through its role as a job workflow orchestrator, enabling the definition of complex parent-child job dependencies and hierarchies for multi-step workflows. It provides sandboxed process execution to isolate heavy workloads and prevent event loop blocking, alongside distributed rate limiting to protect downstream servic
Provides persistent message queues to decouple services and ensure delivery when receivers are offline.
SwiftMessages is an iOS notification bar library and in-app message queue manager. It provides a system for displaying customizable alerts and status messages at the top, bottom, or center of the screen. The library features a hosting bridge that allows declarative SwiftUI views to be rendered within imperative UIKit containers. It includes a queue management system that sequences chronological notifications and filters duplicate alerts to prevent interface overlap. The system handles dynamic layout management to avoid occlusion by the system keyboard and provides an accessibility interface
Implements a sequencing system to manage notification flow and prevent UI overlap.
TubeArchivist is a self-hosted YouTube video archiving system and metadata indexer. It functions as a personal media library and download manager that allows users to create a searchable offline collection of videos, channels, and playlists. The system distinguishes itself by indexing subtitles, comments, and channel information for full-text search and retrieval. It features automated media synchronization to track subscriptions and playlists, ensuring new content is automatically queued and downloaded as it is published. The project provides a broad set of capabilities for digital asset ma
Manages video downloads through a prioritized queue that supports manual overrides.
goim
Provides a Kafka-driven message queue server for asynchronous, non-blocking fan-out delivery to subscribers.
SpringCloud-Learning is an educational project that demonstrates how to build microservices using Spring Cloud, covering the core patterns of service discovery, centralized configuration management, and API gateway routing. The project provides hands-on examples for registering and discovering microservice instances with Nacos, Eureka, or Consul, and for routing external API requests through Spring Cloud Gateway with support for filters and load balancing. The tutorials explore service resilience through circuit breakers and rate limiting with Sentinel and Hystrix, including custom fallback l
Re-injects failed messages back into the queue for retry until processing succeeds.
Noty is a dependency-free JavaScript toast notification library and in-app notification system. It provides a framework for rendering non-blocking alert, success, error, and information messages, as well as interactive confirmation dialogs. The library includes a notification queue manager to organize the sequence and timing of messages and a web push notification manager for delivering system-level alerts via service workers when an application is inactive. The system supports extensive visual customization through HTML templates and theme configurations. It manages the notification lifecyc
Organizes incoming notifications into ordered lists to control their appearance timing and sequence.