17 مستودعات
Background processing components that consume and execute tasks from message queues.
Distinguishing note: Focuses on the infrastructure component for asynchronous task processing rather than the message broker itself.
Explore 17 awesome GitHub repositories matching devops & infrastructure · Message Queue Workers. Refine with filters or upvote what's useful.
OpenFaaS is a serverless function platform that provides a container-native framework for deploying and managing event-driven code. It functions as an abstraction layer over container orchestrators, allowing developers to package code into scalable functions that run across Kubernetes clusters or edge computing environments. The platform distinguishes itself through a developer-centric runtime that utilizes standardized language templates and automated build pipelines to simplify the creation of container images. It features a central API gateway that manages request routing, authentication,
In this guide, you'll learn how to deploy from GitLab CI/CD using OpenFaaS's IAM support and Web Identity Federation. You'll need to create YAML files for an Issuer, a Policy and a Role. These need to be applied through
Postal is an open-source mail delivery platform designed for self-hosted sending and receiving of email using standard mail transfer protocols. It serves as a comprehensive email infrastructure that provides full delivery tracking and management of outgoing mail streams. The system includes an SMTP server manager with a web administration interface for configuring server settings and monitoring delivery status. It distinguishes its outbound traffic management through priority-based IP rotation and assignment to help maintain sender reputation. The platform covers incoming email processing an
Uses background worker processes to decouple email reception from final delivery via asynchronous queuing.
This project is a PHP e-commerce platform and enterprise storefront framework designed for building and managing online stores. It functions as a modular PHP application and headless commerce engine, separating e-commerce business logic from frontend presentation to support multiple storefronts and devices. The system is built on a modular architecture that allows developers to add custom business logic and third-party integrations. It utilizes a service-contract-based API to ensure stability across module implementations and employs dependency-injection for object lifecycle management. Core
Provides background workers that consume tasks from message queues to handle resource-intensive workloads asynchronously.
Gorush is a dedicated notification dispatch server used to send real-time alerts to iOS and Android mobile devices. It functions as a centralized infrastructure for hosting a notification API and managing the delivery of push messages through high-throughput queuing. The system supports both gRPC streams and HTTP endpoints to receive notification triggers and includes a command-line interface for triggering alerts without a running server process. It manages network traffic through proxy routing via HTTP or SOCKS5 and secures server communication using automated TLS certificate management. O
Processes high-volume notification deliveries using a configurable pool of concurrent background workers.
Simplebank is a financial services backend application built with Go that manages bank accounts and transfers. It utilizes a dual-protocol interface, providing both gRPC and REST APIs via Protocol Buffers to support different client communication requirements. The system implements a PostgreSQL data layer with versioned schema migrations and type-safe query generation. It handles financial operations through atomic fund transfers and balance change tracking to maintain consistent audit trails. The architecture includes an asynchronous task worker system using a message queue to offload long-
Offloads time-consuming background tasks to a separate worker system using a message queue.
MISP is an open-source threat intelligence sharing platform designed for collecting, storing, and distributing structured threat indicators and intelligence. At its core, it provides a distributed synchronization protocol for transferring events between instances, an attribute-based correlation engine that links matching indicators across events, and a REST API with an OpenAPI specification for programmatic access to threat data. The platform uses formal data formats for JSON, taxonomy, galaxy, and object templates to enable compatibility across tools and communities. The platform distinguish
Processes scheduled tasks and data operations asynchronously through a dedicated background worker queue system.
This is a distributed voting application designed to demonstrate a multi-service architecture. It uses stateless web frontends for submitting votes and viewing live results, with a Redis-backed queue to buffer incoming votes and a PostgreSQL database for persistent tallying. The application is built around asynchronous message queue processing, decoupling the vote submission from the tallying workflow. The project showcases how to deploy a multi-service application using container orchestration tools. It provides YAML-driven declarative deployment manifests for Docker Compose, Docker Swarm, a
Stores incoming votes in a Redis list for asynchronous consumption by a background worker.
protoactor-go is a framework for building concurrent and distributed systems in Go using the actor model. It provides a distributed actor system that enables isolated entities to communicate via asynchronous messaging and share state across a cluster. The framework implements a multi-language actor protocol, allowing interoperability between actors written in Go, C#, and Java. It further supports a virtual actor implementation, where actors are automatically instantiated across a network based on a unique identity. The system includes a supervision model for managing actor lifecycles and fau
Distributes incoming messages across pools of worker actors to balance load and increase throughput.
Flask-SocketIO is a Flask extension that integrates the Socket.IO real-time communication protocol into Flask applications, enabling persistent bidirectional connections between servers and clients. It transparently switches between WebSocket and HTTP long-polling while maintaining a persistent session identity, and provides a full event-driven framework for real-time web application communication. The library distinguishes itself with support for namespace multiplexing, allowing event handlers to be isolated into separate namespaces with independent lifecycle and routing. It also offers room
Uses a shared message queue to synchronize events across multiple worker processes for horizontal scaling.
Dramatiq is a distributed task queue and workload manager used to offload function execution to background workers. It functions as an asynchronous task orchestrator that enables the distribution of computational tasks across a cluster using a pluggable transport layer supporting RabbitMQ and Redis. The framework provides specialized tools for complex task orchestration, including the ability to link background jobs into sequences, pipelines, and barriers. It further manages distributed concurrency through the use of shared mutexes, rate limiters, and exponential backoff retries to prevent re
Provides background processing components that consume and execute tasks from declared message queues.
Ockam هو إطار عمل للتشفير من النهاية إلى النهاية ومزود هوية موزع مصمم لإنشاء اتصال آمن بين التطبيقات والأجهزة. يوفر تراكب شبكة آمن يستخدم هويات تشفير وتحكماً في الوصول قائماً على السمات لتنفيذ الوصول إلى الشبكة بانعدام الثقة (zero trust). يتميز المشروع بالتوجيه متعدد القفزات القائم على البيانات الوصفية وطبقة نقل قابلة للتوصيل، مما يسمح لحركة المرور المشفرة بالتحرك عبر طوبولوجيا شبكة متنوعة دون الحاجة إلى تراكبات IP افتراضية. وهو يمكن بشكل خاص الأنفاق الآمنة للتطبيقات القديمة عن طريق تغليف حركة مرور TCP الخام في قنوات مشفرة، مما يسمح باتصال الشبكة الخاصة وتجاوز جدار الحماية عبر مرحلات خارجية. تغطي المنصة نطاقاً واسعاً من القدرات، بما في ذلك إدارة الهوية الموزعة، وإصدار والتحقق من بيانات الاعتماد المشفرة، وتنفيذ الجهات الفاعلة المتزامنة ذات الحالة. كما توفر أدوات لتوفير العقد على نطاق سحابي والنشر المؤتمت باستخدام قوالب البنية التحتية ككود.
Directs communication between decoupled actor components using a registry to resolve addresses and relay messages.
Ockam هو إطار عمل للشبكات بانعدام الثقة (zero-trust) مصمم لتأمين نقل البيانات بين التطبيقات الموزعة باستخدام تراكب شبكة قائم على الهوية. يوفر البدائيات اللازمة لإنشاء اتصالات مصادق عليها متبادلاً ومشفرة من النهاية إلى النهاية، مما يزيل الاعتماد على أمن طبقة الشبكة التقليدي. يتميز المشروع باستخدامه للتحكم في الوصول القائم على السمات وبيانات الاعتماد القابلة للتحقق لإدارة الثقة على نطاق واسع. وينفذ تدوير هوية التشفير للحفاظ على استمرارية الهوية ويتكامل مع أنظمة إدارة المفاتيح المدعومة بالأجهزة لتأمين المفاتيح الخاصة داخل الجيوب (enclaves) أو خدمات إدارة المفاتيح السحابية. تغطي المنصة نطاقاً واسعاً من القدرات بما في ذلك التوجيه الثنائي متعدد القفزات وجسر الشبكة القائم على الترحيل لربط الشبكات المتباينة. يمكنه تغليف حركة مرور TCP أو Kafka القديمة في أنفاق آمنة، مما يسمح للخدمات الخاصة بالتواصل دون كشف منافذ الاستماع. بالإضافة إلى ذلك، يستخدم نموذج جهة فاعلة ذات حالة لمعالجة الرسائل بشكل غير متزامن عبر العقد الموزعة. يتم دعم النشر من خلال قوالب البنية التحتية ككود لتوفير العقد والبوابات الآمنة في البيئات السحابية.
Sends data to specific worker addresses through a router that resolves locations and manages delivery across different contexts.
useSend is an email marketing platform and transactional delivery system built to manage bulk campaigns and individual messages using AWS SES for delivery. It provides a RESTful API, an SMTP relay gateway, and a visual management interface for designing HTML templates and coordinating email broadcasts. The platform distinguishes itself by translating legacy SMTP traffic into modern API calls and implementing an event-driven webhook system. This system pushes real-time delivery, bounce, and click notifications to external servers using HMAC-SHA256 signed payloads to ensure authenticity. The s
Processes bulk email campaigns and scheduled messages using background workers for high throughput.
This project is a PHP compatibility library that provides a polyfill for the native ctype extension. It serves as a portability layer, implementing missing core language functions to ensure consistent behavior across different server environments and PHP versions. The library specializes in character type validation, recreating the behavior of native extension functions using pure PHP code. It uses runtime environment detection to identify if the native extension is missing and injects fallback implementations directly into the global namespace to prevent collisions and maintain code portabil
Runs background processes that poll transports for pending messages and execute their associated handlers.
This project is a headless WhatsApp client and automation server that exposes multi-device functionality as a REST API gateway. It provides a centralized interface for managing multiple account sessions, automating messages, and developing bots through a standardized protocol. The system distinguishes itself by providing a secure webhook provider that forwards real-time messages and reactions to external URLs using signature verification. It includes specialized media utilities for converting images and GIFs into the WebP format required for stickers, as well as tools to simulate active user
Uses Go routines to process incoming events and outgoing messages without blocking the main execution thread.
The Cloudflare Workers Rust SDK is a toolchain and framework for building high-performance serverless functions using Rust. It compiles code into WebAssembly modules that execute within a sandboxed host environment at the edge. The project provides a typed remote procedure call system for invoking functions across different worker instances and utilizes procedural macros for mapping these distributed interfaces. It uses trait-based bindings and standardized wrappers to provide type-safe access to platform services and web-standard request and response objects. The SDK covers a broad range of
Implements background processing components that consume and execute tasks from message queues.
S3Scanner is a security tool for auditing public access and misconfigured permissions across S3-compatible storage providers. It functions as a storage auditor and security scanner designed to identify open buckets, enumerate publicly accessible objects, and exfiltrate data from misconfigured cloud environments. The project is distinguished by its integration with message brokers, allowing it to consume target lists for large-scale cloud infrastructure audits. It also provides utilities for dumping the entire contents of misconfigured buckets to local directories and calculating the total siz
Uses background workers to consume targets from message queues for distributed high-volume scanning.