17 repository-uri
Background processing components that consume and execute tasks from message queues.
Distinguishing note: Focuses on the infrastructure component for asynchronous task processing rather than the message broker itself.
Explore 17 awesome GitHub repositories matching devops & infrastructure · Message Queue Workers. Refine with filters or upvote what's useful.
OpenFaaS is a serverless function platform that provides a container-native framework for deploying and managing event-driven code. It functions as an abstraction layer over container orchestrators, allowing developers to package code into scalable functions that run across Kubernetes clusters or edge computing environments. The platform distinguishes itself through a developer-centric runtime that utilizes standardized language templates and automated build pipelines to simplify the creation of container images. It features a central API gateway that manages request routing, authentication,
In this guide, you'll learn how to deploy from GitLab CI/CD using OpenFaaS's IAM support and Web Identity Federation. You'll need to create YAML files for an Issuer, a Policy and a Role. These need to be applied through
Postal is an open-source mail delivery platform designed for self-hosted sending and receiving of email using standard mail transfer protocols. It serves as a comprehensive email infrastructure that provides full delivery tracking and management of outgoing mail streams. The system includes an SMTP server manager with a web administration interface for configuring server settings and monitoring delivery status. It distinguishes its outbound traffic management through priority-based IP rotation and assignment to help maintain sender reputation. The platform covers incoming email processing an
Uses background worker processes to decouple email reception from final delivery via asynchronous queuing.
This project is a PHP e-commerce platform and enterprise storefront framework designed for building and managing online stores. It functions as a modular PHP application and headless commerce engine, separating e-commerce business logic from frontend presentation to support multiple storefronts and devices. The system is built on a modular architecture that allows developers to add custom business logic and third-party integrations. It utilizes a service-contract-based API to ensure stability across module implementations and employs dependency-injection for object lifecycle management. Core
Provides background workers that consume tasks from message queues to handle resource-intensive workloads asynchronously.
Gorush is a dedicated notification dispatch server used to send real-time alerts to iOS and Android mobile devices. It functions as a centralized infrastructure for hosting a notification API and managing the delivery of push messages through high-throughput queuing. The system supports both gRPC streams and HTTP endpoints to receive notification triggers and includes a command-line interface for triggering alerts without a running server process. It manages network traffic through proxy routing via HTTP or SOCKS5 and secures server communication using automated TLS certificate management. O
Processes high-volume notification deliveries using a configurable pool of concurrent background workers.
Simplebank is a financial services backend application built with Go that manages bank accounts and transfers. It utilizes a dual-protocol interface, providing both gRPC and REST APIs via Protocol Buffers to support different client communication requirements. The system implements a PostgreSQL data layer with versioned schema migrations and type-safe query generation. It handles financial operations through atomic fund transfers and balance change tracking to maintain consistent audit trails. The architecture includes an asynchronous task worker system using a message queue to offload long-
Offloads time-consuming background tasks to a separate worker system using a message queue.
MISP is an open-source threat intelligence sharing platform designed for collecting, storing, and distributing structured threat indicators and intelligence. At its core, it provides a distributed synchronization protocol for transferring events between instances, an attribute-based correlation engine that links matching indicators across events, and a REST API with an OpenAPI specification for programmatic access to threat data. The platform uses formal data formats for JSON, taxonomy, galaxy, and object templates to enable compatibility across tools and communities. The platform distinguish
Processes scheduled tasks and data operations asynchronously through a dedicated background worker queue system.
This is a distributed voting application designed to demonstrate a multi-service architecture. It uses stateless web frontends for submitting votes and viewing live results, with a Redis-backed queue to buffer incoming votes and a PostgreSQL database for persistent tallying. The application is built around asynchronous message queue processing, decoupling the vote submission from the tallying workflow. The project showcases how to deploy a multi-service application using container orchestration tools. It provides YAML-driven declarative deployment manifests for Docker Compose, Docker Swarm, a
Stores incoming votes in a Redis list for asynchronous consumption by a background worker.
protoactor-go is a framework for building concurrent and distributed systems in Go using the actor model. It provides a distributed actor system that enables isolated entities to communicate via asynchronous messaging and share state across a cluster. The framework implements a multi-language actor protocol, allowing interoperability between actors written in Go, C#, and Java. It further supports a virtual actor implementation, where actors are automatically instantiated across a network based on a unique identity. The system includes a supervision model for managing actor lifecycles and fau
Distributes incoming messages across pools of worker actors to balance load and increase throughput.
Flask-SocketIO is a Flask extension that integrates the Socket.IO real-time communication protocol into Flask applications, enabling persistent bidirectional connections between servers and clients. It transparently switches between WebSocket and HTTP long-polling while maintaining a persistent session identity, and provides a full event-driven framework for real-time web application communication. The library distinguishes itself with support for namespace multiplexing, allowing event handlers to be isolated into separate namespaces with independent lifecycle and routing. It also offers room
Uses a shared message queue to synchronize events across multiple worker processes for horizontal scaling.
Dramatiq is a distributed task queue and workload manager used to offload function execution to background workers. It functions as an asynchronous task orchestrator that enables the distribution of computational tasks across a cluster using a pluggable transport layer supporting RabbitMQ and Redis. The framework provides specialized tools for complex task orchestration, including the ability to link background jobs into sequences, pipelines, and barriers. It further manages distributed concurrency through the use of shared mutexes, rate limiters, and exponential backoff retries to prevent re
Provides background processing components that consume and execute tasks from declared message queues.
Ockam este un framework de criptare end-to-end și un furnizor de identitate distribuit conceput pentru a stabili o comunicare securizată între aplicații și dispozitive. Oferă un overlay de rețea securizat care utilizează identități criptografice și controlul accesului bazat pe atribute pentru a implementa accesul la rețea de tip zero trust. Proiectul se distinge prin rutarea multi-hop bazată pe metadate și un strat de transport pluggable, permițând traficului criptat să se deplaseze prin topologii de rețea diverse fără a necesita overlay-uri IP virtuale. Acesta permite în mod specific tunelarea securizată pentru aplicațiile legacy prin împachetarea traficului TCP brut în canale criptate, permițând conectivitatea la rețeaua privată și bypass-ul firewall-ului prin relee de ieșire. Platforma acoperă o gamă largă de capabilități, inclusiv gestionarea identității distribuite, emiterea și verificarea acreditărilor criptografice și execuția actorilor concurenți cu stare. De asemenea, oferă instrumente pentru provizionarea nodurilor la scară cloud și implementarea automată folosind template-uri de infrastructură ca cod.
Directs communication between decoupled actor components using a registry to resolve addresses and relay messages.
Ockam este un framework de rețea zero-trust conceput pentru a securiza tranzitul datelor între aplicații distribuite folosind un overlay de rețea bazat pe identitate. Oferă primitivele necesare pentru a stabili conexiuni autentificate reciproc și criptate end-to-end, eliminând dependența de securitatea tradițională la nivel de rețea. Proiectul se distinge prin utilizarea controlului accesului bazat pe atribute și a acreditărilor verificabile pentru a gestiona încrederea la scară. Implementează rotația identității criptografice pentru a menține continuitatea identității și se integrează cu sisteme de gestionare a cheilor bazate pe hardware pentru a securiza cheile private în enclave sau servicii de gestionare a cheilor în cloud. Platforma acoperă o gamă largă de capabilități, inclusiv rutarea binară multi-hop și bridging-ul de rețea bazat pe relee pentru a conecta rețele disparate. Poate împacheta traficul legacy TCP sau Kafka în tuneluri securizate, permițând serviciilor private să comunice fără a expune porturi de ascultare. În plus, utilizează un model de actor cu stare pentru a procesa mesajele asincron pe noduri distribuite. Implementarea este susținută prin template-uri de infrastructură ca cod pentru provizionarea nodurilor și gateway-urilor securizate în medii cloud.
Sends data to specific worker addresses through a router that resolves locations and manages delivery across different contexts.
useSend is an email marketing platform and transactional delivery system built to manage bulk campaigns and individual messages using AWS SES for delivery. It provides a RESTful API, an SMTP relay gateway, and a visual management interface for designing HTML templates and coordinating email broadcasts. The platform distinguishes itself by translating legacy SMTP traffic into modern API calls and implementing an event-driven webhook system. This system pushes real-time delivery, bounce, and click notifications to external servers using HMAC-SHA256 signed payloads to ensure authenticity. The s
Processes bulk email campaigns and scheduled messages using background workers for high throughput.
Acest proiect este o bibliotecă de compatibilitate PHP care oferă un polyfill pentru extensia nativă ctype. Acesta servește drept strat de portabilitate, implementând funcțiile de bază lipsă ale limbajului pentru a asigura un comportament consistent în diferite medii de server și versiuni PHP. Biblioteca se specializează în validarea tipului de caracter, recreând comportamentul funcțiilor native ale extensiei folosind cod PHP pur. Utilizează detectarea mediului de runtime pentru a identifica dacă extensia nativă lipsește și injectează implementări de fallback direct în namespace-ul global pentru a preveni coliziunile și a menține portabilitatea codului.
Runs background processes that poll transports for pending messages and execute their associated handlers.
This project is a headless WhatsApp client and automation server that exposes multi-device functionality as a REST API gateway. It provides a centralized interface for managing multiple account sessions, automating messages, and developing bots through a standardized protocol. The system distinguishes itself by providing a secure webhook provider that forwards real-time messages and reactions to external URLs using signature verification. It includes specialized media utilities for converting images and GIFs into the WebP format required for stickers, as well as tools to simulate active user
Uses Go routines to process incoming events and outgoing messages without blocking the main execution thread.
The Cloudflare Workers Rust SDK is a toolchain and framework for building high-performance serverless functions using Rust. It compiles code into WebAssembly modules that execute within a sandboxed host environment at the edge. The project provides a typed remote procedure call system for invoking functions across different worker instances and utilizes procedural macros for mapping these distributed interfaces. It uses trait-based bindings and standardized wrappers to provide type-safe access to platform services and web-standard request and response objects. The SDK covers a broad range of
Implements background processing components that consume and execute tasks from message queues.
S3Scanner is a security tool for auditing public access and misconfigured permissions across S3-compatible storage providers. It functions as a storage auditor and security scanner designed to identify open buckets, enumerate publicly accessible objects, and exfiltrate data from misconfigured cloud environments. The project is distinguished by its integration with message brokers, allowing it to consume target lists for large-scale cloud infrastructure audits. It also provides utilities for dumping the entire contents of misconfigured buckets to local directories and calculating the total siz
Uses background workers to consume targets from message queues for distributed high-volume scanning.