Awesome GitHub RepositoriesTask Queue Management

ToolJet is a low-code development platform designed for building and deploying internal business applications. It provides a visual interface where users can drag and drop components to design layouts, connect to various data sources, and execute custom logic. The platform is built on a containerized architecture, ensuring that applications remain portable and consistent across different cloud and server environments. The platform distinguishes itself through integrated artificial intelligence capabilities that assist in the generation of user interfaces, database schemas, and data queries fr

Deepagents is an LLM agent orchestration platform and stateful application server designed for deploying and managing AI agents built with computational graphs. It provides a containerized runtime environment that handles agent execution, state persistence, and the versioning of AI assistants. The platform distinguishes itself through deep integration with the Model Context Protocol, allowing agents to function as servers that expose tools and capabilities to external clients. It features a sophisticated observability suite for capturing execution traces, performing LLM-based evaluations agai

Crawlee is a web scraping framework designed for building scalable, reliable, and distributed data extraction pipelines. It provides a unified interface for managing headless browser automation and lightweight HTTP requests, allowing developers to handle complex web navigation, dynamic content rendering, and large-scale data collection within a single, modular architecture. The project distinguishes itself through its resource-aware concurrency controller, which dynamically scales task execution based on real-time CPU and memory usage to prevent host machine exhaustion. It also features a rob

Prefect is a workflow orchestration platform designed to define, schedule, and monitor complex data pipelines as Python code. It functions as a container-native engine that wraps individual tasks in isolated environments, ensuring consistent dependencies and resource allocation across diverse infrastructure. By utilizing a state-machine-based orchestration model, the system tracks execution progress through discrete transitions and persistent event logs to maintain reliable and observable task processing. The platform distinguishes itself through a decoupled worker-API architecture, which sep

DouyinLiveRecorder is a multi-platform live stream recorder designed to monitor and save broadcasts from various global streaming platforms to local storage. It functions as an automated stream watchdog that continuously tracks broadcast statuses to initiate recording immediately when a stream goes live. The tool is a proxy-aware capture system capable of routing network requests through external servers to access and record content restricted to specific geographic regions. It utilizes platform-specific resolvers to map API patterns to stream identifiers, ensuring content can be archived fro

RoadRunner is a high-performance application server and process manager designed to serve PHP applications using a persistent worker model. It eliminates bootload overhead and initialization time by keeping application processes alive between requests, acting as a protocol-agnostic proxy that routes traffic to a pool of supervised workers. The server is built with a plugin-based modular architecture, allowing it to be extended with custom Go plugins and compiled into tailored binaries. It distinguishes itself by providing a unified execution model for a wide array of communication protocols,

Flower is a monitoring and administration tool for Celery task queues. It provides a real-time web dashboard and a REST API to monitor distributed task clusters, manage worker instances, and observe message broker health. The project distinguishes itself by offering centralized control over the task lifecycle, allowing users to trigger, revoke, or terminate tasks and apply execution rate limits. It also includes a Prometheus metrics exporter to surface internal performance and status data for external monitoring and alerting systems. The tool covers a broad range of observability and managem

Soketi est un serveur WebSocket haute performance et un courtier d'événements en temps réel qui implémente le protocole Pusher. Il fonctionne comme une passerelle WebSocket multi-tenant, permettant à plusieurs applications isolées de gérer des connexions client persistantes et de diffuser des événements à travers des canaux publics, privés et de présence. Le projet se distingue par son architecture distribuée, utilisant la synchronisation d'état pub-sub via Redis ou NATS pour monter en charge horizontalement sur plusieurs instances de serveur. Il dispose d'un chiffrement symétrique des charges utiles pour les canaux privés, garantissant que le serveur agit comme un relais sans accéder au contenu en clair, et fournit une isolation des ressources multi-tenant pour appliquer des limites granulaires de connexion et de charge utile par application. Le système couvre de larges domaines de capacités, incluant l'observabilité en temps réel via des métriques compatibles Prometheus, le déchargement asynchrone de webhooks vers Redis ou SQS, et un stockage de configuration flexible utilisant des bases de données relationnelles ou des magasins clé-valeur. Il inclut également des outils de gestion du trafic tels que la protection des ressources basée sur la mémoire, la limitation de débit (rate limiting) et des procédures d'arrêt gracieux. Le déploiement est pris en charge via des images de conteneur Docker et des charts Helm Kubernetes.

PraisonAI is an autonomous AI agent platform that coordinates multiple LLM-powered agents for research, planning, and execution of complex workflows. It functions as a multi-agent orchestration framework, a workflow builder, and a Model Context Protocol server, while also providing retrieval-augmented generation through vector knowledge bases. Agents can interact via CLI, web, or standardized protocols with sandboxed code execution. The platform distinguishes itself with a rich set of agent communication protocols, including A2A, REST, WebSocket, voice and telephony integration, and MCP, allo

Agenta is a Prompt Ops lifecycle manager and prompt management platform that decouples prompt engineering from application code. It serves as a centralized system for developing, versioning, and deploying prompt templates and model configurations across different environments. The platform functions as an AI agent orchestrator with a visual interface for building agent workflows and connecting models to external tools. It further acts as an evaluation framework and observability tool, utilizing OpenTelemetry to capture execution traces, monitor latency, and track token costs. The system cove

Good Job est un processeur de tâches d'arrière-plan pour Ruby on Rails qui utilise une base de données PostgreSQL comme moteur de stockage principal. En tirant parti des transactions de base de données relationnelle, il garantit une exécution de tâches persistante et fiable, s'intégrant directement au framework Active Job pour gérer les opérations asynchrones et la planification de tâches récurrentes au sein des environnements d'application existants. Le système se distingue par un modèle d'exécution en processus qui permet aux travailleurs d'arrière-plan de s'exécuter au sein du même processus que le serveur web, simplifiant le déploiement en supprimant le besoin de services de travail séparés. Il emploie une exécution de travailleur multithreadée et des verrous consultatifs au niveau de la base de données pour coordonner les tâches sur des processus distribués, garantissant une exécution unique pour les tâches récurrentes et une utilisation efficace des ressources. La bibliothèque fournit des contrôles opérationnels complets, incluant la capacité de regrouper des tâches liées en lots pour un suivi collectif du cycle de vie et l'utilisation de l'insertion en masse pour optimiser l'ingestion de tâches à haute fréquence. Les administrateurs peuvent gérer les limites de concurrence, allouer des pools de threads dédiés pour des files d'attente spécifiques et surveiller la santé du système via un tableau de bord web intégré et personnalisable. Le projet inclut une interface intégrée pour inspecter, mettre en pause et dépanner les tâches en temps réel, ainsi qu'une rétention d'historique configurable pour l'audit et l'analyse des performances.