Awesome GitHub RepositoriesTask Queues

React هي مكتبة JavaScript لبناء واجهات المستخدم تعتمد على معمارية قائمة على المكونات وتدفق بيانات أحادي الاتجاه.

Umi-OCR is an optical character recognition engine designed to convert visual text from images and documents into machine-readable character data. It functions as a local-first toolkit, processing all visual data directly on the host machine using embedded neural network models to maintain privacy and offline availability. The project distinguishes itself through its focus on automated document digitization and integrated barcode and QR code decoding. By utilizing a modular, Python-based orchestration layer, it enables users to transform static image files and multi-page documents into search

Marker is a comprehensive document processing platform designed to automate the conversion, extraction, and structuring of data from complex files. It functions as an orchestration engine that chains modular processing steps into versioned, reusable pipelines, allowing organizations to standardize document handling and automate repetitive business tasks at scale. The platform distinguishes itself through its support for secure, private infrastructure deployment, enabling users to run containerized services within their own environments to maintain strict data privacy. It features specialized

Celery is an asynchronous job processor and distributed task queue designed to offload time-consuming operations to background worker nodes. By utilizing a message-passing architecture, it decouples task producers from consumers, allowing applications to maintain responsiveness while scaling workloads across multiple isolated environments. The system functions as a distributed workload orchestrator that manages the lifecycle of deferred operations through persistent queues. It distinguishes itself by providing a pluggable transport abstraction, which allows the core task logic to remain indep

NSQ is a distributed, brokerless messaging platform designed for high-throughput, fault-tolerant communication. By utilizing a decentralized topology, it eliminates single points of failure and allows for horizontal scaling across clusters. The system organizes message streams into topics and channels, effectively decoupling producers from consumers to support both streaming and job-oriented workloads. The platform distinguishes itself through a lookup-service-based discovery mechanism that enables clients to dynamically locate producers at runtime without requiring centralized coordination.

Pyvideotrans is an automated video localization platform designed to transcribe, translate, and dub media content for international distribution. It functions as an end-to-end workflow that combines speech recognition, text translation, and synthetic voice generation to process video files into localized versions. The system distinguishes itself by offering a choice between local model inference for privacy and integration with third-party cloud services via user-provided credentials. This architecture allows users to maintain control over their billing and data security while utilizing modul

Taskflow is a C++ task-parallel framework designed to build high-performance parallel workflows and complex dependency graphs. It provides a programming model that organizes computational work into directed acyclic graphs, enabling developers to manage concurrency, resource scheduling, and task dependencies across multi-core CPUs and GPU accelerators. The framework distinguishes itself through its ability to orchestrate heterogeneous systems, allowing for the integration of hardware-accelerated kernels and memory operations into unified execution pipelines. It supports dynamic runtime subflow

HVM2 is a high-performance execution environment for pure functional programs, implemented as a systems-level runtime in Rust. It functions as a massively parallel functional runtime that uses interaction combinators to achieve automatic parallelism across multi-core CPUs and GPUs. The project distinguishes itself by using a graph-rewriting computational model to execute programs via local reduction rules, which eliminates the need for manual locks or atomic operations. It employs beta-optimal reduction and lazy evaluation to optimize higher-order functions and eliminate redundant computation

Crossbeam is a concurrency toolkit for Rust providing low-level primitives for writing multi-threaded programs. It focuses on lock-free data structures and memory management primitives designed for shared-memory concurrent environments. The project includes a work-stealing scheduler that uses double-ended queues to balance workloads across multiple processor cores. This system enables the implementation of work-stealing deques to distribute tasks and prevent bottlenecks. The toolkit covers broader capabilities for parallel algorithm development, multi-threaded task scheduling, and general co

Fastdom is a task scheduler and rendering optimizer designed to prevent layout thrashing by batching document reads and writes. It functions as a coordination system that separates document measurements and mutations into distinct queues, ensuring that asynchronous operations are executed in a way that minimizes browser reflows. The project provides a performance profiling capability that detects improper document API access. By enforcing strict execution phases, it identifies layout thrashing bottlenecks and flags unauthorized DOM access occurring outside of the coordinated batching cycles.

Hazelcast is a distributed data platform that combines an in-memory data grid with a stream processing engine to support real-time analytics and event-driven applications. It functions as a partitioned, distributed key-value store that replicates data across cluster nodes to provide low-latency access and high availability. The platform also serves as a distributed SQL query engine, allowing users to execute standard SQL statements against both in-memory datasets and external data sources. What distinguishes Hazelcast is its use of a distributed consensus subsystem to maintain strongly consis

async-std is a Rust asynchronous runtime and non-blocking I/O library. It serves as an asynchronous standard library, providing a set of alternatives to the core Rust library for managing concurrent tasks, networking, and file system access. The project implements a standard-library-mirrored API, offering asynchronous versions of existing synchronous types to maintain a familiar interface. This approach allows for asynchronous Rust development using an interface that reflects the ergonomics of the language's standard library. The runtime includes a task executor for scheduling and executing

RavenDB is a multi-model NoSQL document database designed for high-performance, ACID-compliant data storage. It persists structured information as schema-flexible JSON documents and utilizes a unit-of-work session pattern to track entity changes and batch modifications into atomic transactions. The platform is built on a distributed architecture that supports horizontal scaling through sharding and ensures high availability via multi-node, master-to-master cluster replication. The database distinguishes itself through a self-optimizing query engine that automatically creates and maintains ind

This repository serves as a tutorial and sample collection for designing cloud-native distributed systems, featuring deployment configurations and sample code focused on recurring microservice patterns. It functions as an architectural pattern guide for building distributed applications on container platforms. The collection demonstrates standard patterns through container configurations and application manifests, covering multi-container pod designs and container interactions. It details infrastructure and orchestration approaches such as running auxiliary containers alongside primary applic