Awesome GitHub RepositoriesDistributed Execution Runtimes

Ce projet est un répertoire complet, organisé par la communauté, qui structure un vaste paysage de bibliothèques, frameworks et outils logiciels Python. Il sert de base de connaissances centralisée conçue pour faciliter la navigation dans l'écosystème et accélérer la découverte par les développeurs tout au long du cycle de vie du développement logiciel. Le répertoire se distingue en fournissant un index structuré de ressources classées par domaine technique, allant des utilitaires de développement fondamentaux aux domaines d'ingénierie spécialisés. Il couvre des capacités de haut niveau, notamment l'intelligence artificielle, la science des données, le développement web et la gestion d'infrastructure, permettant aux développeurs d'identifier des solutions éprouvées pour des défis techniques spécifiques. Le projet englobe une large surface de capacités, notamment des outils pour la gestion des dépendances, l'analyse de code statique et les tests automatisés. Il catalogue également des ressources pour le stockage de données persistantes, l'orchestration d'infrastructure cloud et le développement d'interfaces, fournissant une référence unifiée pour la construction et la maintenance de systèmes logiciels complexes.

TensorFlow is a comprehensive machine learning framework designed for the construction, training, and deployment of complex mathematical models. It utilizes a graph-based execution model that represents operations as directed acyclic graphs, enabling automatic differentiation and efficient parallel processing. The system provides high-level interfaces for defining neural network architectures, alongside a robust engine for managing multidimensional array structures and tensor mathematics. The framework distinguishes itself through a scalable distributed runtime that orchestrates workloads acr

PaddleOCR is a comprehensive optical character recognition framework designed for detecting and transcribing text from images and documents into structured, machine-readable formats. It provides a modular computer vision pipeline that decouples image preprocessing, text detection, and character recognition into independent, configurable stages. This architecture supports automated document digitization and multilingual text recognition, capable of identifying text in over one hundred languages across diverse environments ranging from scanned documents to industrial scenes. The framework disti

JAX is a hardware-accelerated array library and automatic differentiation system for numerical computing. It provides a framework compatible with NumPy that extends array operations with a just-in-time compiler to transform Python functions into optimized kernels for execution on GPU and TPU accelerators. The system differentiates itself through the use of an XLA-based compiler and a single program multiple data sharding model. These capabilities allow the library to distribute large-scale computations across multiple hardware accelerators using both automatic parallelization and manual shard

This project is a machine learning array framework and tensor computation library designed for high-performance numerical computing. It provides a comprehensive suite of tools for constructing and training neural networks, featuring an automatic differentiation engine that facilitates gradient-based optimization and complex mathematical modeling. The library distinguishes itself through a unified memory architecture that allows data to be shared across CPU and GPU devices without explicit copies, significantly reducing data movement overhead. Its execution model relies on a lazy evaluation en

Paddle is a deep learning framework designed for building, training, and deploying neural networks. It provides a platform for constructing models using tensor-based computations and supports both dynamic and static execution graphs to facilitate research and production workflows. The platform functions as a distributed machine learning system, enabling the scaling of training workloads across multiple nodes and hardware clusters. It includes a comprehensive toolkit for model deployment and optimization, allowing users to convert external model formats, compress trained models for resource-co

Bend is a high-level parallel programming language and compiler designed to execute code across multi-core CPUs and GPUs automatically. By translating functional source code into a graph-based intermediate representation, it enables massive parallel execution without requiring manual management of threads, locks, or atomic operations. The runtime operates as an interaction net engine, where computations are represented as networks of nodes that reduce through local rewriting rules. This model utilizes a work-stealing scheduler to distribute tasks across thousands of hardware threads, ensuring

ds4 is a local inference engine for DeepSeek models that includes a distributed runtime for splitting transformer layers across networked computers. It functions as a reasoning controller with a local weight streamer and an API server that streams chat completions via industry standard endpoints. The system employs a memory management model that loads model experts from disk on demand to execute models that exceed available system RAM. It provides controls for reasoning effort and model behavior steering, allowing the modification of response characteristics through activation directions. Th

OSS-Fuzz is a distributed, containerized platform for continuous fuzzing and memory safety analysis. It functions as a bug hunting infrastructure that identifies security vulnerabilities and stability bugs through automated, coverage-guided fuzz testing across a scalable cluster of containers. The system provides a continuous security testing pipeline that manages the entire lifecycle of vulnerability discovery, from bootstrapping project templates and compiling targets to executing long-running batch tests. It specifically focuses on memory safety, utilizing sanitizers to detect buffer overf

OpenVINO is an AI inference engine and model serving platform designed to execute optimized deep learning models across CPUs, GPUs, and NPUs through a unified API. It includes a model optimization toolkit for converting, quantizing, and compressing models from various frameworks, alongside a specialized generative AI runtime for large language models. The project distinguishes itself through a plugin-based hardware acceleration layer that maps neural network operations to vendor-specific drivers. It features advanced execution mechanisms such as continuous batching, speculative decoding, and

UFO is a multi-device task orchestrator and LLM agent orchestration framework designed to decompose natural language requests into executable task graphs. It functions as a cross-platform UI automation tool capable of performing interactions on Windows and mobile devices while routing tasks to distributed agents based on their hardware and software capabilities. The system is distinguished by its RAG-enhanced agent architecture, which integrates external documentation and previous execution traces to improve decision-making. It employs a hybrid UI detection approach that combines computer vis

trlx est une bibliothèque d'apprentissage par renforcement et un framework d'entraînement conçu pour aligner les grands modèles de langage en utilisant le feedback humain. Il sert d'entraîneur distribué et d'orchestrateur de calcul pour scaler des modèles à haut nombre de paramètres sur plusieurs GPU et nœuds. Le projet fournit des outils pour l'apprentissage par renforcement à partir de feedback humain et l'alignement de modèles. Il implémente l'optimisation basée sur un modèle de récompense et l'optimisation de politique proximale (PPO) pour affiner le comportement du modèle en fonction de récompenses orientées vers des objectifs ou de jeux de données étiquetés par des humains. Le framework couvre des stratégies d'entraînement distribué, notamment le parallélisme de modèle, le sharding de paramètres et la synchronisation de gradients multi-nœuds. Il intègre également des contraintes comme la divergence KL pour gérer la dérive du modèle pendant le processus d'apprentissage par renforcement.

Aurae is a memory-safe distributed systems runtime daemon written in Rust that acts as a container and process orchestrator. It manages and schedules workloads, containers, and virtual machines across distributed infrastructure nodes using remote procedure calls and isolated kernel-level boundaries. The platform provides a mutual transport-layer security gateway that enforces cryptographic identity and socket-level authentication across distributed infrastructure. It includes enterprise workload isolation features to secure control planes and manage multi-tenant processes on host operating sy