Awesome GitHub RepositoriesComputation and Optimization

TensorFlow is a comprehensive machine learning framework designed for the construction, training, and deployment of complex mathematical models. It utilizes a graph-based execution model that represents operations as directed acyclic graphs, enabling automatic differentiation and efficient parallel processing. The system provides high-level interfaces for defining neural network architectures, alongside a robust engine for managing multidimensional array structures and tensor mathematics. The framework distinguishes itself through a scalable distributed runtime that orchestrates workloads acr

PyTorch is a machine learning framework centered on a GPU-ready tensor library that supports multi-dimensional array operations across both CPU and accelerator hardware. It provides a foundational infrastructure for mathematical computation and dynamic neural network construction, utilizing a tape-based automatic differentiation system that allows for flexible, non-static graph execution. The framework is designed for deep integration with Python, enabling natural usage alongside standard scientific computing ecosystems. It distinguishes itself through a comprehensive distributed training sui

Scikit-learn is a machine learning library for predictive data analysis that provides a collection of algorithms for supervised and unsupervised learning. It functions as a comprehensive toolkit for data preprocessing, dimensionality reduction, and model selection, allowing users to classify data objects, predict continuous values, and cluster similar items based on historical patterns. The project is defined by a unified interface design where objects either learn from data, transform data, or chain these operations into sequential workflows. To ensure performance on large or high-dimensiona

Ray is a distributed computing framework designed to scale Python and Java applications across clusters by abstracting task scheduling and resource management. It functions as a resource-aware execution engine that manages task dependencies, placement, and fault tolerance across networked compute nodes. At its core, the system provides a stateful actor model, allowing developers to define classes that run in dedicated processes to maintain and mutate internal state across remote method calls. The framework distinguishes itself through a robust cross-language interoperability layer, enabling f

DeepSpeed is a high-performance library designed to scale deep learning model training and inference across massive clusters of GPUs and compute nodes. It provides a comprehensive suite of tools for distributed training, enabling the execution of models that exceed the memory capacity of single devices through advanced parameter partitioning, pipeline-based model parallelism, and memory-efficient state offloading. The framework distinguishes itself through specialized communication-efficient optimizers and hardware-aware acceleration techniques. By utilizing gradient compression, quantization

ColossalAI is a distributed deep learning framework designed for training and deploying massive artificial intelligence models across clusters of hardware accelerators. It functions as a parallel computing engine that partitions model workloads and data across multiple processors to maximize memory efficiency and throughput. The platform distinguishes itself through a comprehensive suite of parallelization strategies, including multi-dimensional tensor parallelism and pipeline-based model parallelism, which segment neural network layers and stages across devices. To support large-scale genera

This project is a high-performance numerical computing library designed for large-scale scientific and machine learning workloads. It functions as an automatic differentiation framework and a just-in-time compilation engine, transforming high-level Python code into optimized machine instructions. By enforcing pure functional programming patterns and immutable array semantics, the library ensures that mathematical functions remain compatible with automated graph transformations and symbolic differentiation. The platform distinguishes itself through its distributed array computing capabilities,

PyTorch Lightning is a deep learning research framework that provides a structured environment for organizing machine learning code. It functions as a unified trainer orchestrator, centralizing the execution flow by managing the interaction between hardware resources, data loaders, and model components. By decoupling model architecture from training logic, the framework enables researchers to maintain clean, modular codebases that remain portable across different environments. The framework distinguishes itself through a hardware-agnostic abstraction layer that scales deep learning workloads

XGBoost is a distributed machine learning library for implementing scalable gradient boosting decision trees used for regression, classification, and ranking. It functions as a predictive model framework and a cross-language toolkit, providing a core implementation with native bindings for Python, R, Java, Scala, and C++. The system is designed as a GPU-accelerated library that utilizes CUDA and NCCL to speed up the training of decision tree ensembles. It operates as a distributed framework capable of scaling training and prediction across multi-node clusters and GPU environments to process m

This project is a machine learning array framework and tensor computation library designed for high-performance numerical computing. It provides a comprehensive suite of tools for constructing and training neural networks, featuring an automatic differentiation engine that facilitates gradient-based optimization and complex mathematical modeling. The library distinguishes itself through a unified memory architecture that allows data to be shared across CPU and GPU devices without explicit copies, significantly reducing data movement overhead. Its execution model relies on a lazy evaluation en

Paddle is a deep learning framework designed for building, training, and deploying neural networks. It provides a platform for constructing models using tensor-based computations and supports both dynamic and static execution graphs to facilitate research and production workflows. The platform functions as a distributed machine learning system, enabling the scaling of training workloads across multiple nodes and hardware clusters. It includes a comprehensive toolkit for model deployment and optimization, allowing users to convert external model formats, compress trained models for resource-co

This library provides a framework for parameter-efficient fine-tuning, enabling the adaptation of large pretrained models by training only a small subset of parameters. It functions as a distributed model training system and optimization toolkit, designed to reduce the computational and memory requirements typically associated with full model fine-tuning. The project distinguishes itself through a suite of methods for modular adapter composition, including low-rank matrix decomposition and activation-based scaling. It supports the integration of multiple task-specific adapter modules, allowin

Triton is a parallel computing framework and high-level programming language designed for writing custom compute kernels. It functions as a deep learning compiler, translating complex mathematical operations into high-throughput instructions that maximize hardware utilization and memory efficiency on graphics processing units. The framework distinguishes itself through a hardware-agnostic compute abstraction that allows developers to define kernels without manual low-level tuning. It employs just-in-time compilation to generate optimized binary instructions at runtime, utilizing static data f

LightGBM is a high-performance machine learning framework designed for constructing gradient-boosted decision tree ensembles. It provides a platform for training classification, regression, and ranking models, with a focus on memory efficiency and large-scale distributed computing. The framework distinguishes itself through specialized algorithmic strategies, including leaf-wise tree growth and histogram-based decision learning, which prioritize convergence speed. It optimizes memory usage by bundling mutually exclusive features and employs gradient-based sampling to reduce training complexit

Horovod is a distributed deep learning framework and gradient synchronizer designed to scale model training across multiple GPUs and compute nodes. It functions as a distributed training orchestrator and an elastic training engine, utilizing an MPI collective communication library to synchronize weights and gradients across TensorFlow, PyTorch, Keras, and MXNet models. The system distinguishes itself through dynamic elastic scaling, which allows it to adjust the number of active workers at runtime and recover from node failures. It optimizes communication efficiency using tensor fusion batchi

DGL is a Python library for building and training graph neural networks. It functions as a graph message passing framework and a geometric deep learning tool, enabling the development of models that analyze graph-structured data. The library is designed for large-scale graph processing, utilizing distributed training and neighbor sampling to handle datasets with billions of edges. It provides specialized support for heterogeneous graph modeling, allowing for the representation of complex real-world entities with multiple node and edge types. Its capabilities cover a wide range of graph tasks

Dask est un framework de calcul parallèle et un planificateur de tâches distribué conçu pour mettre à l'échelle les flux de travail de science des données Python, des machines uniques aux grands clusters. Il fonctionne comme un gestionnaire de ressources de cluster qui orchestre la logique computationnelle en représentant les tâches et leurs dépendances sous forme de graphes acycliques dirigés. Cette architecture permet au système d'automatiser la distribution des charges de travail sur le matériel disponible tout en gérant des exigences d'exécution complexes. Le projet se distingue par un moteur d'évaluation paresseuse qui diffère les opérations sur les données jusqu'à ce qu'elles soient explicitement demandées, permettant une optimisation globale du graphe et une allocation efficace des ressources. Il intègre le déversement de données conscient de la mémoire pour éviter les plantages du système lors du traitement de jeux de données dépassant la mémoire disponible, et il utilise la fusion de graphes de tâches pour combiner des séquences d'opérations en étapes d'exécution uniques, minimisant la surcharge de planification et la communication entre nœuds. La plateforme fournit une surface de capacités complète pour l'analyse de données à grande échelle, incluant le support pour l'apprentissage automatique distribué, l'intégration du calcul haute performance et le traitement de données parallèle. Elle offre des outils étendus pour la gestion du cycle de vie des clusters, le profilage des performances et la surveillance en temps réel de l'exécution des tâches. Les utilisateurs peuvent déployer ces environnements sur diverses infrastructures, incluant le matériel local, les fournisseurs cloud, les systèmes conteneurisés et les clusters de calcul haute performance.

TensorRT est un moteur d'inférence d'apprentissage profond et un kit de développement logiciel conçu pour optimiser et déployer des réseaux de neurones pour une exécution haute performance sur les GPU NVIDIA. Il fonctionne comme un framework d'accélération GPU qui réduit la latence et augmente le débit des modèles entraînés lors du déploiement en production. Le toolkit importe des modèles depuis le format Open Neural Network Exchange et les transforme en moteurs optimisés. Il utilise l'optimisation de modèles basée sur les graphes, la génération de noyaux de fusion de couches et la quantification basée sur la précision pour convertir les poids en virgule flottante en formats de précision inférieure. Le framework fournit des capacités de sérialisation de moteur spécifique au matériel et prend en charge l'extension des capacités d'inférence via des plugins personnalisés pour des couches de réseaux de neurones spécialisées.

CuPy est une bibliothèque de calcul de tableaux CUDA qui implémente une interface compatible avec NumPy pour exécuter des opérations sur tableaux et du calcul numérique sur des GPU NVIDIA. Elle sert de bibliothèque numérique accélérée par GPU et d'implémentation SciPy basée sur CUDA, déchargeant les calculs lourds sur le matériel graphique pour augmenter la vitesse de traitement pour les charges de travail scientifiques et d'ingénierie. La bibliothèque permet l'échange de tenseurs multi-framework, permettant aux tampons de données d'être partagés entre différents frameworks d'apprentissage profond en utilisant des mises en page mémoire standardisées pour éviter les copies mémoire. Elle prend également en charge l'intégration de noyaux GPU personnalisés, permettant aux données de tableaux d'être connectées à des API de bas niveau pour un contrôle précis sur l'exécution matérielle. Globalement, le projet couvre le traitement de tableaux haute performance et les flux de travail de calcul scientifique. Ses capacités incluent l'accélération des calculs de tableaux et la fourniture d'outils pour les calculs numériques à grande échelle.

Numba est un compilateur juste-à-temps (JIT) qui traduit des fonctions Python de haut niveau en code machine optimisé lors de l'exécution. En tirant parti de l'infrastructure de compilation LLVM, il fournit un framework pour accélérer le traitement des données numériques et les calculs mathématiques, permettant des niveaux de performance comparables aux langages compilés statiquement. Le projet se distingue par sa capacité à effectuer une spécialisation basée sur l'inférence de type, qui génère des instructions machine adaptées aux types de données spécifiques utilisés lors de l'exécution. Il emploie un pipeline de compilation paresseuse qui diffère la traduction jusqu'au moment de l'invocation, minimisant la surcharge au démarrage tout en maintenant des performances constantes sur diverses architectures de processeurs et systèmes d'exploitation. Au-delà de la compilation de base, le toolkit offre un support étendu pour l'accélération matérielle en distribuant les opérations itératives et les expressions de tableaux sur plusieurs cœurs CPU et unités de traitement graphique. Il utilise des stratégies de vectorisation et de parallélisation pour maximiser le débit pour les grands jeux de données numériques, permettant aux développeurs de cibler du matériel spécialisé directement depuis du code standard.