Awesome GitHub RepositoriesPerformance Optimization

TVM is a machine learning compiler framework designed to convert deep learning models from various frameworks into optimized machine code. It functions as a cross-platform deployment engine that transforms high-level model definitions into efficient, hardware-specific binaries for diverse computing architectures. The system utilizes a multi-level compilation pipeline that decouples algorithm logic from hardware implementation through tensor-operator abstractions. It employs a graph-level intermediate representation to perform cross-operator optimizations and memory planning before lowering co

Scalene is a high-performance diagnostic utility designed to measure resource consumption during the execution of Python applications. It functions as a line-level monitor, providing granular insights that pinpoint the specific source code responsible for performance overhead. The tool distinguishes itself through statistical profiling that captures stack traces and resource usage without requiring manual instrumentation of the source code. It tracks CPU, GPU, and memory consumption by intercepting library-level calls and hardware driver commands, allowing for the analysis of both managed and

bitsandbytes is a deep learning quantization tool and library designed to reduce the memory footprint of large language models. It serves as a GPU memory optimizer and quantization framework, compressing model weights and features to 8-bit and 4-bit precision to enable inference and training on hardware with limited memory. The project provides a framework for low-rank adaptation, allowing the fine-tuning of quantized models by combining 4-bit weights with small trainable matrices. It further distinguishes itself through memory paging, which moves optimizer states between CPU and GPU memory t

AI-System est une ressource éducative et une boîte à outils conçue pour apprendre les fondements matériels et logiciels des systèmes d'apprentissage profond. Il fournit un programme et des exercices pratiques pour construire une infrastructure IA, allant du développement de noyaux CUDA bas niveau à la gestion système haut niveau. Le projet inclut une boîte à outils pour développer des opérations sur tenseurs et optimiser les performances GPU via la programmation matérielle directe. Il dispose également d'un framework pour l'entraînement distribué, se concentrant sur la planification des ressources et les protocoles de communication pour gérer des modèles à grande échelle sur plusieurs nœuds de calcul. Le système couvre l'analyse de sécurité de l'IA pour identifier les vulnérabilités de confidentialité et les attaques adverses, ainsi que l'optimisation des performances via la compilation consciente du matériel, la compression pilotée par la sparsité et les graphes de calcul basés sur des tenseurs. Il fournit en outre des outils pour gérer l'infrastructure IA et coordonner les stratégies de déploiement pour des environnements d'inférence haute performance.

Representation and Reference Lowering of ONNX Models in MLIR Compiler Infrastructure

Home for OctoML PyTorch Profiler