Awesome GitHub RepositoriesJust-In-Time Kernel Compilers

JAX is a hardware-accelerated array library and automatic differentiation system for numerical computing. It provides a framework compatible with NumPy that extends array operations with a just-in-time compiler to transform Python functions into optimized kernels for execution on GPU and TPU accelerators. The system differentiates itself through the use of an XLA-based compiler and a single program multiple data sharding model. These capabilities allow the library to distribute large-scale computations across multiple hardware accelerators using both automatic parallelization and manual shard

Tinygrad is a deep learning framework and tensor computation engine designed for building and training neural networks. It functions as a hardware abstraction layer that manages device memory, command queues, and kernel dispatching across heterogeneous computing architectures. By utilizing a lazy-evaluation approach, the framework constructs computational graphs that defer execution until data is explicitly required, allowing it to process only the necessary operations for a given result. The project distinguishes itself through a just-in-time compilation layer that transforms abstract comput

Sglang is a high-performance inference engine and serving system designed for large language and multimodal models. It provides a programmable interface for orchestrating complex generation workflows, enabling developers to coordinate multi-turn dialogues, tool invocations, and reasoning chains through a domain-specific language. The platform is built to support production-scale deployments, offering an OpenAI-compatible API that allows for integration with existing application ecosystems. The system distinguishes itself through a disaggregated architecture that separates compute-intensive pr

This project is a deep learning library designed for training neural networks on irregular data structures, including graphs, 3D meshes, and point clouds. It functions as an extension to the PyTorch framework, providing specialized layers and kernels that enable the processing of complex, non-Euclidean information. The library distinguishes itself through a geometric deep learning toolkit that manages the unique requirements of graph-based data. It utilizes sparse matrix-based message passing to aggregate information across nodes and employs dynamic computational graph construction to accommo

Triton is a parallel computing framework and high-level programming language designed for writing custom compute kernels. It functions as a deep learning compiler, translating complex mathematical operations into high-throughput instructions that maximize hardware utilization and memory efficiency on graphics processing units. The framework distinguishes itself through a hardware-agnostic compute abstraction that allows developers to define kernels without manual low-level tuning. It employs just-in-time compilation to generate optimized binary instructions at runtime, utilizing static data f

This project serves as a comprehensive educational resource and technical guide for mastering deep learning through the PyTorch framework. It provides structured tutorials and practical code examples designed to teach core machine learning principles, ranging from fundamental tensor operations to the construction of complex neural network architectures. The repository distinguishes itself by bridging the gap between theoretical concepts and hands-on implementation. It covers the development of generative applications, such as image synthesis and style transfer, while offering guidance on opti

DeepEP is a distributed model accelerator and expert-parallel communication library designed to optimize the training and inference of large-scale neural networks. It provides specialized GPU communication kernels and a remote GPU memory interface to facilitate high-throughput data exchange between hardware nodes. The system utilizes dynamic kernel generation to compile optimized GPU kernels during execution, removing the need for separate installation compilation steps. It implements virtual-lane traffic isolation to prevent interference between different data streams and employs routing met

jetson-inference is a set of libraries and tools for executing optimized deep learning models on embedded GPU hardware. Its primary purpose is to enable real-time computer vision and AI inference at the edge with low latency and high throughput. The project distinguishes itself through high-performance streaming analytics and the ability to execute concurrent AI pipelines on auto-grade silicon. It provides specialized support for multi-sensor stream processing, utilizing zero-copy data transport to load camera frames directly into GPU memory. The codebase covers a broad surface of capabiliti

DeepGEMM is a suite of specialized GPU kernels and a just-in-time compiler designed for low-precision matrix operations, Mixture-of-Experts models, and attention processing. It provides a library of high-performance matrix multiplication kernels using FP8 precision to increase compute throughput and reduce memory usage. The project features a JIT CUDA kernel compiler that generates and loads optimized compute kernels at runtime to eliminate the need for manual compilation during installation. It includes specialized implementations for grouped matrix multiplication that process multiple group

FlashInfer is a library of high-performance GPU kernels purpose-built for accelerating large language model inference. It provides optimized implementations for attention operations (including flash attention, page attention, multi-head latent attention, and cascade attention) using paged key-value caches, fused kernel composition, and just-in-time compilation. The library also includes specialized kernels for mixture-of-experts layers, block-scaled low-precision quantization (FP8, FP4), and distributed collective communication. What distinguishes FlashInfer is its fused all-reduce communicat

Flux.jl est un framework de deep learning et une boîte à outils de calcul numérique écrite en Julia. Il sert de bibliothèque de machine learning pour concevoir et entraîner des réseaux de neurones, fournissant un système pour la différenciation automatique afin d'optimiser les paramètres du modèle. Le framework permet le développement de deep learning et la recherche en machine learning en représentant les couches comme des fonctions paramétrées. Il prend en charge le machine learning scientifique, intégrant les réseaux de neurones dans des flux de travail pour résoudre des problèmes physiques et mathématiques. La boîte à outils fournit une accélération GPU native pour les calculs de tenseurs et utilise la différenciation automatique en mode inverse pour calculer les gradients. Elle emploie des noyaux compilés juste-à-temps (JIT) pour générer du code machine optimisé pour les opérations mathématiques sur matériel CPU et GPU.

This project is a high-performance C++ and CUDA neural network library designed for fast training and inference of small networks on NVIDIA GPUs. It serves as a specialized backend for neural radiance fields and coordinate-based networks, providing a fused GPU kernel library and a hash grid encoder for transforming raw input dimensions into high-dimensional representations. The library distinguishes itself through the use of C++ template metaprogramming and fused-kernel execution, which merge neural network layers into single GPU device functions to eliminate memory bottlenecks. It leverages

oneDNN is a library for deep learning acceleration that provides optimized building blocks for neural network training and inference. It manages tensor computation across CPU and GPU hardware, enabling the execution of high-performance primitives for model training and neural network inference optimization. The project distinguishes itself through hardware-specific kernel optimization and the use of just-in-time compilation to target specific processor instruction sets. It supports quantized neural network execution using both static and dynamic quantization to reduce memory usage and increas

Neon is a deep learning framework and hardware-abstraction machine learning stack used for designing, training, and deploying neural network architectures. It functions as a graph-based computation engine that utilizes just-in-time kernel compilation to optimize machine code for tensors. The platform decouples model definitions from execution kernels, allowing it to support multiple CPU and GPU backends. This architecture enables the distribution of computational workloads across parallelized hardware environments to increase processing speed and overall efficiency. The system covers the ful

This project is a comprehensive collection of technical manuals, tutorials, and guides for implementing machine learning models and numerical computations using the TensorFlow framework. It serves as an educational resource and technical library designed to help developers build and maintain models across diverse hardware environments. The repository includes a multilingual technical guide and a collaborative translation project focused on standardizing industry terminology. These efforts ensure that complex machine learning concepts and technical documentation are accessible and accurately i

Ce projet est une ressource éducative consistant en un cursus structuré de notebooks interactifs conçus pour enseigner les concepts d'apprentissage profond et les architectures de réseaux de neurones. Il se concentre sur la fourniture d'une expérience pratique avec le framework TensorFlow 2 et l'API Keras, guidant les utilisateurs à travers des exercices pratiques pour maîtriser les techniques d'apprentissage automatique. Le dépôt se distingue en combinant le contenu pédagogique avec les exigences techniques pour le calcul haute performance. Il inclut des guides spécifiques pour configurer les environnements de développement locaux afin de prendre en charge l'entraînement accéléré par le matériel sur des unités de traitement graphique (GPU), garantissant que les apprenants peuvent effectuer des calculs mathématiques intensifs pour leurs modèles. La plateforme prend en charge une gamme de workflows de développement, du prototypage rapide d'architectures de réseaux de neurones à l'exécution d'expériences en science des données. Elle fournit des utilitaires pour rendre les équations mathématiques et le code au sein des documents, et elle s'adapte à la fois au développement local hors ligne et aux environnements d'exécution basés sur le cloud.

Segment Anything Fast est un moteur d'inférence de vision par ordinateur et un framework de segmentation d'image haute performance construit pour PyTorch. Il fournit un environnement spécialisé pour l'isolation automatisée d'objets et la génération de masques, conçu pour traiter des jeux de données visuels à grande échelle avec un débit accru. Le projet se distingue par une suite de stratégies d'optimisation au niveau système qui accélèrent les performances des modèles de deep learning. En utilisant la compilation de modèles basée sur des graphes, la fusion de noyaux juste-à-temps (JIT) et la quantification consciente du matériel, il réduit la latence computationnelle et l'empreinte mémoire. Ces techniques sont complétées par l'arithmétique en précision mixte et le parallélisme des entrées par lots, qui permettent au moteur de saturer les ressources matérielles lors de tâches de segmentation complexes. Au-delà de ses capacités de segmentation de base, la bibliothèque inclut des outils de diagnostic intégrés pour le profilage des performances du deep learning. Ces utilitaires permettent aux utilisateurs de surveiller les modèles d'allocation mémoire et de capturer des traces de timing d'exécution, offrant une visibilité sur les goulots d'étranglement du système lors de l'analyse de données à grande échelle.