Awesome GitHub RepositoriesJust-In-Time Kernel Compilers

JAX is a hardware-accelerated array library and automatic differentiation system for numerical computing. It provides a framework compatible with NumPy that extends array operations with a just-in-time compiler to transform Python functions into optimized kernels for execution on GPU and TPU accelerators. The system differentiates itself through the use of an XLA-based compiler and a single program multiple data sharding model. These capabilities allow the library to distribute large-scale computations across multiple hardware accelerators using both automatic parallelization and manual shard

Tinygrad is a deep learning framework and tensor computation engine designed for building and training neural networks. It functions as a hardware abstraction layer that manages device memory, command queues, and kernel dispatching across heterogeneous computing architectures. By utilizing a lazy-evaluation approach, the framework constructs computational graphs that defer execution until data is explicitly required, allowing it to process only the necessary operations for a given result. The project distinguishes itself through a just-in-time compilation layer that transforms abstract comput

Sglang is a high-performance inference engine and serving system designed for large language and multimodal models. It provides a programmable interface for orchestrating complex generation workflows, enabling developers to coordinate multi-turn dialogues, tool invocations, and reasoning chains through a domain-specific language. The platform is built to support production-scale deployments, offering an OpenAI-compatible API that allows for integration with existing application ecosystems. The system distinguishes itself through a disaggregated architecture that separates compute-intensive pr

This project is a deep learning library designed for training neural networks on irregular data structures, including graphs, 3D meshes, and point clouds. It functions as an extension to the PyTorch framework, providing specialized layers and kernels that enable the processing of complex, non-Euclidean information. The library distinguishes itself through a geometric deep learning toolkit that manages the unique requirements of graph-based data. It utilizes sparse matrix-based message passing to aggregate information across nodes and employs dynamic computational graph construction to accommo

Triton is a parallel computing framework and high-level programming language designed for writing custom compute kernels. It functions as a deep learning compiler, translating complex mathematical operations into high-throughput instructions that maximize hardware utilization and memory efficiency on graphics processing units. The framework distinguishes itself through a hardware-agnostic compute abstraction that allows developers to define kernels without manual low-level tuning. It employs just-in-time compilation to generate optimized binary instructions at runtime, utilizing static data f

This project serves as a comprehensive educational resource and technical guide for mastering deep learning through the PyTorch framework. It provides structured tutorials and practical code examples designed to teach core machine learning principles, ranging from fundamental tensor operations to the construction of complex neural network architectures. The repository distinguishes itself by bridging the gap between theoretical concepts and hands-on implementation. It covers the development of generative applications, such as image synthesis and style transfer, while offering guidance on opti

DeepEP is a distributed model accelerator and expert-parallel communication library designed to optimize the training and inference of large-scale neural networks. It provides specialized GPU communication kernels and a remote GPU memory interface to facilitate high-throughput data exchange between hardware nodes. The system utilizes dynamic kernel generation to compile optimized GPU kernels during execution, removing the need for separate installation compilation steps. It implements virtual-lane traffic isolation to prevent interference between different data streams and employs routing met

jetson-inference is a set of libraries and tools for executing optimized deep learning models on embedded GPU hardware. Its primary purpose is to enable real-time computer vision and AI inference at the edge with low latency and high throughput. The project distinguishes itself through high-performance streaming analytics and the ability to execute concurrent AI pipelines on auto-grade silicon. It provides specialized support for multi-sensor stream processing, utilizing zero-copy data transport to load camera frames directly into GPU memory. The codebase covers a broad surface of capabiliti

DeepGEMM is a suite of specialized GPU kernels and a just-in-time compiler designed for low-precision matrix operations, Mixture-of-Experts models, and attention processing. It provides a library of high-performance matrix multiplication kernels using FP8 precision to increase compute throughput and reduce memory usage. The project features a JIT CUDA kernel compiler that generates and loads optimized compute kernels at runtime to eliminate the need for manual compilation during installation. It includes specialized implementations for grouped matrix multiplication that process multiple group

FlashInfer is a library of high-performance GPU kernels purpose-built for accelerating large language model inference. It provides optimized implementations for attention operations (including flash attention, page attention, multi-head latent attention, and cascade attention) using paged key-value caches, fused kernel composition, and just-in-time compilation. The library also includes specialized kernels for mixture-of-experts layers, block-scaled low-precision quantization (FP8, FP4), and distributed collective communication. What distinguishes FlashInfer is its fused all-reduce communicat

Flux.jl ist ein Deep-Learning-Framework und ein Toolkit für numerisches Rechnen, das in Julia geschrieben wurde. Es dient als Machine-Learning-Bibliothek für das Entwerfen und Trainieren neuronaler Netze und bietet ein System zur automatischen Differenzierung zur Optimierung von Modellparametern. Das Framework ermöglicht Deep-Learning-Entwicklung und Machine-Learning-Forschung, indem es Schichten als parametrisierte Funktionen darstellt. Es unterstützt wissenschaftliches Machine Learning und integriert neuronale Netze in Workflows zur Lösung physikalischer und mathematischer Probleme. Das Toolkit bietet native GPU-Beschleunigung für Tensor-Berechnungen und nutzt Reverse-Mode-automatische Differenzierung zur Berechnung von Gradienten. Es verwendet Just-in-Time-kompilierte Kernels, um optimierten Maschinencode für mathematische Operationen auf CPU- und GPU-Hardware zu generieren.

This project is a high-performance C++ and CUDA neural network library designed for fast training and inference of small networks on NVIDIA GPUs. It serves as a specialized backend for neural radiance fields and coordinate-based networks, providing a fused GPU kernel library and a hash grid encoder for transforming raw input dimensions into high-dimensional representations. The library distinguishes itself through the use of C++ template metaprogramming and fused-kernel execution, which merge neural network layers into single GPU device functions to eliminate memory bottlenecks. It leverages

oneDNN is a library for deep learning acceleration that provides optimized building blocks for neural network training and inference. It manages tensor computation across CPU and GPU hardware, enabling the execution of high-performance primitives for model training and neural network inference optimization. The project distinguishes itself through hardware-specific kernel optimization and the use of just-in-time compilation to target specific processor instruction sets. It supports quantized neural network execution using both static and dynamic quantization to reduce memory usage and increas

Neon is a deep learning framework and hardware-abstraction machine learning stack used for designing, training, and deploying neural network architectures. It functions as a graph-based computation engine that utilizes just-in-time kernel compilation to optimize machine code for tensors. The platform decouples model definitions from execution kernels, allowing it to support multiple CPU and GPU backends. This architecture enables the distribution of computational workloads across parallelized hardware environments to increase processing speed and overall efficiency. The system covers the ful

This project is a comprehensive collection of technical manuals, tutorials, and guides for implementing machine learning models and numerical computations using the TensorFlow framework. It serves as an educational resource and technical library designed to help developers build and maintain models across diverse hardware environments. The repository includes a multilingual technical guide and a collaborative translation project focused on standardizing industry terminology. These efforts ensure that complex machine learning concepts and technical documentation are accessible and accurately i

Dieses Projekt ist eine Bildungsressource, die aus einem strukturierten Lehrplan interaktiver Notebooks besteht, die Deep-Learning-Konzepte und neuronale Netzwerkarchitekturen vermitteln sollen. Es konzentriert sich auf die Bereitstellung praktischer Erfahrungen mit dem TensorFlow 2-Framework und der Keras-API und führt Benutzer durch praktische Übungen zur Beherrschung von Machine-Learning-Techniken. Das Repository zeichnet sich durch die Kombination von instruktiven Inhalten mit den technischen Anforderungen für High-Performance-Computing aus. Es enthält spezifische Anleitungen zur Konfiguration lokaler Entwicklungsumgebungen zur Unterstützung hardwarebeschleunigten Trainings auf Grafikprozessoren, um sicherzustellen, dass Lernende intensive mathematische Berechnungen für ihre Modelle durchführen können. Die Plattform unterstützt eine Reihe von Entwicklungs-Workflows, vom Rapid Prototyping neuronaler Netzwerkarchitekturen bis zur Ausführung von Data-Science-Experimenten. Sie bietet Utilities zum Rendern mathematischer Gleichungen und Code innerhalb von Dokumenten und unterstützt sowohl lokale Offline-Entwicklung als auch Cloud-basierte Ausführungsumgebungen.

Segment Anything Fast is a high-performance computer vision inference engine and image segmentation framework built for PyTorch. It provides a specialized environment for automated object isolation and mask generation, designed to process large-scale visual datasets with increased throughput. The project distinguishes itself through a suite of system-level optimization strategies that accelerate deep learning model performance. By utilizing graph-based model compilation, just-in-time kernel fusion, and hardware-aware quantization, it reduces computational latency and memory footprint. These t