GPU acceleration libraries

JAX is a hardware-accelerated array library and automatic differentiation system for numerical computing. It provides a framework compatible with NumPy that extends array operations with a just-in-time compiler to transform Python functions into optimized kernels for execution on GPU and TPU accelerators. The system differentiates itself through the use of an XLA-based compiler and a single program multiple data sharding model. These capabilities allow the library to distribute large-scale computations across multiple hardware accelerators using both automatic parallelization and manual shard

cuda-python provides low-level Python bindings for the CUDA Driver and Runtime APIs. It serves as a programmatic wrapper for controlling device memory, managing hardware toolchains, and orchestrating execution graphs on NVIDIA GPUs, allowing for the compilation and launching of parallel kernels directly from Python. The project enables the development of SIMT kernels and the execution of mathematical algorithms on device memory. It integrates pre-compiled bytecode as custom operators and interfaces with accelerated device libraries to access low-level hardware functions without leaving the la

PyTorch is a machine learning framework centered on a GPU-ready tensor library that supports multi-dimensional array operations across both CPU and accelerator hardware. It provides a foundational infrastructure for mathematical computation and dynamic neural network construction, utilizing a tape-based automatic differentiation system that allows for flexible, non-static graph execution. The framework is designed for deep integration with Python, enabling natural usage alongside standard scientific computing ecosystems. It distinguishes itself through a comprehensive distributed training sui

Thrust is a heterogeneous computing library and C++ template library that provides a collection of high-level templates for executing data-parallel operations. It functions as a parallel algorithms library designed to work across different hardware backends, including multicore CPUs and NVIDIA GPU hardware. The framework utilizes a header-only implementation and a generic-programming policy interface to abstract the differences between CPU and GPU memory and execution models. It employs an iterator-based data abstraction to provide a uniform interface for accessing elements across host RAM an

CuPy is a CUDA array computing library that implements a NumPy-compatible interface for executing array operations and numerical computing on NVIDIA GPUs. It serves as a GPU-accelerated numerical library and a CUDA-based SciPy implementation, offloading heavy calculations to graphics hardware to increase processing speed for scientific and engineering workloads. The library enables multi-framework tensor exchange, allowing data buffers to be shared between different deep learning frameworks using standardized memory layouts to avoid memory copies. It also supports custom GPU kernel integratio

Taichi is a domain-specific programming language embedded in Python designed for high-performance numerical computing and computer graphics. It functions as a parallel compiler that translates high-level mathematical expressions into optimized machine instructions, enabling developers to write compute-intensive algorithms that execute across diverse hardware architectures, including CPUs, GPUs, and specialized accelerators. The project distinguishes itself through a hardware-agnostic execution layer that maps parallel operations to multiple backends such as CUDA, Metal, and Vulkan. By utilizi

Tinygrad is a deep learning framework and tensor computation engine designed for building and training neural networks. It functions as a hardware abstraction layer that manages device memory, command queues, and kernel dispatching across heterogeneous computing architectures. By utilizing a lazy-evaluation approach, the framework constructs computational graphs that defer execution until data is explicitly required, allowing it to process only the necessary operations for a given result. The project distinguishes itself through a just-in-time compilation layer that transforms abstract comput

This project is a high-performance numerical computing library designed for large-scale scientific and machine learning workloads. It functions as an automatic differentiation framework and a just-in-time compilation engine, transforming high-level Python code into optimized machine instructions. By enforcing pure functional programming patterns and immutable array semantics, the library ensures that mathematical functions remain compatible with automated graph transformations and symbolic differentiation. The platform distinguishes itself through its distributed array computing capabilities,

Bend is a high-level parallel programming language and compiler designed to execute code across multi-core CPUs and GPUs automatically. By translating functional source code into a graph-based intermediate representation, it enables massive parallel execution without requiring manual management of threads, locks, or atomic operations. The runtime operates as an interaction net engine, where computations are represented as networks of nodes that reduce through local rewriting rules. This model utilizes a work-stealing scheduler to distribute tasks across thousands of hardware threads, ensuring

Cutlass is a collection of C++ templates and Python interfaces for implementing high-performance linear algebra operations on NVIDIA GPUs. It provides a kernel composition framework for designing custom GPU kernels and a mixed-precision tensor library capable of executing operations across diverse data formats, ranging from 64-bit floating point to 4-bit integers. The project features a toolkit for operator fusion that integrates activation functions and bias calculations directly into matrix multiplication kernels to reduce memory passes. It also includes a Python-based domain-specific langu

Thrust is a C++ parallel algorithms library that provides a suite of standard-library-inspired interfaces for execution on multi-core and accelerator hardware. It serves as a CUDA-accelerated data library and a generic parallel programming interface designed to enable high-performance data processing across GPUs and CPUs. The project implements a portable abstraction layer that allows for heterogeneous computing workflows, enabling the same core algorithm logic to run on different hardware accelerators. This is achieved through a generic programming policy design and a backend-agnostic execut

CUDA.jl provides a programming interface for executing custom kernels and performing parallel array computing directly on NVIDIA graphics hardware using the Julia language. It serves as a framework for compiling and scheduling user-defined functions across multiple processing cores, enabling high-performance data processing and task synchronization. The library distinguishes itself through a custom compiler backend that translates high-level language functions into hardware-specific machine code. It manages complex hardware interactions through asynchronous stream scheduling, unified memory m

Accelerate is a framework for high-performance array computing that provides a domain-specific language for expressing complex mathematical and parallel computations. By utilizing a declarative programming interface, it allows users to define high-level array transformations that are automatically translated into optimized machine code for diverse hardware architectures. The system distinguishes itself through a modular architecture that decouples high-level array operations from hardware-specific instructions. It employs just-in-time compilation and kernel fusion to transform programs into e

Taco is a sparse tensor algebra compiler that translates high-level tensor index expressions into optimized machine code. It functions as a numerical code generator, producing specialized C kernels designed to execute complex multidimensional array operations efficiently on both CPU and GPU hardware. The project distinguishes itself by allowing users to define custom tensor storage layouts by composing dimension-level formats, such as dense or compressed structures, to match the specific sparsity patterns of their datasets. By analyzing the mathematical structure of tensor operations at compi

Numba is a just-in-time compiler that translates high-level Python functions into optimized machine code at runtime. By leveraging the LLVM compiler infrastructure, it provides a framework for accelerating numerical data processing and mathematical computations, enabling performance levels comparable to statically compiled languages. The project distinguishes itself through its ability to perform type-inference-based specialization, which generates machine instructions tailored to the specific data types used during execution. It employs a lazy compilation pipeline that defers translation unt

Taskflow is a C++ task-parallel framework designed to build high-performance parallel workflows and complex dependency graphs. It provides a programming model that organizes computational work into directed acyclic graphs, enabling developers to manage concurrency, resource scheduling, and task dependencies across multi-core CPUs and GPU accelerators. The framework distinguishes itself through its ability to orchestrate heterogeneous systems, allowing for the integration of hardware-accelerated kernels and memory operations into unified execution pipelines. It supports dynamic runtime subflow

This library is a JavaScript framework for general-purpose computing on graphics processing units. It enables the execution of parallel mathematical operations directly within the browser by offloading data-heavy calculations to graphics hardware. The project functions as a web-based math accelerator that converts standard JavaScript functions into shader code for execution on the graphics processor. It provides a unified interface that detects available graphics APIs and manages data transfer between system and graphics memory. To ensure compatibility across diverse environments, the library

gpu.cpp is a lightweight C++ library for executing low-level general-purpose GPU computation across different hardware vendors and operating systems. It functions as a portable GPU wrapper, kernel orchestrator, and tensor management system using the WebGPU specification to abstract device initialization, buffer transfers, and compute shader dispatching. The library provides a framework for defining compute kernels from shader code and managing their asynchronous dispatch and synchronization. It enables the execution of cross-platform compute shaders and the orchestration of GPU tasks through

DeepSpeed is a high-performance library designed to scale deep learning model training and inference across massive clusters of GPUs and compute nodes. It provides a comprehensive suite of tools for distributed training, enabling the execution of models that exceed the memory capacity of single devices through advanced parameter partitioning, pipeline-based model parallelism, and memory-efficient state offloading. The framework distinguishes itself through specialized communication-efficient optimizers and hardware-aware acceleration techniques. By utilizing gradient compression, quantization

IREE is an MLIR-based compiler toolchain and runtime designed to translate machine learning models from various frameworks into optimized binaries for execution across diverse hardware targets. It provides a unified pipeline to ingest models from PyTorch, TensorFlow, JAX, and ONNX, lowering them into a common intermediate representation for deployment on CPUs, GPUs, and bare-metal embedded systems. The project distinguishes itself through a bytecode virtual machine and a hardware abstraction layer that decouple high-level model logic from specific hardware instruction sets. It supports sophis