NVIDIA/cutlass
Cutlass
Cutlass is a collection of C++ templates and Python interfaces for implementing high-performance linear algebra operations on NVIDIA GPUs. It provides a kernel composition framework for designing custom GPU kernels and a mixed-precision tensor library capable of executing operations across diverse data formats, ranging from 64-bit floating point to 4-bit integers.
The project features a toolkit for operator fusion that integrates activation functions and bias calculations directly into matrix multiplication kernels to reduce memory passes. It also includes a Python-based domain-specific language for defining high-performance GPU operations, which eliminates the need for C++ glue code.
The framework covers broader capabilities in GPU memory layout optimization, hierarchical tiling strategies, and the development of specialized CUDA kernels through modular software hierarchies.
Features
- GPU Kernel Implementations - Provides a framework for implementing custom-written hardware-level kernels for accelerated parallel computing on NVIDIA GPUs.
- CUDA-Accelerated Libraries - A CUDA-accelerated library of C++ templates and Python interfaces for high-performance matrix operations.
- Kernel Composition Frameworks - Provides a modular software hierarchy for composing specialized GPU kernels by tuning tiling sizes and data types.
- Compute Engines - Implements a mixed-precision tensor library supporting data formats from 64-bit floating point down to 4-bit integers.
- Mixed-Precision Compute Engines - Provides a unified interface for hardware-accelerated tensor cores supporting numerical formats from 64-bit float to 4-bit integers.
- Mixed-Precision Computing - Executes operations across various data types from 64-bit floating point to 4-bit formats to balance precision and speed.
- Matrix Operation Fusions - Integrates activation functions and bias calculations directly into matrix multiplication kernels to reduce memory passes.
- Gathered Matrix Multiplication - Implements the fusion of gathering and matrix multiplication into a single optimized operation to reduce memory overhead.
- Tensor Libraries - Provides a framework for creating and manipulating multidimensional arrays across diverse numerical precisions.
- Tensor Mappings - Maps multi-dimensional tensor data into linear memory to optimize direct hardware access and movement.
- Python GPU Kernels - Provides a domain-specific language to define high-performance GPU kernels directly in Python, eliminating the need for C++ glue code.
- Multidimensional Arrays - Defines multidimensional array structures to simplify indexing and data movement across GPU threads.
- GPU Linear Algebra Libraries - Implements high-performance matrix multiplication and tensor computations on NVIDIA hardware using modular templates.
- GPU Matrix Operation Implementations - Builds high-performance matrix multiplication and related GPU computations using modular templates and language interfaces.
- Memory Layout Optimizations - Optimizes the organization of multidimensional tensors to improve cache locality and reduce memory overhead.
- Template-Based Kernel Composition - Uses C++ templates to generate specialized GPU kernels by combining modular software components and hardware tuning parameters.
- Tiled Memory Access Patterns - Organizes data movement into structured blocks to maximize cache locality across the GPU memory hierarchy.
- Kernel Configuration DSLs - Provides a high-level Python DSL to define kernel configurations and layouts without requiring C++ glue code.
- AI & Machine Learning - CUDA templates for high-performance linear algebra
- Tensor Core Optimization - Template-based library for implementing high-performance tensor computations on NVIDIA GPUs.
