Cutlass

Cutlass is a collection of C++ templates and Python interfaces for implementing high-performance linear algebra operations on NVIDIA GPUs. It provides a kernel composition framework for designing custom GPU kernels and a mixed-precision tensor library capable of executing operations across diverse data formats, ranging from 64-bit floating point to 4-bit integers.

The project features a toolkit for operator fusion that integrates activation functions and bias calculations directly into matrix multiplication kernels to reduce memory passes. It also includes a Python-based domain-specific language for defining high-performance GPU operations, which eliminates the need for C++ glue code.

The framework covers broader capabilities in GPU memory layout optimization, hierarchical tiling strategies, and the development of specialized CUDA kernels through modular software hierarchies.

Features

GPU Kernel Implementations - Provides a framework for implementing custom-written hardware-level kernels for accelerated parallel computing on NVIDIA GPUs.

CUDA-Accelerated Libraries - A CUDA-accelerated library of C++ templates and Python interfaces for high-performance matrix operations.

Kernel Composition Frameworks - Provides a modular software hierarchy for composing specialized GPU kernels by tuning tiling sizes and data types.

Compute Engines - Implements a mixed-precision tensor library supporting data formats from 64-bit floating point down to 4-bit integers.

Mixed-Precision Compute Engines - Provides a unified interface for hardware-accelerated tensor cores supporting numerical formats from 64-bit float to 4-bit integers.

Mixed-Precision Computing - Executes operations across various data types from 64-bit floating point to 4-bit formats to balance precision and speed.

Matrix Operation Fusions - Integrates activation functions and bias calculations directly into matrix multiplication kernels to reduce memory passes.

Gathered Matrix Multiplication - Implements the fusion of gathering and matrix multiplication into a single optimized operation to reduce memory overhead.

Tensor Libraries - Provides a framework for creating and manipulating multidimensional arrays across diverse numerical precisions.

Tensor Mappings - Maps multi-dimensional tensor data into linear memory to optimize direct hardware access and movement.

Python GPU Kernels - Provides a domain-specific language to define high-performance GPU kernels directly in Python, eliminating the need for C++ glue code.

Multidimensional Arrays - Defines multidimensional array structures to simplify indexing and data movement across GPU threads.

GPU Linear Algebra Libraries - Implements high-performance matrix multiplication and tensor computations on NVIDIA hardware using modular templates.

GPU Matrix Operation Implementations - Builds high-performance matrix multiplication and related GPU computations using modular templates and language interfaces.

Memory Layout Optimizations - Optimizes the organization of multidimensional tensors to improve cache locality and reduce memory overhead.

Template-Based Kernel Composition - Uses C++ templates to generate specialized GPU kernels by combining modular software components and hardware tuning parameters.

Tiled Memory Access Patterns - Organizes data movement into structured blocks to maximize cache locality across the GPU memory hierarchy.

Kernel Configuration DSLs - Provides a high-level Python DSL to define kernel configurations and layouts without requiring C++ glue code.

AI & Machine Learning - CUDA templates for high-performance linear algebra

Tensor Core Optimization - Template-based library for implementing high-performance tensor computations on NVIDIA GPUs.

NVIDIAcutlass

Features

Star history