NVIDIA/cuda-python
Cuda Python
cuda-python provides low-level Python bindings for the CUDA Driver and Runtime APIs. It serves as a programmatic wrapper for controlling device memory, managing hardware toolchains, and orchestrating execution graphs on NVIDIA GPUs, allowing for the compilation and launching of parallel kernels directly from Python.
The project enables the development of SIMT kernels and the execution of mathematical algorithms on device memory. It integrates pre-compiled bytecode as custom operators and interfaces with accelerated device libraries to access low-level hardware functions without leaving the language environment.
The toolkit covers hardware management through host and runtime API integration, as well as device memory management for structured data layouts and temporary buffers. It includes capabilities for system hardware inspection, GPU execution debugging, and performance optimizations such as disk-backed kernel caching and cooperative reduction execution.
Features
- Python GPU Development - Enables developers to write and execute high-performance GPU-accelerated applications directly within Python.
- CUDA Driver Wrappers - Provides a comprehensive programmatic wrapper for controlling device memory and orchestrating execution graphs on NVIDIA GPUs.
- Hardware Driver API Mappings - Maps high-level Python function calls directly to the underlying CUDA driver and runtime APIs.
- CUDA Driver API Integrations - Provides a programmatic interface to CUDA driver and compiler tools for coordinating hardware processing and memory.
- Driver Runtime Integrations - Provides direct access to CUDA driver and runtime functions for managing hardware resources.
- Hardware Abstraction Layers - Provides an object-oriented hardware abstraction layer to manage GPU device properties and memory allocation.
- Device-Local Memory Layouts - Handles the allocation and control of hardware memory buffers and structured data types for efficient GPU processing.
- Device Buffer Managers - Provides a tool for allocating and controlling hardware memory buffers using standard array interfaces.
- JIT Kernel Compilers - Transforms high-level functions into machine code at runtime for parallel execution on NVIDIA GPUs.
- GPU Kernel Orchestration - Manages the compilation and launching of parallel kernels and execution graphs to optimize processing speed.
- Dynamic CUDA Runtime Invocations - Implements dynamic calls to CUDA runtime functions for device management and memory allocation.
- GPU Kernel Programming - Supports the development and compilation of SIMT kernels for high-performance workloads on NVIDIA hardware.
- Foreign Function Interfaces - Implements a foreign function interface to bind Python calls to low-level CUDA C headers and binary libraries.
- Python Bindings - Offers a set of low-level Python bindings for the CUDA Driver and Runtime APIs.
- GPU Kernel Performance Tuning - Optimizes GPU execution speed using techniques like cooperative reductions and bytecode caching.
- GPU Acceleration Libraries - Interfaces with accelerated device libraries to access low-level hardware functions from Python.
- GPU Buffer Sharing Allocations - Provides mechanisms to share GPU buffers across processes and external libraries using a standardized memory interface.
- GPU Memory Layout Definitions - Creates structured data types that handle complex memory layouts for arrays processed on graphics hardware.
- GPU Memory Pools - Implements device memory pools to reuse buffers and reduce allocation overhead and fragmentation.
- GPU-Accelerated Reductions - Uses custom operators to perform efficient data aggregation and reduction algorithms across GPU blocks and warps.
- Pre-compiled Bytecode Integration - Allows importing pre-compiled bytecode as custom operators to skip the standard JIT compilation process.
- Kernel Artifact Caches - Caches compiled GPU kernel bytecode on the local filesystem to accelerate subsequent program startup.
- Compiled Kernel Executions - Executes compiled GPU kernels on active streams and devices using custom launch configurations.
- Custom Kernel Compilers - Transforms custom Python functions into optimized machine code to increase execution speed on the GPU.
- Device-Side Algorithm Execution - Executes fast mathematical operations such as sorting and transforming directly on device memory.
- Execution Graphs - Orchestrates sequences of GPU operations as execution graphs to minimize CPU-to-GPU launch overhead.
- Graph Execution Compilers - Compiles and executes captured operation graphs into executable objects for optimized GPU processing.
