AnswerDotAI/gpu.cpp
Gpu.cpp
gpu.cpp is a lightweight C++ library for executing low-level general-purpose GPU computation across different hardware vendors and operating systems. It functions as a portable GPU wrapper, kernel orchestrator, and tensor management system using the WebGPU specification to abstract device initialization, buffer transfers, and compute shader dispatching.
The library provides a framework for defining compute kernels from shader code and managing their asynchronous dispatch and synchronization. It enables the execution of cross-platform compute shaders and the orchestration of GPU tasks through a standardized graphics processor specification.
The system handles the full lifecycle of GPU memory, including the allocation of multi-dimensional tensors, bidirectional data movement between host and device via staging buffers, and resource tracking to prevent memory leaks. It also supports tensor slicing for creating non-owning views of memory segments and includes utilities for system message logging and severity filtering.
Features
- WebGPU Implementations - Provides a C++ implementation and abstraction layer based on the WebGPU specification for cross-platform GPU compute.
- GPU Hardware Abstraction Layers - Acts as a portable interface abstracting device initialization and shader dispatch across diverse operating systems and hardware.
- Compute Shader Programming - Enables the execution of general-purpose compute kernels across different hardware vendors and operating systems.
- Tensor Lifecycle Management - Provides a system for allocating and managing the lifetime of multi-dimensional memory buffers on graphics hardware.
- GPU Compute Frameworks - Offers a portable framework for executing parallel mathematical operations across various GPU hardware vendors.
- GPU Resource Management - Manages the allocation and lifecycle of GPU tensors and buffers for high-speed computation.
- Compute Shader Dispatchers - Implements mechanisms for executing general-purpose parallel processing tasks on graphics processors across platforms.
- Tensor Memory Lifetime Management - Tracks and releases hardware tensor buffers to manage the lifetime of GPU assets and prevent memory leaks.
- GPU Device Initializations - Implements the processes for requesting hardware adapters and creating logical devices to enable GPU access.
- GPU Memory Orchestration - Orchestrates the transfer of data and synchronization between host memory and GPU memory.
- GPU Staging Buffers - Coordinates bidirectional data movement between the system processor and graphics processor using intermediate staging buffers.
- GPU Kernel Orchestration - Manages the full lifecycle of defining, launching, and synchronizing parallel compute kernels on GPU hardware.
- GPU Kernel Programming - Provides low-level primitives for defining and dispatching compute kernels with custom shader code.
- Compute Shader Kernels - Defines reusable compute units by binding shader code and workgroup metadata to hardware handles.
- Kernel Definitions - Creates reusable GPU kernels from shader code by binding input and output tensors and parameters.
- GPU Command Queues - Provides a non-blocking command queue to dispatch compute kernels for asynchronous GPU execution.
- Asynchronous Kernel Launchers - Decouples task scheduling from hardware execution by submitting kernels to the GPU queue asynchronously.
- Tensor Memory Management - Specifies the rank and size of dimensions to determine the structural memory layout of tensors.
- Manual Memory Management - Allows for the explicit disposal of tensor resources to manually update the resource pool and free memory.
- Device-to-Host Data Transfers - Coordinates staging buffers and callbacks to move information from hardware memory back to system memory.
- Host-to-Device Data Transfers - Moves memory from the system processor to the graphics processor to provide inputs for compute kernels.
- Non-Owning Tensor Views - Creates non-owning views of hardware buffers using offsets and shapes to avoid duplicating GPU memory.
- Tensor Slicing - Creates non-owning views of tensors using offsets to reference specific data segments without duplicating memory.
- GPU Operation Synchronizers - Ensures data consistency by waiting for asynchronous graphics processing operations to complete.
- Shader Metadata Management - Stores compute shader code and metadata such as workgroup size and entry points to prepare for execution.
- GPU Bind Group Mappings - Implements mechanisms for mapping tensors and parameters to WebGPU bind group indices for kernel access.
- GPU Buffer Management - Allocates and maintains contiguous blocks of hardware memory for mathematical computation storage.
- Resource Bindings - Maps buffers and tensors to specific indices so a GPU kernel can access them during execution.
- GPU Memory Allocators - Creates multi-dimensional memory buffers on the graphics processor for high-speed mathematical calculations.
- Handle Management - Provides a mechanism to store and reuse compute kernel handles to avoid expensive hardware setup costs.
- Asynchronous GPU Tasking - Coordinates the dispatch of GPU kernels using asynchronous primitives to manage non-blocking execution.
