AnswerDotAI/gpu.cpp
Gpu.cpp
gpu.cpp este o bibliotecă C++ ușoară pentru executarea calculului GPU general-purpose de nivel scăzut pe diferite furnizori de hardware și sisteme de operare. Acesta funcționează ca un wrapper GPU portabil, orchestrator de kernel și sistem de gestionare a tensorilor folosind specificația WebGPU pentru a abstractiza inițializarea dispozitivului, transferurile de buffer și dispatching-ul compute shader-elor.
Biblioteca oferă un framework pentru definirea compute kernel-urilor din codul shader și gestionarea dispatch-ului și sincronizării lor asincrone. Permite executarea compute shader-elor multi-platformă și orchestrarea sarcinilor GPU printr-o specificație standardizată a procesorului grafic.
Sistemul gestionează întregul ciclu de viață al memoriei GPU, incluzând alocarea tensorilor multidimensionali, mișcarea bidirecțională a datelor între gazdă și dispozitiv prin staging buffers și urmărirea resurselor pentru a preveni scurgerile de memorie. De asemenea, suportă slicing-ul tensorilor pentru crearea de vizualizări non-owning ale segmentelor de memorie și include utilitare pentru logarea mesajelor de sistem și filtrarea severității.
Features
- WebGPU Implementations - Provides a C++ implementation and abstraction layer based on the WebGPU specification for cross-platform GPU compute.
- GPU Hardware Abstraction Layers - Acts as a portable interface abstracting device initialization and shader dispatch across diverse operating systems and hardware.
- Compute Shader Programming - Enables the execution of general-purpose compute kernels across different hardware vendors and operating systems.
- Tensor Lifecycle Management - Provides a system for allocating and managing the lifetime of multi-dimensional memory buffers on graphics hardware.
- GPU Compute Frameworks - Offers a portable framework for executing parallel mathematical operations across various GPU hardware vendors.
- GPU Resource Management - Manages the allocation and lifecycle of GPU tensors and buffers for high-speed computation.
- Compute Shader Dispatchers - Implements mechanisms for executing general-purpose parallel processing tasks on graphics processors across platforms.
- Tensor Memory Lifetime Management - Tracks and releases hardware tensor buffers to manage the lifetime of GPU assets and prevent memory leaks.
- GPU Device Initializations - Implements the processes for requesting hardware adapters and creating logical devices to enable GPU access.
- GPU Memory Orchestration - Orchestrates the transfer of data and synchronization between host memory and GPU memory.
- GPU Staging Buffers - Coordinates bidirectional data movement between the system processor and graphics processor using intermediate staging buffers.
- GPU Kernel Orchestration - Manages the full lifecycle of defining, launching, and synchronizing parallel compute kernels on GPU hardware.
- GPU Kernel Programming - Provides low-level primitives for defining and dispatching compute kernels with custom shader code.
- Compute Shader Kernels - Defines reusable compute units by binding shader code and workgroup metadata to hardware handles.
- Kernel Definitions - Creates reusable GPU kernels from shader code by binding input and output tensors and parameters.
- GPU Command Queues - Provides a non-blocking command queue to dispatch compute kernels for asynchronous GPU execution.
- Asynchronous Kernel Launchers - Decouples task scheduling from hardware execution by submitting kernels to the GPU queue asynchronously.
- Tensor Memory Management - Specifies the rank and size of dimensions to determine the structural memory layout of tensors.
- Manual Memory Management - Allows for the explicit disposal of tensor resources to manually update the resource pool and free memory.
- Device-to-Host Data Transfers - Coordinates staging buffers and callbacks to move information from hardware memory back to system memory.
- Host-to-Device Data Transfers - Moves memory from the system processor to the graphics processor to provide inputs for compute kernels.
- Non-Owning Tensor Views - Creates non-owning views of hardware buffers using offsets and shapes to avoid duplicating GPU memory.
- Tensor Slicing - Creates non-owning views of tensors using offsets to reference specific data segments without duplicating memory.
- GPU Operation Synchronizers - Ensures data consistency by waiting for asynchronous graphics processing operations to complete.
- Shader Metadata Management - Stores compute shader code and metadata such as workgroup size and entry points to prepare for execution.
- GPU Bind Group Mappings - Implements mechanisms for mapping tensors and parameters to WebGPU bind group indices for kernel access.
- GPU Buffer Management - Allocates and maintains contiguous blocks of hardware memory for mathematical computation storage.
- Resource Bindings - Maps buffers and tensors to specific indices so a GPU kernel can access them during execution.
- GPU Memory Allocators - Creates multi-dimensional memory buffers on the graphics processor for high-speed mathematical calculations.
- Handle Management - Provides a mechanism to store and reuse compute kernel handles to avoid expensive hardware setup costs.
- Asynchronous GPU Tasking - Coordinates the dispatch of GPU kernels using asynchronous primitives to manage non-blocking execution.
