# answerdotai/gpu.cpp

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/answerdotai-gpu-cpp).**

3,981 stars · 195 forks · C++ · Apache-2.0

## Links

- GitHub: https://github.com/AnswerDotAI/gpu.cpp
- Homepage: https://gpucpp.answer.ai
- awesome-repositories: https://awesome-repositories.com/repository/answerdotai-gpu-cpp.md

## Description

gpu.cpp is a lightweight C++ library for executing low-level general-purpose GPU computation across different hardware vendors and operating systems. It functions as a portable GPU wrapper, kernel orchestrator, and tensor management system using the WebGPU specification to abstract device initialization, buffer transfers, and compute shader dispatching.

The library provides a framework for defining compute kernels from shader code and managing their asynchronous dispatch and synchronization. It enables the execution of cross-platform compute shaders and the orchestration of GPU tasks through a standardized graphics processor specification.

The system handles the full lifecycle of GPU memory, including the allocation of multi-dimensional tensors, bidirectional data movement between host and device via staging buffers, and resource tracking to prevent memory leaks. It also supports tensor slicing for creating non-owning views of memory segments and includes utilities for system message logging and severity filtering.

## Tags

### Graphics & Multimedia

- [WebGPU Implementations](https://awesome-repositories.com/f/graphics-multimedia/webgpu-implementations.md) — Provides a C++ implementation and abstraction layer based on the WebGPU specification for cross-platform GPU compute.
- [Compute Shader Programming](https://awesome-repositories.com/f/graphics-multimedia/compute-shader-programming.md) — Enables the execution of general-purpose compute kernels across different hardware vendors and operating systems.
- [Tensor Lifecycle Management](https://awesome-repositories.com/f/graphics-multimedia/gpu-buffer-management/tensor-lifecycle-management.md) — Provides a system for allocating and managing the lifetime of multi-dimensional memory buffers on graphics hardware.
- [GPU Compute Frameworks](https://awesome-repositories.com/f/graphics-multimedia/gpu-compute-frameworks.md) — Offers a portable framework for executing parallel mathematical operations across various GPU hardware vendors.
- [GPU Resource Management](https://awesome-repositories.com/f/graphics-multimedia/gpu-resource-management.md) — Manages the allocation and lifecycle of GPU tensors and buffers for high-speed computation.
- [Compute Shader Dispatchers](https://awesome-repositories.com/f/graphics-multimedia/graphics-engines-rendering/shaders/compute-shader-dispatchers.md) — Implements mechanisms for executing general-purpose parallel processing tasks on graphics processors across platforms. ([source](https://cdn.jsdelivr.net/gh/answerdotai/gpu.cpp@main/README.md))
- [Tensor Memory Lifetime Management](https://awesome-repositories.com/f/graphics-multimedia/graphics-resource-lifetime-management/tensor-memory-lifetime-management.md) — Tracks and releases hardware tensor buffers to manage the lifetime of GPU assets and prevent memory leaks.
- [Shader Metadata Management](https://awesome-repositories.com/f/graphics-multimedia/compute-shader-programming/shader-metadata-management.md) — Stores compute shader code and metadata such as workgroup size and entry points to prepare for execution. ([source](https://gpucpp.answer.ai/api/structgpu_1_1_kernel_code.html))
- [GPU Bind Group Mappings](https://awesome-repositories.com/f/graphics-multimedia/gpu-bind-group-mappings.md) — Implements mechanisms for mapping tensors and parameters to WebGPU bind group indices for kernel access.
- [GPU Buffer Management](https://awesome-repositories.com/f/graphics-multimedia/gpu-buffer-management.md) — Allocates and maintains contiguous blocks of hardware memory for mathematical computation storage. ([source](https://gpucpp.answer.ai/api/structgpu_1_1_array.html))

### Operating Systems & Systems Programming

- [GPU Hardware Abstraction Layers](https://awesome-repositories.com/f/operating-systems-systems-programming/gpu-hardware-abstraction-layers.md) — Acts as a portable interface abstracting device initialization and shader dispatch across diverse operating systems and hardware.
- [GPU Device Initializations](https://awesome-repositories.com/f/operating-systems-systems-programming/gpu-device-initializations.md) — Implements the processes for requesting hardware adapters and creating logical devices to enable GPU access. ([source](https://gpucpp.answer.ai/api/namespacegpu.html))
- [GPU Memory Orchestration](https://awesome-repositories.com/f/operating-systems-systems-programming/gpu-memory-orchestration.md) — Orchestrates the transfer of data and synchronization between host memory and GPU memory.
- [GPU Staging Buffers](https://awesome-repositories.com/f/operating-systems-systems-programming/gpu-staging-buffers.md) — Coordinates bidirectional data movement between the system processor and graphics processor using intermediate staging buffers.
- [Resource Bindings](https://awesome-repositories.com/f/operating-systems-systems-programming/hardware-interfacing-drivers/hardware-acceleration/gpu-acceleration/gpu-accelerated-compilers/pure-function-kernels/kernel-to-template-bindings/resource-bindings.md) — Maps buffers and tensors to specific indices so a GPU kernel can access them during execution. ([source](https://gpucpp.answer.ai/api/structgpu_1_1_bindings.html))
- [GPU Memory Allocators](https://awesome-repositories.com/f/operating-systems-systems-programming/kernel-core-internals/process-and-memory-management/memory-management/allocation-strategies/dynamic-memory-allocation/gpu-memory-allocators.md) — Creates multi-dimensional memory buffers on the graphics processor for high-speed mathematical calculations. ([source](https://gpucpp.answer.ai/api/structgpu_1_1_tensor.html))

### Programming Languages & Runtimes

- [GPU Kernel Orchestration](https://awesome-repositories.com/f/programming-languages-runtimes/compiler-interpreter-internals/compiler-infrastructure/jit-kernel-compilers/gpu-kernel-orchestration.md) — Manages the full lifecycle of defining, launching, and synchronizing parallel compute kernels on GPU hardware.
- [GPU Kernel Programming](https://awesome-repositories.com/f/programming-languages-runtimes/gpu-kernel-programming.md) — Provides low-level primitives for defining and dispatching compute kernels with custom shader code.

### Scientific & Mathematical Computing

- [Compute Shader Kernels](https://awesome-repositories.com/f/scientific-mathematical-computing/gpu-accelerated-computation/compute-shader-kernels.md) — Defines reusable compute units by binding shader code and workgroup metadata to hardware handles.
- [Kernel Definitions](https://awesome-repositories.com/f/scientific-mathematical-computing/gpu-accelerated-computation/compute-shader-kernels/kernel-definitions.md) — Creates reusable GPU kernels from shader code by binding input and output tensors and parameters. ([source](https://gpucpp.answer.ai/api/namespacegpu.html))
- [Handle Management](https://awesome-repositories.com/f/scientific-mathematical-computing/gpu-accelerated-computation/compute-shader-kernels/handle-management.md) — Provides a mechanism to store and reuse compute kernel handles to avoid expensive hardware setup costs. ([source](https://gpucpp.answer.ai/api/structgpu_1_1_kernel.html))

### Software Engineering & Architecture

- [GPU Command Queues](https://awesome-repositories.com/f/software-engineering-architecture/background-thread-dispatchers/dispatch-queues/gpu-command-queues.md) — Provides a non-blocking command queue to dispatch compute kernels for asynchronous GPU execution.
- [Asynchronous GPU Tasking](https://awesome-repositories.com/f/software-engineering-architecture/asynchronous-task-processors/asynchronous-gpu-tasking.md) — Coordinates the dispatch of GPU kernels using asynchronous primitives to manage non-blocking execution. ([source](https://cdn.jsdelivr.net/gh/answerdotai/gpu.cpp@main/README.md))

### Artificial Intelligence & ML

- [Asynchronous Kernel Launchers](https://awesome-repositories.com/f/artificial-intelligence-ml/kernel-schedulers/asynchronous-kernel-launchers.md) — Decouples task scheduling from hardware execution by submitting kernels to the GPU queue asynchronously. ([source](https://gpucpp.answer.ai/api/namespacegpu.html))
- [Tensor Memory Management](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/infrastructure/model-optimization-and-inference/hardware-and-acceleration/tensor-computing-libraries/tensor-memory-management.md) — Specifies the rank and size of dimensions to determine the structural memory layout of tensors. ([source](https://gpucpp.answer.ai/api/structgpu_1_1_shape.html))
- [Manual Memory Management](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/infrastructure/model-optimization-and-inference/hardware-and-acceleration/tensor-computing-libraries/tensor-operations/manual-memory-management.md) — Allows for the explicit disposal of tensor resources to manually update the resource pool and free memory. ([source](https://gpucpp.answer.ai/api/namespacegpu.html))

### Data & Databases

- [Device-to-Host Data Transfers](https://awesome-repositories.com/f/data-databases/data-processing-pipelines/data-processing/ml-data-pipelines/training-data-pipelines/pytorch-streaming-integrations/cuda-stream-synchronizations/device-to-host-data-transfers.md) — Coordinates staging buffers and callbacks to move information from hardware memory back to system memory. ([source](https://gpucpp.answer.ai/api/structgpu_1_1_copy_data.html))
- [Host-to-Device Data Transfers](https://awesome-repositories.com/f/data-databases/data-processing-pipelines/data-processing/ml-data-pipelines/training-data-pipelines/pytorch-streaming-integrations/cuda-stream-synchronizations/host-to-device-data-transfers.md) — Moves memory from the system processor to the graphics processor to provide inputs for compute kernels. ([source](https://cdn.jsdelivr.net/gh/answerdotai/gpu.cpp@main/README.md))
- [Non-Owning Tensor Views](https://awesome-repositories.com/f/data-databases/data-processing-pipelines/data-transformation/array-tensor-manipulation/tensor-transformations/tensor-cloning/non-owning-tensor-views.md) — Creates non-owning views of hardware buffers using offsets and shapes to avoid duplicating GPU memory.
- [Tensor Slicing](https://awesome-repositories.com/f/data-databases/data-processing-pipelines/data-transformation/array-tensor-manipulation/tensor-transformations/tensor-slicing.md) — Creates non-owning views of tensors using offsets to reference specific data segments without duplicating memory. ([source](https://gpucpp.answer.ai/api/structgpu_1_1_tensor_view.html))
- [GPU Operation Synchronizers](https://awesome-repositories.com/f/data-databases/data-synchronization/cross-device-synchronization-engines/cross-device-operation-execution/gpu-operation-synchronizers.md) — Ensures data consistency by waiting for asynchronous graphics processing operations to complete. ([source](https://gpucpp.answer.ai/api/namespacegpu.html))