Awesome GitHub RepositoriesHardware Buffer Zero-Copy

NCCL is a high-performance communication library and distributed GPU computing framework designed for executing collective and point-to-point data exchanges across multiple GPUs in single or multi-node systems. It serves as an RDMA GPU transport layer and memory orchestrator, facilitating high-bandwidth synchronization of data and model gradients for distributed GPU training and inference. The library is distinguished by its ability to execute communication primitives directly from GPU kernels, removing the host CPU from the critical path. It utilizes topology-aware path selection to optimize

QtAV is a cross-platform media engine and multimedia framework that combines FFmpeg decoding with the Qt framework for audio and video rendering. It functions as a hardware-accelerated video player, an OpenGL video renderer, and a multimedia stream transcoder. The project distinguishes itself through a hardware-abstraction decoding layer that utilizes GPU interfaces such as VA-API and VideoToolbox to decode high-resolution video. It employs a zero-copy memory transfer path to move decoded video data directly to graphics APIs, reducing CPU overhead and enabling high-performance YUV rendering.

LiteRT is a runtime and API for executing machine learning and generative AI models on mobile, desktop, and IoT hardware. It consists of an inference engine and a specialized environment for running quantized large language and diffusion models locally on edge hardware. The system includes an ahead-of-time model compiler that translates models into hardware-specific bytecode to reduce startup latency and memory overhead. It provides a unified interface for Neural Processing Units with automatic fallback routing to CPUs or GPUs when specific subgraph support is unavailable. An edge model conve