Awesome GitHub RepositoriesRemote GPU Memory Access

AISystem is a comprehensive AI full-stack infrastructure project covering the entire pipeline from AI chip architecture to high-level training frameworks. It encompasses the development of AI compiler frameworks, inference engines, and distributed training orchestrators designed to coordinate workloads across a heterogeneous compute stack of CPUs, GPUs, and NPUs. The project focuses on the deep integration of software and hardware, employing software-hardware co-design to align tensor layouts with physical memory structures. It provides specialized capabilities for accelerating Transformer mo

DeepEP is a distributed model accelerator and expert-parallel communication library designed to optimize the training and inference of large-scale neural networks. It provides specialized GPU communication kernels and a remote GPU memory interface to facilitate high-throughput data exchange between hardware nodes. The system utilizes dynamic kernel generation to compile optimized GPU kernels during execution, removing the need for separate installation compilation steps. It implements virtual-lane traffic isolation to prevent interference between different data streams and employs routing met

jetson-inference is a set of libraries and tools for executing optimized deep learning models on embedded GPU hardware. Its primary purpose is to enable real-time computer vision and AI inference at the edge with low latency and high throughput. The project distinguishes itself through high-performance streaming analytics and the ability to execute concurrent AI pipelines on auto-grade silicon. It provides specialized support for multi-sensor stream processing, utilizing zero-copy data transport to load camera frames directly into GPU memory. The codebase covers a broad surface of capabiliti

Warp is a Python framework that JIT-compiles Python functions into CUDA kernels for GPU-accelerated parallel computation, with built-in automatic differentiation and multi-framework array interoperability. At its core, it provides a GPU kernel compilation system that enables writing and executing custom GPU kernels directly from Python, while supporting automatic gradient computation through those kernels for integration with machine learning pipelines. The framework also includes tile-based cooperative computing, where thread blocks partition into tiles for shared-memory and tensor-core opera

NCCL 是一个高性能通信库和分布式 GPU 计算框架，专为在单节点或多节点系统中的多个 GPU 之间执行集合和点对点数据交换而设计。它充当 RDMA GPU 传输层和内存编排器，为分布式 GPU 训练和推理提供高带宽的数据和模型梯度同步。 该库的特色在于能够直接从 GPU 内核执行通信原语，将主机 CPU 从关键路径中移除。它利用拓扑感知路径选择来优化数据移动，并采用包括 InfiniBand 和 NVLink 在内的基于 RDMA 的网络传输，以实现设备跨不同物理节点之间的零拷贝内存访问。 该项目涵盖了广泛的集合通信模式，包括归约（Reductions）、广播（Broadcasts）、收集（Gathers）和全对全交换（All-to-all exchanges），以及点对点远程内存访问。它提供全面的通信器管理，用于初始化、分区和调整 GPU 组大小，以及用于注册缓冲区和协调共享设备内存的专用内存管理。 该系统包括一套用于健康跟踪、诊断日志记录和实时事件监控的监控与可观测性工具，以及用于机器学习框架、CUDA Graphs、MPI 和 Python 的集成接口。