1 Repo
Awesome GitHub RepositoriesMulti-Rank Shared Buffer Allocations
Allocates a buffer on each GPU rank and maps IPC handles so every rank can access the buffer of any other rank.
Distinct from GPU Buffer Allocators: Distinct from GPU Buffer Allocators: focuses on allocating and sharing buffers across multiple GPU ranks via IPC, not general single-device buffer allocation.
Explore 1 awesome GitHub repository matching data & databases · Multi-Rank Shared Buffer Allocations. Refine with filters or upvote what's useful.
Awesome Multi-Rank Shared Buffer Allocations GitHub Repositories
Finde die besten Repos mit KI.Wir suchen mit KI nach den am besten passenden Repositories.
- flashinfer-ai/flashinfer
flashinfer-ai/flashinfer
4,996Auf GitHub ansehenFlashInfer is a library of high-performance GPU kernels purpose-built for accelerating large language model inference. It provides optimized implementations for attention operations (including flash attention, page attention, multi-head latent attention, and cascade attention) using paged key-value caches, fused kernel composition, and just-in-time compilation. The library also includes specialized kernels for mixture-of-experts layers, block-scaled low-precision quantization (FP8, FP4), and distributed collective communication. What distinguishes FlashInfer is its fused all-reduce communicat
This GPU kernel library allocates a buffer on each GPU rank and maps IPC handles so every rank can access the buffer of any other rank.
Pythonattentioncudadistributed-inference
