1 repository
Awesome GitHub RepositoriesShared Buffer Releases
Releases shared buffers created for multi-rank GPU communication, freeing local allocations while preserving IPC-mapped peer pointers.
Distinct from Multi-Rank Shared Buffer Allocations: Distinct from Multi-Rank Shared Buffer Allocations: handles cleanup and deallocation of shared buffers, not their creation.
Explore 1 awesome GitHub repository matching data & databases · Shared Buffer Releases. Refine with filters or upvote what's useful.
Awesome Shared Buffer Releases GitHub Repositories
Găsește cele mai bune repo-uri cu AI.Vom căuta cele mai potrivite repository-uri folosind AI.
- flashinfer-ai/flashinfer
flashinfer-ai/flashinfer
4,996Vezi pe GitHubFlashInfer is a library of high-performance GPU kernels purpose-built for accelerating large language model inference. It provides optimized implementations for attention operations (including flash attention, page attention, multi-head latent attention, and cascade attention) using paged key-value caches, fused kernel composition, and just-in-time compilation. The library also includes specialized kernels for mixture-of-experts layers, block-scaled low-precision quantization (FP8, FP4), and distributed collective communication. What distinguishes FlashInfer is its fused all-reduce communicat
This GPU kernel library releases a shared buffer created earlier, freeing rank-local allocations while avoiding freeing IPC-mapped peer pointers.
Pythonattentioncudadistributed-inference
