1 रिपॉजिटरी
Awesome GitHub RepositoriesSmall-Batch MoE Matrix Multiply Kernels
GPU kernels that multiply matrices for mixture-of-experts models optimized for small token batches.
Distinct from Batch Matrix Multiplication Utilities: Distinct from Batch Matrix Multiplication Utilities: optimized for small token batches in MoE routing, not general batch matmul.
Explore 1 awesome GitHub repository matching data & databases · Small-Batch MoE Matrix Multiply Kernels. Refine with filters or upvote what's useful.
Awesome Small-Batch MoE Matrix Multiply Kernels GitHub Repositories
AI के साथ बेहतरीन रिपॉजिटरी खोजें।हम AI का उपयोग करके सबसे सटीक रिपॉजिटरी खोजेंगे।
- flashinfer-ai/flashinfer
flashinfer-ai/flashinfer
4,996GitHub पर देखेंFlashInfer is a library of high-performance GPU kernels purpose-built for accelerating large language model inference. It provides optimized implementations for attention operations (including flash attention, page attention, multi-head latent attention, and cascade attention) using paged key-value caches, fused kernel composition, and just-in-time compilation. The library also includes specialized kernels for mixture-of-experts layers, block-scaled low-precision quantization (FP8, FP4), and distributed collective communication. What distinguishes FlashInfer is its fused all-reduce communicat
Multipies matrices optimized for small-batch expert routing in MoE models.
Pythonattentioncudadistributed-inference
