1 dépôt
Awesome GitHub RepositoriesResidual LayerNorm with Scale-Shift
Fused GPU kernels that combine residual addition, layer normalization, and learned scale or shift into a single operation for efficient self-attention.
Distinct from Fused GPU Kernel Composition: Distinct from Fused GPU Kernel Composition: this kernel specifically fuses residual addition with layer normalization and scale-shift, not general kernel composition.
Explore 1 awesome GitHub repository matching artificial intelligence & ml · Residual LayerNorm with Scale-Shift. Refine with filters or upvote what's useful.
Awesome Residual LayerNorm with Scale-Shift GitHub Repositories
Trouvez les meilleurs dépôts grâce à l'IA.Nous recherchons les dépôts les plus pertinents grâce à l'IA.
- flashinfer-ai/flashinfer
flashinfer-ai/flashinfer
4,996Voir sur GitHubFlashInfer is a library of high-performance GPU kernels purpose-built for accelerating large language model inference. It provides optimized implementations for attention operations (including flash attention, page attention, multi-head latent attention, and cascade attention) using paged key-value caches, fused kernel composition, and just-in-time compilation. The library also includes specialized kernels for mixture-of-experts layers, block-scaled low-precision quantization (FP8, FP4), and distributed collective communication. What distinguishes FlashInfer is its fused all-reduce communicat
Provides fused GPU kernels that combine residual addition, layer normalization, and learned scale or shift into a single operation.
Pythonattentioncudadistributed-inference
