awesome-repositories.com
博客
MCP
awesome-repositories.com

通过 AI 驱动的搜索，发现最优秀的开源仓库。

探索精选搜索开源替代品自托管软件博客网站地图
项目MCP 服务器关于排名机制媒体报道
法律隐私政策服务条款
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
·

1 个仓库

Awesome GitHub RepositoriesBlock-Scaled Quantization Kernels

GPU kernels that encode weights and activations in block-scaled low-precision formats such as FP4 and FP8, reducing memory bandwidth while preserving accuracy.

Distinct from Block-Wise Quantization: Distinct from Block-Wise Quantization: focuses on GPU kernel implementations for block-scaled formats (FP4, FP8) rather than general quantization strategies for optimizer states.

Explore 1 awesome GitHub repository matching artificial intelligence & ml · Block-Scaled Quantization Kernels. Refine with filters or upvote what's useful.

Awesome Block-Scaled Quantization Kernels GitHub Repositories

用 AI 发现最棒的仓库。我们将通过 AI 为您搜索最匹配的仓库。
  • flashinfer-ai/flashinferflashinfer-ai 的头像

    flashinfer-ai/flashinfer

    4,996在 GitHub 上查看↗

    FlashInfer is a library of high-performance GPU kernels purpose-built for accelerating large language model inference. It provides optimized implementations for attention operations (including flash attention, page attention, multi-head latent attention, and cascade attention) using paged key-value caches, fused kernel composition, and just-in-time compilation. The library also includes specialized kernels for mixture-of-experts layers, block-scaled low-precision quantization (FP8, FP4), and distributed collective communication. What distinguishes FlashInfer is its fused all-reduce communicat

    Provides GPU-accelerated block-scaled quantization kernels for FP4 and FP8 formats to reduce memory bandwidth in LLM inference.

    Pythonattentioncudadistributed-inference
    在 GitHub 上查看↗4,996
  1. Home
  2. Artificial Intelligence & ML
  3. Quantized Inference Runtimes
  4. Weight Quantization
  5. Quantized Training
  6. Block-Wise Quantization
  7. Block-Scaled Quantization Kernels

探索子标签

  • FP8 Block-Scaled MoE KernelsGPU kernels that execute mixture-of-experts layers using 8-bit floating point block-scaled quantization. **Distinct from Block-Scaled Quantization Kernels:** Distinct from Block-Scaled Quantization Kernels: applies block-scaled quantization specifically to mixture-of-experts layer computations.
  • Layout Conversion UtilitiesTensor reshaping and transposition operations that convert dense tensors into block layouts required by block-scaled MoE kernels. **Distinct from Block-Scaled Quantization Kernels:** Distinct from quantization kernels: provides tensor reshaping and transposition for block-scaled formats, not the quantization itself.
  • MXFP4 Scale Layout ConversionsRearranges MXFP4 block scales from the quantization output layout into the format required by SM90 mixed-input GEMM kernels. **Distinct from Block-Scaled Quantization Kernels:** Distinct from Block-Scaled Quantization Kernels: focuses specifically on converting MXFP4 block scales for mixed-input GEMM, not general FP8/FP4 quantization kernels.
  • Per-Token Scaled MoE KernelsGPU kernels that multiply groups of rows with corresponding matrices using FP8 per-token scaling for MoE routing. **Distinct from FP8 Block-Scaled MoE Kernels:** Uses per-token scaling instead of block-scaling, distinct from block-scaled MoE kernels.