awesome-repositories.com
Blog
MCP
awesome-repositories.com

Descubre los mejores repositorios open-source con nuestra búsqueda potenciada por IA.

ExplorarBúsquedas curadasAlternativas open-sourceSoftware autohospedableBlogMapa del sitio
ProyectoServidor MCPAcerca deCómo clasificamosPrensa
Aviso legalPrivacidadTérminos
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
·

1 repositorio

Awesome GitHub RepositoriesBlock-Scaled Quantization Kernels

GPU kernels that encode weights and activations in block-scaled low-precision formats such as FP4 and FP8, reducing memory bandwidth while preserving accuracy.

Distinct from Block-Wise Quantization: Distinct from Block-Wise Quantization: focuses on GPU kernel implementations for block-scaled formats (FP4, FP8) rather than general quantization strategies for optimizer states.

Explore 1 awesome GitHub repository matching artificial intelligence & ml · Block-Scaled Quantization Kernels. Refine with filters or upvote what's useful.

Awesome Block-Scaled Quantization Kernels GitHub Repositories

Encuentra los mejores repositorios con IA.Buscaremos los repositorios que mejor coincidan usando IA.
  • flashinfer-ai/flashinferAvatar de flashinfer-ai

    flashinfer-ai/flashinfer

    4,996Ver en GitHub↗

    FlashInfer is a library of high-performance GPU kernels purpose-built for accelerating large language model inference. It provides optimized implementations for attention operations (including flash attention, page attention, multi-head latent attention, and cascade attention) using paged key-value caches, fused kernel composition, and just-in-time compilation. The library also includes specialized kernels for mixture-of-experts layers, block-scaled low-precision quantization (FP8, FP4), and distributed collective communication. What distinguishes FlashInfer is its fused all-reduce communicat

    Provides GPU-accelerated block-scaled quantization kernels for FP4 and FP8 formats to reduce memory bandwidth in LLM inference.

    Pythonattentioncudadistributed-inference
    Ver en GitHub↗4,996
  1. Home
  2. Artificial Intelligence & ML
  3. Quantized Inference Runtimes
  4. Weight Quantization
  5. Quantized Training
  6. Block-Wise Quantization
  7. Block-Scaled Quantization Kernels

Explorar subetiquetas

  • FP8 Block-Scaled MoE KernelsGPU kernels that execute mixture-of-experts layers using 8-bit floating point block-scaled quantization. **Distinct from Block-Scaled Quantization Kernels:** Distinct from Block-Scaled Quantization Kernels: applies block-scaled quantization specifically to mixture-of-experts layer computations.
  • Layout Conversion UtilitiesTensor reshaping and transposition operations that convert dense tensors into block layouts required by block-scaled MoE kernels. **Distinct from Block-Scaled Quantization Kernels:** Distinct from quantization kernels: provides tensor reshaping and transposition for block-scaled formats, not the quantization itself.
  • MXFP4 Scale Layout ConversionsRearranges MXFP4 block scales from the quantization output layout into the format required by SM90 mixed-input GEMM kernels. **Distinct from Block-Scaled Quantization Kernels:** Distinct from Block-Scaled Quantization Kernels: focuses specifically on converting MXFP4 block scales for mixed-input GEMM, not general FP8/FP4 quantization kernels.
  • Per-Token Scaled MoE KernelsGPU kernels that multiply groups of rows with corresponding matrices using FP8 per-token scaling for MoE routing. **Distinct from FP8 Block-Scaled MoE Kernels:** Uses per-token scaling instead of block-scaling, distinct from block-scaled MoE kernels.