awesome-repositories.com
Blog
MCP
awesome-repositories.com

Discover the best open-source repositories with AI-powered search.

ExploreCurated searchesOpen-source alternativesSelf-hosted softwareBlogSitemap
ProjectMCP serverAboutHow we rankPress
LegalPrivacyTerms
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
deepseek-ai avatar

deepseek-ai/FlashMLA

0
View on GitHub↗

FlashMLA

FlashMLA is an LLM attention kernel library and inference acceleration library providing a collection of high-performance CUDA kernels. It implements multi-head latent attention mechanisms designed to reduce memory overhead and increase throughput during the forward and backward passes of large language model inference.

The library utilizes quantized cache attention kernels to improve computation efficiency across both sparse and dense token processing. It specifically optimizes the prefill and decoding phases of model inference through these latent attention implementations.

The project covers GPU kernel development for NVIDIA hardware, focusing on KV cache memory reduction and sparse attention computation. It includes capabilities for executing multi-head attention, dense attention, and sparse attention across different inference stages.

AI search

Explore more awesome repositories

Describe what you need in plain English — the AI ranks thousands of curated open-source projects by relevance.

Start searching with AI

Features

  • GPU-Accelerated Inference - Implements high-performance CUDA kernels specifically to accelerate the inference phase of LLMs on NVIDIA GPUs.
  • Latent Attention Mechanisms - Provides high-performance kernels for computing multi-head latent attention to increase inference throughput.
  • Attention Kernel Libraries - Provides a high-performance collection of CUDA kernels for multi-head latent attention to increase LLM throughput.
  • Latent Space Projections - Compresses attention keys and values into smaller latent representations to reduce memory bandwidth requirements.
  • GPU Kernel Implementations - Implements high-performance CUDA kernels specifically for accelerating attention computations on NVIDIA hardware.
  • Inference Optimization Kernels - Improves speed and memory efficiency of LLM decoding and prefill stages using specialized kernels.
  • Attention Kernel Fusion - Combines multiple attention operations into single GPU kernels to minimize memory data movement.
  • KV Cache Management - Lowers the memory footprint of long-context generation using quantized caches and sparse attention.
  • Cache Quantization - Stores latent attention states in low-precision formats to reduce memory usage during decoding.
  • Attention Mechanisms - Provides the capability to compute full attention across all tokens for complete contextual representation.
  • Triton Kernels - Uses the Triton domain-specific language to generate high-performance GPU code and optimize memory tiling.
  • Multi-Head Attention Mechanisms - Computes forward and backward attention operations for standard multi-head attention architectures.
  • Prefill-Decode Disaggregation - Implements distinct computational paths to optimize the transition between prefill and decoding phases.
  • Sparse Attention Kernels - Utilizes specialized kernels to compute attention only for a selective subset of tokens.
  • Quantized Cache Kernels - Ships attention kernels that use quantized caches to improve efficiency for both sparse and dense token processing.
  • AI & Machine Learning - Efficient MLA decoding kernels.
  • Attention Optimization - Hardware-aligned attention optimization for latent attention models.
  • Inference Kernels - Efficient decoding kernel for multi-head latent attention on Hopper GPUs.
12,706 stars·1,061 forks·C++·MIT·45 views

Star history

Star history chart for deepseek-ai/flashmlaStar history chart for deepseek-ai/flashmla

How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.

Projects sharing features with FlashMLA

These projects share indexed features with FlashMLA. Shared tags can include platform or build tooling; verify the primary use case before treating a result as a replacement.
  • sgl-project/sglangsgl-project avatar

    sgl-project/sglang

    29,079View on GitHub↗

    Sglang is a high-performance inference engine and serving system designed for large language and multimodal models. It provides a programmable interface for orchestrating complex generation workflows, enabling developers to coordinate multi-turn dialogues, tool invocations, and reasoning chains through a domain-specific language. The platform is built to support production-scale deployments, offering an OpenAI-compatible API that allows for integration with existing application ecosystems. The system distinguishes itself through a disaggregated architecture that separates compute-intensive pr

    Pythonattentionblackwellcuda
    View on GitHub↗29,079
  • zhaochenyang20/awesome-ml-sys-tutorialzhaochenyang20 avatar

    zhaochenyang20/Awesome-ML-SYS-Tutorial

    5,371View on GitHub↗

    This project provides a comprehensive technical guide and framework for engineering large-scale machine learning systems. It covers the full lifecycle of model development, focusing on the infrastructure and computational principles required to build, train, and serve generative AI models across distributed GPU clusters. The repository distinguishes itself by offering deep-dive tutorials and implementation strategies for complex system challenges. It emphasizes high-performance architectural primitives, such as collective communication orchestration, distributed tensor sharding, and static gr

    Python
    View on GitHub↗5,371
  • nvidia/isaac-gr00tNVIDIA avatar

    NVIDIA/Isaac-GR00T

    6,222View on GitHub↗
    Jupyter Notebook
    View on GitHub↗6,222
  • xlite-dev/leetcudaxlite-dev avatar

    xlite-dev/LeetCUDA

    9,694View on GitHub↗

    LeetCUDA is a collection of high-performance GPU kernel libraries focusing on memory optimization, activation functions, and attention mechanisms. It serves as a reference library for CUDA kernel implementations, ranging from basic element-wise operations to complex neural network components, and provides Python bindings to integrate these kernels into deep learning workflows. The project is distinguished by its focus on low-level hardware optimizations. This includes the use of tensor cores for half-precision matrix multiplication, asynchronous data pipelining with double buffering, and shar

    Cudacudacuda-12cuda-cpp
    View on GitHub↗9,694
Compare all 30 related projects→

Frequently asked questions

What does deepseek-ai/flashmla do?

FlashMLA is an LLM attention kernel library and inference acceleration library providing a collection of high-performance CUDA kernels. It implements multi-head latent attention mechanisms designed to reduce memory overhead and increase throughput during the forward and backward passes of large language model inference.

What are the main features of deepseek-ai/flashmla?

The main features of deepseek-ai/flashmla are: GPU-Accelerated Inference, Latent Attention Mechanisms, Attention Kernel Libraries, Latent Space Projections, GPU Kernel Implementations, Inference Optimization Kernels, Attention Kernel Fusion, KV Cache Management.

Which projects share features with deepseek-ai/flashmla?

Projects with overlapping indexed features include: sgl-project/sglang — Sglang is a high-performance inference engine and serving system designed for large language and multimodal models. It… zhaochenyang20/awesome-ml-sys-tutorial — This project provides a comprehensive technical guide and framework for engineering large-scale machine learning… nvidia/isaac-gr00t. xlite-dev/leetcuda — LeetCUDA is a collection of high-performance GPU kernel libraries focusing on memory optimization, activation… skyzh/tiny-llm — tiny-llm is a large language model inference engine and transformer model implementation. It serves as a quantized… microsoft/unilm — This project is a comprehensive framework and toolkit for developing, optimizing, and deploying transformer-based…