awesome-repositories.com
Blog
MCP
awesome-repositories.com

Discover the best open-source repositories with AI-powered search.

ExploreCurated searchesOpen-source alternativesSelf-hosted softwareBlogSitemap
ProjectMCP serverAboutHow we rankPress
LegalPrivacyTerms
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
deepseek-ai avatar

deepseek-ai/DeepGEMM

0
View on GitHub↗

DeepGEMM

DeepGEMM is a suite of specialized GPU kernels and a just-in-time compiler designed for low-precision matrix operations, Mixture-of-Experts models, and attention processing. It provides a library of high-performance matrix multiplication kernels using FP8 precision to increase compute throughput and reduce memory usage.

The project features a JIT CUDA kernel compiler that generates and loads optimized compute kernels at runtime to eliminate the need for manual compilation during installation. It includes specialized implementations for grouped matrix multiplication that process multiple grouped axes in a single kernel call and fused acceleration kernels for Mixture-of-Experts models that overlap communication with tensor core computation.

The compute engine also provides a logits processor for multi-query attention that functions across both paged and non-paged memory layouts. Its broader capabilities include dense and grouped matrix operations and the fusion of dispatch, linear layers, and activation functions into single kernels.

AI search

Explore more awesome repositories

Describe what you need in plain English — the AI ranks thousands of curated open-source projects by relevance.

Start searching with AI

Features

  • FP8 Matrix Multiplication - Provides a library of high-performance matrix multiplication kernels using FP8 precision for deep learning.
  • FP8 GEMM Libraries - Provides a suite of high-performance matrix multiplication kernels using FP8 precision.
  • Expert Communication Kernels - Provides fused compute kernels for Mixture-of-Experts models that overlap communication with tensor core computation.
  • FP8 Scaling - Implements fine-grained FP8 scaling to maintain high precision during low-precision matrix multiplications.
  • Just-In-Time Kernel Compilers - Generates and loads optimized compute kernels at runtime to eliminate the need for manual installation compilation.
  • Mixture-of-Experts Inference Optimizers - Optimizes Mixture-of-Experts models by fusing dispatch and linear layers to reduce inference latency.
  • MoE Kernel Fusions - Combines dispatch, linear layers, and activation functions into a single operation to overlap communication with tensor core compute.
  • Tensor Core Optimization - Optimizes matrix operations by mapping them directly to hardware tensor cores using FP8 formats.
  • Grouped GEMM - Provides optimized grouped matrix multiplication that processes multiple grouped axes in a single kernel call.
  • JIT Kernel Compilers - Features a runtime system that generates and loads optimized GPU compute kernels without manual compilation.
  • Kernel Fusion Operations - Fuses dispatch, linear layers, and activation functions into a single kernel to minimize memory overhead.
  • Generalized Matrix Multiplications - Performs high-performance dense matrix multiplication using low-precision formats across various memory layouts.
  • Multi-Query Attention Scoring - Implements a specialized scoring mechanism for multi-query attention that works across paged and non-paged memory.
  • Logit Processors - Includes a specialized compute kernel for calculating multi-query attention logits across different memory layouts.
  • Attention Backends - Accelerates multi-query attention by calculating token logits to speed up indexing and inference.
  • Attention Optimization - Optimized matrix multiplication kernels for deep learning.
  • Inference Kernels - High-throughput FP8 matrix multiplication library for dense and MoE models.
7,385 stars·1,051 forks·Cuda·MIT·29 views

Star history

Star history chart for deepseek-ai/deepgemmStar history chart for deepseek-ai/deepgemm

How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.

Frequently asked questions

What does deepseek-ai/deepgemm do?

DeepGEMM is a suite of specialized GPU kernels and a just-in-time compiler designed for low-precision matrix operations, Mixture-of-Experts models, and attention processing. It provides a library of high-performance matrix multiplication kernels using FP8 precision to increase compute throughput and reduce memory usage.

What are the main features of deepseek-ai/deepgemm?

The main features of deepseek-ai/deepgemm are: FP8 Matrix Multiplication, FP8 GEMM Libraries, Expert Communication Kernels, FP8 Scaling, Just-In-Time Kernel Compilers, Mixture-of-Experts Inference Optimizers, MoE Kernel Fusions, Tensor Core Optimization.

Which projects share features with deepseek-ai/deepgemm?

Projects with overlapping indexed features include: nvidia/isaac-gr00t. deepseek-ai/deepep — DeepEP is a distributed model accelerator and expert-parallel communication library designed to optimize the training… sgl-project/sglang — Sglang is a high-performance inference engine and serving system designed for large language and multimodal models. It… dao-ailab/flash-attention — FlashAttention is an attention mechanism optimization library and machine learning acceleration framework designed to… dusty-nv/jetson-inference — jetson-inference is a set of libraries and tools for executing optimized deep learning models on embedded GPU… flashinfer-ai/flashinfer — FlashInfer is a library of high-performance GPU kernels purpose-built for accelerating large language model inference.…

Projects sharing features with DeepGEMM

These projects share indexed features with DeepGEMM. Shared tags can include platform or build tooling; verify the primary use case before treating a result as a replacement.
  • nvidia/isaac-gr00tNVIDIA avatar

    NVIDIA/Isaac-GR00T

    6,222View on GitHub↗
    Jupyter Notebook
    View on GitHub↗6,222
  • deepseek-ai/deepepdeepseek-ai avatar

    deepseek-ai/DeepEP

    9,736View on GitHub↗

    DeepEP is a distributed model accelerator and expert-parallel communication library designed to optimize the training and inference of large-scale neural networks. It provides specialized GPU communication kernels and a remote GPU memory interface to facilitate high-throughput data exchange between hardware nodes. The system utilizes dynamic kernel generation to compile optimized GPU kernels during execution, removing the need for separate installation compilation steps. It implements virtual-lane traffic isolation to prevent interference between different data streams and employs routing met

    Cuda
    View on GitHub↗9,736
  • sgl-project/sglangsgl-project avatar

    sgl-project/sglang

    29,079View on GitHub↗

    Sglang is a high-performance inference engine and serving system designed for large language and multimodal models. It provides a programmable interface for orchestrating complex generation workflows, enabling developers to coordinate multi-turn dialogues, tool invocations, and reasoning chains through a domain-specific language. The platform is built to support production-scale deployments, offering an OpenAI-compatible API that allows for integration with existing application ecosystems. The system distinguishes itself through a disaggregated architecture that separates compute-intensive pr

    Pythonattentionblackwellcuda
    View on GitHub↗29,079
  • dao-ailab/flash-attentionDao-AILab avatar

    Dao-AILab/flash-attention

    24,220View on GitHub↗

    FlashAttention is an attention mechanism optimization library and machine learning acceleration framework designed to increase training speed and reduce memory footprint for large-scale neural network models. It functions as a collection of low-level CUDA kernels that optimize memory-bound operations to improve hardware utilization on graphics processing units. The library distinguishes itself through an input-output-aware algorithm design that minimizes data movement between different levels of memory. By employing kernel fusion and tiled matrix multiplication, it combines sequential operati

    Python
    View on GitHub↗24,220
Compare all 30 related projects→