1 repository
Awesome GitHub RepositoriesQuantized Cache Kernels
Low-level GPU kernels that utilize quantized key-value caches to improve computation efficiency and reduce memory footprint.
Distinct from Sparse Attention Kernels: Combines kernel-level attention computation with low-precision cache management, distinct from purely sparse indexing.
Explore 1 awesome GitHub repository matching artificial intelligence & ml · Quantized Cache Kernels. Refine with filters or upvote what's useful.
Awesome Quantized Cache Kernels GitHub Repositories
Găsește cele mai bune repo-uri cu AI.Vom căuta cele mai potrivite repository-uri folosind AI.
- deepseek-ai/flashmla
deepseek-ai/FlashMLA
12,706Vezi pe GitHubFlashMLA is an LLM attention kernel library and inference acceleration library providing a collection of high-performance CUDA kernels. It implements multi-head latent attention mechanisms designed to reduce memory overhead and increase throughput during the forward and backward passes of large language model inference. The library utilizes quantized cache attention kernels to improve computation efficiency across both sparse and dense token processing. It specifically optimizes the prefill and decoding phases of model inference through these latent attention implementations. The project cov
Ships attention kernels that use quantized caches to improve efficiency for both sparse and dense token processing.
C++
