4 repository-uri
Awesome GitHub RepositoriesSparse Softmax Kernels
Computational kernels for applying softmax operations while maintaining sparsity constraints in neural networks.
Distinguishing note: Specifically targets the softmax operation within sparse contexts, distinct from general sparse matrix multiplication.
Explore 4 awesome GitHub repositories matching artificial intelligence & ml · Sparse Softmax Kernels. Refine with filters or upvote what's useful.
Awesome Sparse Softmax Kernels GitHub Repositories
Găsește cele mai bune repo-uri cu AI.Vom căuta cele mai potrivite repository-uri folosind AI.
- deepspeedai/deepspeed
deepspeedai/DeepSpeed
42,528Vezi pe GitHubDeepSpeed is a high-performance library designed to scale deep learning model training and inference across massive clusters of GPUs and compute nodes. It provides a comprehensive suite of tools for distributed training, enabling the execution of models that exceed the memory capacity of single devices through advanced parameter partitioning, pipeline-based model parallelism, and memory-efficient state offloading. The framework distinguishes itself through specialized communication-efficient optimizers and hardware-aware acceleration techniques. By utilizing gradient compression, quantization
The framework maintains sparsity constraints within attention mechanisms by applying block-sparse softmax operations during forward and backward passes.
Pythonbillion-parameterscompressiondata-parallelism - ml-explore/mlx
ml-explore/mlx
27,047Vezi pe GitHubThis project is a machine learning array framework and tensor computation library designed for high-performance numerical computing. It provides a comprehensive suite of tools for constructing and training neural networks, featuring an automatic differentiation engine that facilitates gradient-based optimization and complex mathematical modeling. The library distinguishes itself through a unified memory architecture that allows data to be shared across CPU and GPU devices without explicit copies, significantly reducing data movement overhead. Its execution model relies on a lazy evaluation en
The library multiplies two arrays while applying block-level masks to the input or output matrices to selectively ignore specific segments.
C++mlx - linkedin/liger-kernel
linkedin/Liger-Kernel
6,148Vezi pe GitHubLiger-Kernel is a collection of pre-built fused Triton kernels and patching utilities designed to accelerate large language model training. It provides drop-in kernel replacements for common LLM operations such as RMSNorm, cross-entropy loss, and attention, enabling increased throughput and reduced memory usage while preserving bitwise-exact gradients. The project serves as a toolkit for composing custom model architectures from individual optimized kernels and for patching pre-existing models with minimal code changes. The project distinguishes itself through its ability to perform runtime m
Provides a sparsemax kernel that produces sparse probability distributions for attention and classification.
Pythonfinetuninggemma2hacktoberfest - flashinfer-ai/flashinfer
flashinfer-ai/flashinfer
4,996Vezi pe GitHubFlashInfer is a library of high-performance GPU kernels purpose-built for accelerating large language model inference. It provides optimized implementations for attention operations (including flash attention, page attention, multi-head latent attention, and cascade attention) using paged key-value caches, fused kernel composition, and just-in-time compilation. The library also includes specialized kernels for mixture-of-experts layers, block-scaled low-precision quantization (FP8, FP4), and distributed collective communication. What distinguishes FlashInfer is its fused all-reduce communicat
Provides fused GPU kernels for softmax with temperature and top-K/P threshold masking.
Pythonattentioncudadistributed-inference
