1 रिपॉजिटरी
Awesome GitHub RepositoriesSparsemax Computation Kernels
GPU kernels that compute sparse probability distributions as a drop-in replacement for softmax in attention or classification.
Distinct from Sparse Softmax Kernels: Distinct from Sparse Softmax Kernels: implements sparsemax specifically (produces exactly sparse outputs), not softmax with sparsity constraints.
Explore 1 awesome GitHub repository matching artificial intelligence & ml · Sparsemax Computation Kernels. Refine with filters or upvote what's useful.
Awesome Sparsemax Computation Kernels GitHub Repositories
AI के साथ बेहतरीन रिपॉजिटरी खोजें।हम AI का उपयोग करके सबसे सटीक रिपॉजिटरी खोजेंगे।
- linkedin/liger-kernel
linkedin/Liger-Kernel
6,148GitHub पर देखेंLiger-Kernel is a collection of pre-built fused Triton kernels and patching utilities designed to accelerate large language model training. It provides drop-in kernel replacements for common LLM operations such as RMSNorm, cross-entropy loss, and attention, enabling increased throughput and reduced memory usage while preserving bitwise-exact gradients. The project serves as a toolkit for composing custom model architectures from individual optimized kernels and for patching pre-existing models with minimal code changes. The project distinguishes itself through its ability to perform runtime m
Provides a sparsemax kernel that produces sparse probability distributions for attention and classification.
Pythonfinetuninggemma2hacktoberfest
