1 repository
Awesome GitHub RepositoriesMathematical Optimization Kernels
Specialized computational kernels designed to accelerate mathematical operations and reduce memory overhead in large-scale models.
Distinguishing note: Focuses on low-level kernel optimization for specific model architectures rather than general-purpose mathematical libraries.
Explore 1 awesome GitHub repository matching scientific & mathematical computing · Mathematical Optimization Kernels. Refine with filters or upvote what's useful.
Awesome Mathematical Optimization Kernels GitHub Repositories
Găsește cele mai bune repo-uri cu AI.Vom căuta cele mai potrivite repository-uri folosind AI.
- deepspeedai/deepspeed
deepspeedai/DeepSpeed
42,528Vezi pe GitHubDeepSpeed is a high-performance library designed to scale deep learning model training and inference across massive clusters of GPUs and compute nodes. It provides a comprehensive suite of tools for distributed training, enabling the execution of models that exceed the memory capacity of single devices through advanced parameter partitioning, pipeline-based model parallelism, and memory-efficient state offloading. The framework distinguishes itself through specialized communication-efficient optimizers and hardware-aware acceleration techniques. By utilizing gradient compression, quantization
The framework reduces memory usage and increases training speed for structural biology models using specialized kernels designed for large-scale sequence computations.
Pythonbillion-parameterscompressiondata-parallelism
