1 个仓库
Awesome GitHub RepositoriesResource Optimization Tools
Utilities that automatically adjust system and model parameters to improve memory and compute efficiency during training.
Distinguishing note: Focuses on automated system-level heuristics for resource management rather than general model training or architecture.
Explore 1 awesome GitHub repository matching artificial intelligence & ml · Resource Optimization Tools. Refine with filters or upvote what's useful.
Awesome Resource Optimization Tools GitHub Repositories
用 AI 发现最棒的仓库。我们将通过 AI 为您搜索最匹配的仓库。
- deepspeedai/deepspeed
deepspeedai/DeepSpeed
42,528在 GitHub 上查看DeepSpeed is a high-performance library designed to scale deep learning model training and inference across massive clusters of GPUs and compute nodes. It provides a comprehensive suite of tools for distributed training, enabling the execution of models that exceed the memory capacity of single devices through advanced parameter partitioning, pipeline-based model parallelism, and memory-efficient state offloading. The framework distinguishes itself through specialized communication-efficient optimizers and hardware-aware acceleration techniques. By utilizing gradient compression, quantization
The framework optimizes memory and compute efficiency by automatically tuning batch sizes and memory configurations based on model and system heuristics.
Pythonbillion-parameterscompressiondata-parallelism
