1 repo
Awesome GitHub RepositoriesFully Sharded Data Parallelism
Memory-efficient training technique that shards model parameters, gradients, and optimizer states across data-parallel processes.
Explore 1 awesome GitHub repository matching artificial intelligence & ml · Fully Sharded Data Parallelism. Refine with filters or upvote what's useful.
Awesome Fully Sharded Data Parallelism GitHub Repositories
We'll search the best matching repositories with AI.
- pytorch/pytorch
PyTorch is a machine learning framework centered on a GPU-ready tensor library that supports multi-dimensional array operations across both CPU and accelerator hardware. It provides a foundational infrastructure for mathematical computation and dynamic neural network construction, utilizing a tape-based automatic diffe
Shards model parameters, gradients, and optimizer states across processes to enable memory-efficient distributed training.
Pythonautograddeep-learninggpu
