1 repository
Awesome GitHub RepositoriesWorker-Specific Serialization
Mechanisms to restrict serialization operations to a single worker to avoid concurrent write corruption.
Distinct from Model Checkpointing: Focuses on the coordination of who writes the checkpoint, rather than the general process of saving weights.
Explore 1 awesome GitHub repository matching artificial intelligence & ml · Worker-Specific Serialization. Refine with filters or upvote what's useful.
Awesome Worker-Specific Serialization GitHub Repositories
Găsește cele mai bune repo-uri cu AI.Vom căuta cele mai potrivite repository-uri folosind AI.
- horovod/horovod
horovod/horovod
14,686Vezi pe GitHubHorovod is a distributed deep learning framework and gradient synchronizer designed to scale model training across multiple GPUs and compute nodes. It functions as a distributed training orchestrator and an elastic training engine, utilizing an MPI collective communication library to synchronize weights and gradients across TensorFlow, PyTorch, Keras, and MXNet models. The system distinguishes itself through dynamic elastic scaling, which allows it to adjust the number of active workers at runtime and recover from node failures. It optimizes communication efficiency using tensor fusion batchi
Prevents filesystem corruption by restricting model checkpoint saving to a single designated worker.
Python
