1 个仓库
Awesome GitHub RepositoriesWorker-Specific Serialization
Mechanisms to restrict serialization operations to a single worker to avoid concurrent write corruption.
Distinct from Model Checkpointing: Focuses on the coordination of who writes the checkpoint, rather than the general process of saving weights.
Explore 1 awesome GitHub repository matching artificial intelligence & ml · Worker-Specific Serialization. Refine with filters or upvote what's useful.
Awesome Worker-Specific Serialization GitHub Repositories
用 AI 发现最棒的仓库。我们将通过 AI 为您搜索最匹配的仓库。
- horovod/horovod
horovod/horovod
14,686在 GitHub 上查看Horovod is a distributed deep learning framework and gradient synchronizer designed to scale model training across multiple GPUs and compute nodes. It functions as a distributed training orchestrator and an elastic training engine, utilizing an MPI collective communication library to synchronize weights and gradients across TensorFlow, PyTorch, Keras, and MXNet models. The system distinguishes itself through dynamic elastic scaling, which allows it to adjust the number of active workers at runtime and recover from node failures. It optimizes communication efficiency using tensor fusion batchi
Prevents filesystem corruption by restricting model checkpoint saving to a single designated worker.
Python
