1 dépôt
Awesome GitHub RepositoriesDistributed Process Control
Mechanisms for controlling execution flow and restricting tasks to specific ranks within a distributed compute cluster.
Distinct from Process Execution Utilities: The candidates focus on OS process spawning or build-target execution, whereas this is about coordinating distributed ranks in a machine learning cluster.
Explore 1 awesome GitHub repository matching artificial intelligence & ml · Distributed Process Control. Refine with filters or upvote what's useful.
Awesome Distributed Process Control GitHub Repositories
Trouvez les meilleurs dépôts grâce à l'IA.Nous recherchons les dépôts les plus pertinents grâce à l'IA.
- huggingface/accelerate
huggingface/accelerate
9,725Voir sur GitHubAccelerate is a PyTorch distributed training library that abstracts the boilerplate required to run models across multiple GPUs, TPUs, and CPUs. It functions as a deep learning model scaler and distributed hardware orchestrator, allowing the same training script to run on different hardware backends without modifying the core logic. The project provides a distributed training command line interface for configuring compute environments and launching jobs across single or multi-node clusters. It includes a mixed precision training framework to implement FP16 and BF16 precision, reducing memory
The ability to execute a function on a designated process index to ensure a task runs only once within a cluster.
Python
