volcano-sh/volcano
Volcano
Volcano is a Kubernetes-native batch scheduler specialized for AI, machine learning, and high-performance computing workloads. It provides gang scheduling to atomically allocate resources for all tasks of a distributed job, preventing deadlocks from partial allocation, and supports hierarchical queue management for multi-tenant resource isolation with configurable quotas, borrowing, and preemption. Topology-aware placement optimizes communication-intensive workloads by modeling network hierarchy to minimize cross-switch latency.
Volcano differentiates itself with automated orchestration of distributed training workloads, including peer discovery and network setup for frameworks such as PyTorch, Horovod, MPI, and Ray. It handles communication rank assignment, init container injection, and gang scheduling for multi-node training jobs. For inference, it supports deployment scaling, request flow control, and model-aware routing with pluggable strategies. The scheduler also integrates with GPU virtualization, MIG partitioning, and heterogeneous hardware including NPUs and CPUs.
Beyond AI, Volcano manages general batch jobs through cron-based periodic scheduling, DAG workflows, and automatic cleanup of completed jobs. It provides cross-cluster resource pooling and fair scheduling, dynamic load rebalancing, and workload colocation with QoS classification. Resource management spans multi-dimensional quotas, exclusive and shared GPU allocation, and dynamic resource allocation for heterogeneous devices.
Volcano is installed and configured via standard Kubernetes package managers, uses custom resource definitions for job submission, and exposes scheduling metrics and monitoring dashboards for observability.
Features
- Distributed Communication Optimizers - Optimizes placement of communication-intensive tasks to minimize data transfer overhead based on interconnect bandwidth.
- Network and Peer Discovery Setups - Automates network configuration and peer discovery for distributed training frameworks without manual setup.
- Distributed Training Orchestration - Automates environment variable injection and init container creation for PyTorch distributed training.
- Containerized Batch Job Schedulers - Volcano schedules containerized batch jobs and integrates with distributed computing frameworks through custom job resources.
