skypilot-org/skypilot
Skypilot
SkyPilot is a multi-cloud AI orchestrator and distributed task scheduler designed to launch and manage AI workloads across various cloud providers, Kubernetes, and Slurm clusters. It functions as an infrastructure-as-code framework that uses declarative files to define resource requirements and setup commands for consistent execution across different environments.
The project differentiates itself through automated cost optimization, selecting the most affordable GPU or TPU hardware and managing spot instances to reduce expenses. It also provides a remote development environment that bridges local IDEs to remote compute clusters via SSH and code synchronization.
The platform covers broad capability areas including cross-cloud resource provisioning, distributed training coordination, and multi-node task scaling. It incorporates workload orchestration for hyperparameter grid search and model deployment, while utilizing gang scheduling and binpacking to manage high-demand compute resources.
The system also includes utilities for external storage mounting and the use of preconfigured workload templates to accelerate infrastructure setup.
Features
- Multi-Cluster Orchestrators - Provides a single interface to deploy and manage workloads across Kubernetes, Slurm, and various cloud virtual machines.
- Multi-Cloud Orchestrators - Orchestrates AI workloads across multiple cloud providers, Kubernetes, and Slurm clusters using a unified interface.
- Distributed Training Managers - Runs large scale model training across multiple nodes and GPUs with automated resource provisioning and cluster management.
- Cost-Optimization Strategies - Automatically selects and manages preemptible hardware to reduce training expenses while handling node failover.
- Multi-Cloud Deployments - Executes AI tasks across various cloud providers, Kubernetes, or Slurm clusters through a single interface.
- Cloud Infrastructure Automation - Automates the selection of cost-effective infrastructure across different clouds with built-in failover support.
- GPU Provisioning Services - Locates and launches available GPU instances across multiple cloud providers and Kubernetes clusters.
- Cloud Provisioning - Abstracts cloud provider APIs to locate and launch compute instances based on cost and hardware availability.
- Configuration as Code - Defines resource requirements and setup commands in portable files to ensure consistent execution across environments.
- Declarative Infrastructure Definitions - Uses portable configuration files to specify resource requirements and setup commands for consistent execution.
- Distributed Task Schedulers - Coordinates multi-node GPU and TPU jobs using gang scheduling and automated resource provisioning.
- Infrastructure as Code Frameworks - Provides a declarative system for defining AI resource requirements and setup commands using portable configuration files.
- Instance Purchasing Optimization - Reduces training expenses by utilizing managed spot instances and selecting the most affordable available hardware.
- Distributed Cluster Provisioners - Automates the setup and orchestration of distributed compute clusters specifically for multi-node AI training.
- System Resource Provisioning - Provisions GPUs, TPUs, or CPUs based on real-time cost and availability with automatic failover.
- Distributed Training Coordination - Coordinates training tasks across multiple nodes and GPUs using distributed data parallel frameworks.
- AI Workload Orchestration - Orchestrates the serving of open source model architectures across cloud and on-premises infrastructure.
- Hyperparameter Optimization - Runs multiple concurrent trials on a cluster to optimize model hyperparameters or process large datasets.
- TPU Training Accelerators - Deploys and manages high-performance training jobs on specialized TPU virtual machines.
- Remote Development Environments - Connects IDEs and synchronizes code via SSH to simplify the development experience on remote compute clusters.
- Remote File Synchronization - Connects local development environments to remote compute nodes by synchronizing code and providing shell access.
- GPU Training Clusters - Builds containerized runtime environments with networking dependencies for GPU workloads on Kubernetes.
- Gang-Based Schedulers - Coordinates multi node jobs using gang scheduling and queueing to manage high demand workloads across clusters.
- Resource Utilization Optimization - Maximizes hardware efficiency by binpacking workloads and automatically cleaning up idle GPU resources.
- Distributed Task Orchestration - Distributes a single command or task across multiple hostnames within a compute cluster.
- Gang Schedulers - Coordinates multi-node workloads by ensuring all required resources are allocated before starting the execution process.
- Kubernetes Cluster Management - Orchestrates containerized machine learning tasks on Kubernetes clusters with automated GPU provisioning and resource scheduling.
- Kubernetes Orchestration - Uses pre-configured container images and runtimes to orchestrate AI workloads on Kubernetes clusters.
- Remote Development Environments - Connects local IDEs to remote compute clusters via SSH and synchronizes code for a seamless development experience.
- Binpacking Optimizers - Increases hardware efficiency by packing multiple workloads onto shared clusters and cleaning up idle resources.
- Inference and Serving - Unified framework to run ML workloads on any cloud.
- Inference Engines - Tool for running AI jobs on any cloud infrastructure.
- Model Serving & Deployment - Runs AI and batch jobs across any cloud provider.
