ashleve/lightning-hydra-template

Features

• PyTorch Lightning Workflows - A structured codebase for deep learning experiments combining PyTorch Lightning for training and Hydra for configuration management.
• Project Bootstrapping Templates - Ships a standardized project boilerplate with predefined directory structures and configurations for machine learning experiments.
• Hierarchical Configuration Overrides - Implements a hierarchical system for managing settings via YAML files and overrides to ensure reproducible runs.
• Checkpoint-Based Recovery - Provides mechanisms to serialize and restore model training state from checkpoints to resume experiments.
• Distributed Training Configurations - Offers a specialized configuration for executing deep learning workloads across multiple hardware accelerators using parallel strategies.
• Distributed Training Orchestration - Coordinates training workloads across multiple accelerators using data parallel processing and mixed precision.
• Experiment Tracking - Implements integrated experiment tracking to log hyperparameters and visualize performance metrics across training runs.
• Distributed Training - Provides built-in support for scaling deep learning model training across multiple compute nodes and hardware accelerators.
• Experiment Boilerplates - Provides a standardized project layout for organizing data pipelines, model architectures, and training loops.
• Hyperparameter Sweep Orchestrators - Automates the search across large hyperparameter spaces using optimization tools to improve model performance.
• Training Checkpointers - Saves training state to disk and allows resuming execution from a checkpoint to continue training.
• Data Pipelines - Bundles data downloading, splitting, and transformation logic into reusable modules for consistent dataset handling.
• Hydra - Integrates the Hydra framework for hierarchical management of hyperparameters and experiment settings via YAML and CLI.
• Model Encapsulations - Separates model architecture, training logic, and data pipelines into modular classes for reusable workflows.
• ML Experiment Logging - Connects training runs to external monitoring platforms for recording hyperparameters and visualizing metrics.
• Hyperparameter Configurations - Tracks sets of hyperparameters through dedicated configurations to maintain a history of optimal settings.
• Distributed Mixed-Precision Training - Facilitates distributed training using data parallel processing and mixed precision to accelerate model convergence.
• Training and Evaluation Pipelines - Provides a pipeline to toggle between training and testing phases using configuration flags.
• Hyperparameter Optimization - Provides automated hyperparameter optimization and parameter sweeps to improve model performance without source code changes.
• CLI Configuration Overrides - Allows modification of configuration values and experiment presets directly through terminal command-line flags.
• Command Line Interfaces - Maps functions to terminal commands to trigger training and evaluation workflows directly from the shell.
• Experiment Batch Execution - Run multiple experiment configurations or random seeds in a single command by iterating through a specified folder.
• Dynamic Object Instantiation - Creates model and data objects at runtime by mapping configuration paths to Python classes.
• Multi-Backend Metric Exporters - Routes training performance metrics to multiple external tracking frameworks through a unified interface.
• Model Evaluation - Includes utilities for evaluating saved model checkpoints against specific datasets to validate performance.

Star history