optuna/optuna

Features

• Hyperparameter Optimization - Provides a Python library for automating the search for optimal machine learning model parameters using dynamic search spaces.
• Bayesian Optimization Libraries - Employs advanced Bayesian sampling algorithms to navigate complex parameter landscapes and identify the most effective model settings.
• Automated Machine Learning Tools - Streamlines model development by iteratively testing parameter combinations and refining configurations based on performance metrics.
• Search Pruning - Provides automated early termination of underperforming trials to save computational resources during hyperparameter search.
• Optimization Visualizers - Generates interactive charts to visualize optimization results and parameter relationships.
• Distributed Training Optimizers - Scales hyperparameter tuning across multiple processes or machines to accelerate the discovery of high-performing model architectures.
• Dynamic Search Space Construction - Allows building complex, conditional search spaces imperatively using standard programming logic during execution.
• Early Stopping Monitors - Stops underperforming training runs early using automated pruning algorithms to save computational resources.
• Parameter Sampling - Suggests categorical, integer, or floating point values for model parameters, supporting logarithmic scaling and discretization.
• Search Space Definition - Specifies ranges and types for model parameters using standard programming constructs to build complex optimization configurations.
• Training Callbacks - Provides hooks into training loops to monitor progress and trigger early termination of underperforming trials.
• Hyperparameter - Identifies influential variables to focus future optimization efforts.
• Distributed State Persistence - Maintains study results and trial history across distributed environments and sessions.
• Optimization History Stores - Stores study and trial data in relational databases or files to maintain state.
• Optimization State Persistence - Stores trial history and study metadata in external databases to enable distributed execution and cross-session persistence.
• Trial Pruning - Interrupts trial execution when intermediate results indicate poor performance, allowing the system to reclaim resources.
• Hyperparameter Search Algorithms - Selects optimal model parameters by applying various sampling algorithms that narrow the search space based on previous trial results.
• Optimization Extenders - Provides a pluggable architecture for implementing custom samplers, pruners, and training callbacks.
• Optimization Trials - Distributes parameter search trials across multiple processes or machines to accelerate the discovery of optimal configurations.
• Hyperparameter Suggestions - Provides an interface within an objective function to request parameter values for evaluation during optimization.
• Hyperparameter Study Management - Organizes and manages collections of trials to track the progress and history of hyperparameter search tasks.
• Training Integrations - Connects to external model training workflows to systematically test different configurations.
• Multi-Objective Tuning - Enables finding optimal parameter sets that balance multiple competing performance metrics simultaneously during the training process.
• Sampling Strategies - Applies various algorithms to determine the next set of hyperparameters to test, including Bayesian and evolutionary methods.
• Automated Machine Learning - Hyperparameter optimization framework with define-by-run API.
• General Machine Learning - Software framework for automatic hyperparameter optimization.
• Hyperparameter Tuning - Framework-agnostic library for automatic hyperparameter optimization.
• Machine Learning - Hyperparameter optimization framework.
• Machine Learning Libraries - Hyperparameter optimization framework.
• MLOps and Infrastructure - Hyperparameter optimization framework.
• Optimization - Hyperparameter optimization framework for machine learning.
• Automated Machine Learning - Automatic hyperparameter optimization framework for machine learning.
• Relational Database Persistence - Persists optimization studies using relational databases to ensure continuity.
• Automated Parameter Searches - Runs automated grid or random searches for optimal model settings while maintaining compatibility with training workflows.
• Optimization Sessions - Tracks and organizes multiple execution trials within a study object to retrieve results and performance metrics.
• Machine Learning Pipelines - Integrates systematic parameter tuning into machine learning workflows to automate the identification of optimal model architectures.
• Distributed Locks - Coordinates distributed trials using locking mechanisms to ensure data consistency.
• Distributed Optimization Synchronization - Coordinates multiple parallel optimization processes through shared storage backends and locking mechanisms.
• Optimization Objectives - Finds optimal parameter sets that balance multiple competing performance metrics simultaneously during model training.
• Pluggable Samplers - Decouples parameter suggestion logic from the execution engine to allow for interchangeable search strategies.
• Parameter Importance Assessors - Displays the relative impact of hyperparameters on performance metrics.
• Optimization Benchmarkers - Constructs standardized tasks to measure optimization algorithm performance.
• Distributed Training - Supports scaling hyperparameter search across multiple processes and networked machines to accelerate model configuration discovery.
• Optimization Lifecycle Management - Tracks performance metrics and controls the tuning process by connecting directly to training workflows.
• Optimization Objective Configuration - Defines the direction of a search task and sets constraints such as maximum trial counts to control parameter tuning.
• Optimization Logic Overrides - Allows implementation of user-defined sampling strategies and pruning rules to tailor the search process.
• Worker Scaling - Distributes optimization tasks across multiple parallel processes or machines to accelerate parameter search.
• Optimization Termination - Stops the search process when estimated potential improvement falls below statistical error thresholds.
• State Management - Tracks the execution status and lifecycle of evaluation processes to ensure proper control of the optimization flow.
• Training Hooks - Integrates with training loops via event-driven hooks to monitor progress and inject control signals.
• Automated Sampler Selection - Chooses the most appropriate optimization algorithm based on the search space and objective requirements.
• Optimization Plot Customizers - Returns editable figure objects for custom visualization layouts.
• Search Space Compatibility - Compares hyperparameter search spaces to determine compatibility for warm-starting or resuming optimization trials.
• Trial Artifact Persisters - Uploads trial output files to storage backends for record-keeping.
• Objective Function Execution - Runs an objective function repeatedly with varying input parameters to identify the values that minimize or maximize the function output.
• Search Configurations - Converts hyperparameter distribution definitions into portable formats for storage and later restoration.
• Custom Sampler Implementations - Defines user-specific sampling logic by extending base classes to support unique optimization requirements.

Star history