hyperopt/hyperopt
Hyperopt
Hyperopt is a Python library for hyperparameter optimization designed to minimize scalar-valued objective functions. It operates as a stochastic search space engine that finds optimal input parameters by searching through real-valued, discrete, and conditional spaces.
The framework distinguishes itself through its support for complex search space configurations, allowing for conditional parameter hierarchies where specific hyperparameters are sampled only if their parent parameters meet certain criteria. It is built as an asynchronous optimization framework, decoupling the generation of search points from their evaluation.
The system provides capabilities for distributed parameter search, utilizing database-backed state coordination to synchronize trial results across multiple concurrent workers and machines. This infrastructure enables parallel objective evaluation and asynchronous experiment tracking to monitor progress and resume interrupted searches.
Features
- Asynchronous Optimization Frameworks - Provides an asynchronous optimization framework that decouples the generation of search points from their evaluation.
- Hyperparameter Optimization - Implements automated methods for searching and selecting the best configuration parameters for scalar-valued objective functions.
- Hyperparameter Optimizers - A Python library for minimizing objective functions by searching through real-valued, discrete, and conditional hyperparameter spaces.
- Conditional Parameter Dependencies - Provides support for complex search spaces with conditional parameter hierarchies.
- Search Space Definition - Defines stochastic search spaces using expressions that support nested conditional logic and diverse data types.
- Conditional Search Space Configuration - Constructs hierarchical parameter sets where specific distributions are active only when their parent parameters are selected.
- Automated Parameter Searches - Implements a framework for parallelizing the search for optimal model parameters across multiple workers.
- Objective Function Minimization - Minimizes scalar-valued objective functions by iteratively sampling from a defined hyperparameter search space.
- Scalar-Valued Function Optimization - Optimizes potentially stochastic scalar-valued functions to find the best set of input arguments.
- Iterative Error Minimization - Iteratively samples candidate parameters and updates the internal model to minimize a scalar-valued objective function.
- Parallel Evaluators - Spreads the computation of search points across multiple worker processes to accelerate objective function evaluation.
- Experiment Tracking - Tracks long-term progress and allows resuming interrupted searches by storing results and metadata in a database.
- Result Persistence Layers - Persists trial data in a database to track long-term progress and resume interrupted optimization searches.
- Distributed Computing Frameworks - Parallelizes the hyperparameter search process across multiple machines using external clusters or database backends.
- Distributed Optimization Synchronization - Coordinates parallel optimization trials across multiple processes using a shared database for synchronization.
- Distributed Task Workers - Implements a system of distributed workers to evaluate objective functions asynchronously without blocking the main process.
- Master-Worker Coordination - Uses a shared database to coordinate multiple optimization processes across different workers.
- Distributed State Coordination - Uses a database-backed state coordination mechanism to synchronize trial results across distributed workers.
- Real-Time Trial Synchronization - Enables real-time updates of partial results and injection of new search points into parallel experiments.
- Optimization Trials - Runs multiple optimization trials across different machines or clusters to find optimal parameters faster.
- Automated Machine Learning - Library for serial and parallel optimization over awkward search spaces.
- Hyperparameter Tuning - Hyperparameter optimization.
- Optimization - Distributed asynchronous hyperparameter optimization.
