IDSIA/sacred
Sacred
Sacred is an experiment management tool and reproducibility framework designed to organize multiple runs of a process with different configurations. It functions as a machine learning experiment tracker and hyperparameter configuration manager, logging hyperparameters, metrics, and metadata to a database to ensure that experimental executions remain trackable.
The project focuses on scientific result reproducibility by automatically managing random seeds and tracking system dependencies. It allows for the execution of experiment variants through command-line parameter overrides and dynamic parameter injection, enabling the modification of settings without altering the underlying source code.
The framework provides capabilities for database-backed metadata logging, capturing hardware details and software versions to maintain a searchable record of every run. It also supports execution state serialization to enable the exact replication of experimental outcomes.
Features
- Machine Learning Experiment Trackers - Logs and configures experimental executions to ensure that all results remain trackable and reproducible.
- Hyperparameter Configuration Managers - Centralizes model hyperparameters and defines specific settings for runs with runtime overrides.
- Experiment Metadata Tracking - Saves configuration settings, system dependencies, and hardware details to a database for future analysis.
- Result Reproducibility Controls - Manages random seeds automatically to eliminate stochasticity and ensure experimental outcomes can be replicated.
- Experiment Metadata Logging - Records experiment parameters and system state to a persistent database for historical retrieval and auditing.
- Experiment Tracking Tools - Provides a framework for organizing multiple runs of a process and tracking their results in a database.
- Experimental Reproducibility - Manages random seeds and system dependencies to ensure scientific experimental outcomes can be replicated exactly.
- Automated Seed Management - Provides automatic capturing and restoration of random seeds across various libraries to ensure deterministic results.
- Hyperparameter Optimization - Facilitates running multiple versions of a process with different parameters to find optimal settings.
- Command-Line - Allows runtime modification of experiment settings via command-line arguments without altering the source code.
- Experiment Batch Execution - Enables executing different versions of a process by overriding parameters via the command line.
- Environment Dependency Tracking - Captures installed library versions and hardware details to maintain a record for environmental reproducibility.
- Execution State Serializers - Saves the complete configuration and runtime state of a process to enable exact replication of results.
- Parameter-Name Injection - Injects configuration values into function calls by matching variable names with recorded experiment settings.
- Experiment Management - Organizes, logs, and reproduces machine learning experiments.
- MLOps and Infrastructure - Tool for configuring and logging experiments.
- Experiment and Data Management - Tool for configuring and organizing ML experiments.
- Experiment Tracking - Organizes and logs experiment configurations for reproducibility.
- Experimentation Tracking - Organizes and reproduces machine learning experiments.
Star history
How this analysis was created: This summary and feature list were written by an AI model that read the project's README and public documentation pages. Each feature links to the documentation it came from; stars, license and language come straight from the GitHub API. The model does not read the source code, and the analysis is refreshed when the project is re-analysed. Learn more on our About page.
