iterative/dvc

Features

• Dataset Versioning Systems - Tracks large datasets and models using lightweight metadata in version control while storing binaries in an external cache.
• Pointer-Based Tracking - Tracks large datasets using lightweight meta-files in version control while storing binaries in an external cache.
• Experiment Tracking - Logs and tracks combinations of hyperparameters and performance metrics to compare machine learning model iterations.
• Machine Learning Experiment Trackers - Provides systems for monitoring metrics and hyperparameters across multiple machine learning model iterations.
• Model Reproducibility Tools - Ensures model reproducibility by syncing exact data and code versions to reconstruct specific experiment states.
• Model Versioning Systems - Tracks and manages iterations of machine learning models and their associated data artifacts for reproducibility.
• Content-Addressable Storage - Implements a content-addressable storage system using hashes to deduplicate large data artifacts.
• Data Pipeline Automation - Executes structured data processing workflows and automatically reruns only modified components.
• Data Pipeline Orchestration - Allows the definition and orchestration of complex data processing sequences through computational graphs.
• Data Pipeline Orchestrators - Automates complex sequences of data processing tasks using computational graphs with automatic change detection.
• Dataset Versioning Platforms - Provides a platform for versioning large research datasets and ML models to ensure training reproducibility.
• Hash-Based Change Detection - Uses cryptographic checksums to detect changes in data or code and determine if pipeline stages need updating.
• Workflow Orchestration - Provides DAG-based pipeline execution to orchestrate data processing steps and optimize re-execution.
• State Reconstruction - Enables the reconstruction of specific experiment states and data versions to reproduce results.
• Cloud Storage Sync Tools - Synchronizes local data caches with cloud platforms or on-premises network storage.
• Dataset Comparators - Analyzes differences and statistical drift between different versions of datasets, models, and parameters.
• Storage Synchronization Services - Implements automated synchronization of large datasets and models between local caches and remote cloud or on-premises storage.
• Remote Build Caches - Provides a remote cache for pushing and pulling large data artifacts to facilitate team collaboration.
• Machine Learning - CLI tool for version control of machine learning data.
• Machine Learning Operations - Version control system for machine learning projects.
• Model Management - Version control system specifically for data and ML models.
• Data Management - Versions data and models for ML experiment management.
• Data Management Systems - Version control system for data in machine learning projects.
• Data Science and ML - Support and discussion for open-source data version control systems.
• Experiment and Data Management - Git-based version control system for ML models and data.
• MLOps and Pipelines - Version control system for data and models.
• Data Science Tooling - Version control system for data and models.
• Data Science Tools - Version control system for data science projects.
• Experimentation Tracking - Provides version control for data, models, and experiment pipelines.
• Project Documentation Examples - Uses a website-like menu and animation for workflows.
• Version Control - Versioning for datasets and machine learning models.
• Version Control Systems - Version control for data and machine learning models.

Star history