2 Repos
Awesome GitHub RepositoriesMetadata-Driven Dataset Versioning
Tracking large-scale datasets using metadata pointers and abstractions to avoid physical file duplication.
Distinguishing note: None of the candidates cover the specific MLOps pattern of using metadata pointers to avoid duplicating large object storage files.
Explore 2 awesome GitHub repositories matching data & databases · Metadata-Driven Dataset Versioning. Refine with filters or upvote what's useful.
Awesome Metadata-Driven Dataset Versioning GitHub Repositories
Finde die besten Repos mit KI.Wir suchen mit KI nach den am besten passenden Repositories.
- clearml/clearml
clearml/clearml
6,740Auf GitHub ansehenClearML is a comprehensive MLOps platform designed to manage the end-to-end machine learning lifecycle, from initial experimentation to production deployment. It provides a suite of integrated tools including a pipeline orchestrator for automating workflows, an experiment tracking tool for logging hyperparameters and metrics, and a metadata-driven data versioning system for managing large-scale datasets and model artifacts. The platform is distinguished by its advanced compute management and serving capabilities. It features a GPU compute manager that supports fractional resource slicing and
Tracks datasets via metadata abstractions and pointers to avoid duplicating large files in object storage.
Python - allegroai/clearml
allegroai/clearml
6,733Auf GitHub ansehenClearML is a comprehensive MLOps platform designed to manage the entire machine learning lifecycle. It functions as an experiment tracking tool, a data versioning system, and a pipeline orchestrator, while providing infrastructure for GPU cluster management and model serving. The platform is distinguished by its ability to handle hybrid-cloud compute scheduling and fractional GPU allocation, allowing multiple workloads to share a single hardware accelerator. It employs a metadata-based approach to data versioning, using virtual views to track large datasets and artifacts without duplicating r
Tracks dataset versions using metadata pointers to avoid duplicating large physical files in object storage.
Python
