Cml

CML 是一个用于训练和评估机器学习模型的管道自动化工具，作为机器学习的 CI/CD 系统运行。它作为一个云计算编排器和基于 Git 的工作流管理器，通过分支管理、自动提交和集成报告来自动化模型训练周期。

该项目通过配置临时云实例或 Kubernetes 节点来提供计算密集型任务所需的专用硬件，从而脱颖而出。它还管理远程计算运行器，允许连接自托管 GPU 集群或本地机器来执行容器化机器学习工作流。

该系统涵盖了广泛的功能，包括 ML 实验跟踪（性能指标和可视化直接发布到版本控制 Pull Request 中）。它处理从初始数据导入和版本控制到生成格式化工作流报告和外部可视化链接的 ML 管道自动化。

该工具通过基于 SSH 的远程调试和恢复中断作业的能力，为基础设施管理提供了额外的实用性。

Features

Compute Provisioning - Orchestrates the lifecycle of ephemeral compute instances across hybrid cloud and on-premise providers for ML workloads.
Automation Layers - Manages the end-to-end workflow of data importing, model training, and result reporting without manual intervention.
Experiment Tracking - Logs and visualizes machine learning model training and evaluation workflows by posting metrics directly to pull requests.
Cloud Workflow Runners - Launches specialized cloud-based runners to execute and monitor machine learning workflow tasks.
External Process Registration - Allows external GPU clusters or local machines to register as remote compute nodes for workflow execution.
ML Workflow Automation - Automates the iterative cycle of ML training and evaluation using Git-based workflows and pull requests.
CI-Driven Training - Automatically triggers machine learning model training and evaluation during pull or merge requests.
Runner Installers - Configures local machines or GPU clusters as runners to execute containerized machine learning workflows.
PR Test Result Reporting - Posts model metrics and visualizations as comments on merge requests to provide feedback on workflow results.
Compute Instance Ephemerality - Automates the creation and destruction of cloud compute instances specifically for executing high-compute machine learning tasks.
Cloud Host Provisioning - Automates the provisioning and shutdown of ephemeral virtual machines across cloud providers for compute-heavy tasks.
Infrastructure Deployment Provisioning - Orchestrates the provisioning of compute resources across various cloud and on-premise environments to support ML workflows.
Cloud Infrastructure Orchestration - Orchestrates remote cloud resources and specialized hardware to execute compute-heavy machine learning tasks.
Feedback Loops - Implements a developer-centric feedback loop by posting performance metrics and reports directly into pull requests.
Self-Hosted Deployments - Provides the ability to register and operate on-premise machines as compute nodes for executing workflows.
Pull Request Metric Embeds - Posts data visualizations and performance metrics as comments within a pull request to provide immediate feedback.
Experiment Report Generations - Automatically generates reports from ML experiment data and posts them to pull requests for data-driven decision making.
Pull Request Integration - Generates and posts performance metrics and visualizations directly into version control pull requests for review.
VCS Status Comments - Publishes formatted text, images, and tables as commit comments or status checks in the version control system.
ML Workflow Reports - Packages numeric data and performance visualizations into a formatted report for review within a pull request.
External Storage Bridging - Synchronizes large datasets and artifacts from remote cloud storage into the execution environment.
Pipeline-Integrated Versioning - Integrates data versioning tools directly into the ML pipeline to ensure datasets are synchronized across execution environments.
Automated Pull Request Creation - Commits files to a new branch and opens a pull request without triggering recursive execution loops.
Pipeline Report Publishers - Publishes formatted markdown reports as comments on commits, pull requests, or issues within a version control system.
Platform Check Integrations - Reports workflow execution status and results using native platform check interfaces.
CI/CD Workflows - Triggers a previous job or workflow by its identifier to repeat a process without restarting the full pipeline.
Data Imports - Provides mechanisms to retrieve datasets and model files from cloud storage buckets for use in compute environments.
Job Resumption Systems - Resumes workflow executions after cloud instance interruptions or timeouts to ensure all tasks reach completion.
Self-Hosted Compute Runners - Allows the connection of self-hosted GPU clusters or on-premise machines to execute containerized machine learning workflows.
Automatic Pull Request Creation - Automates the creation of pull requests to integrate generated files into the codebase.
Stateful Run Resumption - Tracks workflow progress to restart interrupted or timed-out tasks from the last successfully checkpointed state.
External Dashboard Integrations - Generates dynamic links connecting workflow executions to external training logs and visualization dashboards.
PR Metric Reporting - Creates comments in pull requests to display performance metrics and visualizations within the version control interface.
Pipeline Metric Reports - Generates custom visualizations of pipeline execution metrics and posts them into pull requests.
CI/CD for Machine Learning - Library for implementing CI/CD in machine learning projects.
MLOps and Pipelines - Continuous integration for machine learning projects.
MLOps and Workflows - Library for implementing CI/CD in machine learning projects.
Training and Orchestration - Library for implementing CI/CD in machine learning projects.
Workflow Orchestration - CI/CD for Machine Learning Projects.
Data Science Tooling - Continuous integration for data science projects.
Data Science Tools - Continuous integration for data science projects.

Star 历史

iterativecml

Name: iterative/cml
Author: iterative

View on GitHub

4,178 星标345 分支JavaScriptApache-2.04 次浏览cml.dev

Cml

该工具通过基于 SSH 的远程调试和恢复中断作业的能力，为基础设施管理提供了额外的实用性。

Features

Compute Provisioning - Orchestrates the lifecycle of ephemeral compute instances across hybrid cloud and on-premise providers for ML workloads.
Automation Layers - Manages the end-to-end workflow of data importing, model training, and result reporting without manual intervention.
Experiment Tracking - Logs and visualizes machine learning model training and evaluation workflows by posting metrics directly to pull requests.
Cloud Workflow Runners - Launches specialized cloud-based runners to execute and monitor machine learning workflow tasks.
External Process Registration - Allows external GPU clusters or local machines to register as remote compute nodes for workflow execution.
ML Workflow Automation - Automates the iterative cycle of ML training and evaluation using Git-based workflows and pull requests.
CI-Driven Training - Automatically triggers machine learning model training and evaluation during pull or merge requests.
Runner Installers - Configures local machines or GPU clusters as runners to execute containerized machine learning workflows.
PR Test Result Reporting - Posts model metrics and visualizations as comments on merge requests to provide feedback on workflow results.
Compute Instance Ephemerality - Automates the creation and destruction of cloud compute instances specifically for executing high-compute machine learning tasks.
Cloud Host Provisioning - Automates the provisioning and shutdown of ephemeral virtual machines across cloud providers for compute-heavy tasks.
Infrastructure Deployment Provisioning - Orchestrates the provisioning of compute resources across various cloud and on-premise environments to support ML workflows.
Cloud Infrastructure Orchestration - Orchestrates remote cloud resources and specialized hardware to execute compute-heavy machine learning tasks.
Feedback Loops - Implements a developer-centric feedback loop by posting performance metrics and reports directly into pull requests.
Self-Hosted Deployments - Provides the ability to register and operate on-premise machines as compute nodes for executing workflows.
Pull Request Metric Embeds - Posts data visualizations and performance metrics as comments within a pull request to provide immediate feedback.
Experiment Report Generations - Automatically generates reports from ML experiment data and posts them to pull requests for data-driven decision making.
Pull Request Integration - Generates and posts performance metrics and visualizations directly into version control pull requests for review.
VCS Status Comments - Publishes formatted text, images, and tables as commit comments or status checks in the version control system.
ML Workflow Reports - Packages numeric data and performance visualizations into a formatted report for review within a pull request.
External Storage Bridging - Synchronizes large datasets and artifacts from remote cloud storage into the execution environment.
Pipeline-Integrated Versioning - Integrates data versioning tools directly into the ML pipeline to ensure datasets are synchronized across execution environments.
Automated Pull Request Creation - Commits files to a new branch and opens a pull request without triggering recursive execution loops.
Pipeline Report Publishers - Publishes formatted markdown reports as comments on commits, pull requests, or issues within a version control system.
Platform Check Integrations - Reports workflow execution status and results using native platform check interfaces.
CI/CD Workflows - Triggers a previous job or workflow by its identifier to repeat a process without restarting the full pipeline.
Data Imports - Provides mechanisms to retrieve datasets and model files from cloud storage buckets for use in compute environments.
Job Resumption Systems - Resumes workflow executions after cloud instance interruptions or timeouts to ensure all tasks reach completion.
Self-Hosted Compute Runners - Allows the connection of self-hosted GPU clusters or on-premise machines to execute containerized machine learning workflows.
Automatic Pull Request Creation - Automates the creation of pull requests to integrate generated files into the codebase.
Stateful Run Resumption - Tracks workflow progress to restart interrupted or timed-out tasks from the last successfully checkpointed state.
External Dashboard Integrations - Generates dynamic links connecting workflow executions to external training logs and visualization dashboards.
PR Metric Reporting - Creates comments in pull requests to display performance metrics and visualizations within the version control interface.
Pipeline Metric Reports - Generates custom visualizations of pipeline execution metrics and posts them into pull requests.
CI/CD for Machine Learning - Library for implementing CI/CD in machine learning projects.
MLOps and Pipelines - Continuous integration for machine learning projects.
MLOps and Workflows - Library for implementing CI/CD in machine learning projects.
Training and Orchestration - Library for implementing CI/CD in machine learning projects.
Workflow Orchestration - CI/CD for Machine Learning Projects.
Data Science Tooling - Continuous integration for data science projects.
Data Science Tools - Continuous integration for data science projects.

Cml 的开源替代方案

相似的开源项目，按与 Cml 的功能重合度排序。

maiot-io/zenml
maiot-io/zenml
5,452在 GitHub 上查看
ZenML is an extensible machine learning orchestration framework designed to manage the end-to-end lifecycle of data pipelines and AI agent workflows. It functions as a durable orchestrator that executes machine learning tasks as directed acyclic graphs, ensuring that every step is containerized for consistent performance across local, cloud, and hybrid infrastructure. By decoupling pipeline code from underlying compute and storage backends, the platform allows developers to define infrastructure-agnostic stacks that remain portable across diverse environments. The project distinguishes itself
Python
在 GitHub 上查看5,452
zenml-io/zenml
zenml-io/zenml
5,451在 GitHub 上查看
ZenML is an orchestration platform designed for building, deploying, and monitoring reproducible machine learning pipelines and agentic workflows. It provides a unified framework that manages the entire lifecycle of machine learning assets, from data processing and model training to the deployment of persistent inference services. By decoupling pipeline logic from underlying compute and storage, the platform enables teams to transition workflows seamlessly from local development environments to production-grade cloud infrastructure. The platform distinguishes itself through a service-oriented
Pythonagentopsagentsai
在 GitHub 上查看5,451
iterative/dvc
iterative/dvc
15,680在 GitHub 上查看
DVC is a data versioning tool and pipeline orchestrator designed to track large datasets and machine learning models. It functions as a system for managing large data artifacts by storing lightweight metadata in version control while keeping the actual binaries in a separate cache. The project serves as an experiment tracker and remote storage synchronizer, enabling the execution and comparison of machine learning iterations based on hyperparameters and performance metrics. It provides a bridge for pushing and pulling these large data artifacts between local environments and cloud or on-premi
Python
在 GitHub 上查看15,680

常见问题解答

iterative/cml 是做什么的？

iterative/cml 的主要功能有哪些？

iterative/cml 的主要功能包括：Compute Provisioning, Automation Layers, Experiment Tracking, Cloud Workflow Runners, External Process Registration, ML Workflow Automation, CI-Driven Training, Runner Installers。

iterative/cml 有哪些开源替代品？

iterative/cml 的开源替代品包括： maiot-io/zenml — ZenML is an extensible machine learning orchestration framework designed to manage the end-to-end lifecycle of data… zenml-io/zenml — ZenML is an orchestration platform designed for building, deploying, and monitoring reproducible machine learning… iterative/dvc — DVC is a data versioning tool and pipeline orchestrator designed to track large datasets and machine learning models.… tensorchord/envd — 🏕️ Reproducible development environment for humans and agents. transformerlab/transformerlab-app — TransformerLab is an MLOps orchestration platform and research environment designed for the training, fine-tuning, and… polyaxon/polyaxon — Polyaxon is a Kubernetes-native machine learning orchestration platform and MLOps pipeline orchestrator. It serves as…