Open-source alternatives to Dstack

Axolotl is a configuration-driven framework designed for the fine-tuning, evaluation, and quantization of large language models. It functions as a comprehensive orchestrator for distributed training, enabling users to manage complex workflows across multi-node and multi-GPU environments. By utilizing structured configuration files, the platform streamlines the setup of training parameters, dataset paths, and hardware distribution strategies. The project distinguishes itself through its support for diverse training methodologies, including full-parameter tuning, parameter-efficient adaptation,

High Accuracy and efficiency multi-task fine-tuning framework for Code LLMs. This work has been accepted by KDD 2024.

MLeap: Deploy ML Pipelines to Production

Avalanche: an End-to-End Library for Continual Learning based on PyTorch.

The open agent control plane. Govern autonomous AI agents with pre-execution policy enforcement, approval gates, and audit trails. Works with LangChain, CrewAI, MCP, and any framework.

Unified Interface for Constructing and Managing Workflows on different workflow engines, such as Argo Workflows, Tekton Pipelines, and Apache Airflow.

Apache Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does.

3FS is a distributed file system and RDMA storage cluster designed for high-performance AI training and inference workloads. It functions as a strongly consistent storage layer that utilizes a disaggregated architecture to pool SSDs and memory resources across multiple nodes. The system provides specialized storage implementations including an AI training checkpoint store for parallel state preservation and a distributed key-value cache store for decoder layer vectors to optimize inference processing. It ensures data integrity through chain replication and apportioned query distribution. The

Determined is an open-source machine learning platform that simplifies distributed training, hyperparameter tuning, experiment tracking, and resource management. Works with PyTorch and TensorFlow.

🎲 Dotflow turns an idea into flow! — Lightweight Python library for execution pipelines

Fairseq is a PyTorch toolkit for sequence-to-sequence modeling, specializing in neural machine translation, automatic speech recognition, and large-scale language model training. It provides a framework for processing and aligning diverse data sources, including text, audio, and video, to support tasks such as speech-to-text conversion and multimodal sequence learning. The project is distinguished by its distributed training capabilities, which utilize parameter sharding, mixed-precision training, and CPU offloading to handle models that exceed single-device memory. It also includes specializ

Flyte is a Kubernetes-based machine learning orchestrator and containerized pipeline manager designed for coordinating AI workflows and data pipelines. It functions as an engine for defining and executing resilient pipelines, utilizing a data lineage tracker to maintain immutable execution states and ensure reproducible outputs. The platform distinguishes itself by packaging individual tasks into separate containers to ensure dependency isolation and environment consistency. It provides specialized capabilities for machine learning, including the transformation of trained models into scalable

Enterprise Grade, Voice AI simulation SDK for testing your AI Agents

Skaffold is a command-line tool that automates the build, push, and deployment lifecycle for containerized applications on Kubernetes. It functions as a continuous development engine, monitoring source code for changes to trigger incremental updates, manifest hydration, and automated deployments to a cluster. By abstracting the underlying build and deployment tools, it provides a unified interface for managing the inner development loop. The platform distinguishes itself through its environment-aware configuration and flexible build orchestration. It supports diverse build strategies, includi

h2o-3 is a distributed machine learning platform and automated machine learning framework designed for training and deploying predictive models using distributed in-memory computing. It functions as a deep learning framework and a distributed model scoring engine, capable of operating as a Kubernetes ML cluster to process large datasets in parallel. The platform distinguishes itself through automated machine learning capabilities that automatically select the best algorithms and hyperparameters to optimize model performance. It provides specialized deep learning toolkits for tasks including i

This project is a multimodal model trainer and machine learning fine-tuning tool that provides a containerized workflow for adapting pre-trained models to specific tasks. It features a no-code web interface and a dashboard for training large language models and other machine learning datasets without writing code. The system distinguishes itself by integrating a no-code interface with remote GPU orchestration, allowing users to deploy containerized training environments on cloud infrastructure or local hardware. It includes a dedicated integrator for uploading trained model weights and config

Minimalistic large language model 3D-parallelism training

🔮 Instill Core is a full-stack AI infrastructure tool for data, model and pipeline orchestration, designed to streamline every aspect of building versatile AI-first applications

CML is a pipeline automation tool for training and evaluating machine learning models, functioning as a CI/CD system for machine learning. It serves as a cloud compute orchestrator and Git-based workflow manager that automates model training cycles through branch management, automated commits, and integrated reporting. The project distinguishes itself by provisioning ephemeral cloud instances or Kubernetes nodes to provide specialized hardware for compute-heavy tasks. It also manages remote compute runners, allowing the connection of self-hosted GPU clusters or on-premise machines to execute

Kubeflow’s superfood for Data Scientists

Kubeflow is a Kubernetes machine learning platform and containerized toolkit designed to orchestrate the entire machine learning lifecycle. It functions as an MLOps workflow orchestrator and infrastructure layer for building, training, and deploying models within containerized environments. The project provides specialized infrastructure for scaling compute resources and managing GPU workloads for large-scale distributed training. It automates the transition of models from experimental development to production through workflow orchestration and model deployment services. The platform covers

This project is a containerized machine learning workflow engine and orchestrator designed to automate the end-to-end lifecycle of machine learning models on Kubernetes clusters. It functions as an MLOps pipeline compiler that transforms a domain-specific language into structured specifications for portable and scalable deployment. The platform provides a multi-tenant environment with isolated namespaces and identity provider authentication. It distinguishes itself through a combination of container-based task isolation, strongly typed artifact management for data passing, and content-address

Hopsworks - Data-Intensive AI platform with a Feature Store

Langflow is a low-code platform for designing and deploying multi-step AI agent pipelines and large language model sequences. It provides a visual environment to map logic and data flow between components, serving as an orchestrator for managing conversations and data retrieval across multiple autonomous agents. The platform distinguishes itself through a drag-and-drop interface that allows for the construction of complex AI pipelines without extensive boilerplate code. It enables the conversion of these internal workflows into standardized tools for external connectivity via the Model Contex

Ludwig is a multimodal machine learning platform and low-code framework designed for building, training, and deploying neural networks. It enables the construction of models that process text, images, audio, and tabular data through a unified interface using declarative configuration files rather than custom code. The system features a specialized low-code framework for large language models, supporting supervised fine-tuning, preference alignment, and a constrained decoding tool to force structured data output via logit extraction. It also includes an automated model architecture search to i