Awesome GitHub RepositoriesDistributed ML Trainers

CatBoost is a gradient boosting machine learning library used to train decision tree ensembles for regression, classification, and ranking tasks. It functions as a high-performance framework that provides a categorical data processor for transforming non-numeric features, a distributed trainer for large-scale datasets, and GPU acceleration to speed up model construction. The library distinguishes itself through native handling of categorical data and text features, removing the need for manual encoding. It includes a specialized model interpretability tool that leverages SHAP values and featu

TensorTrade is a reinforcement learning trading framework designed for training and deploying autonomous agents that optimize financial market strategies. It provides an algorithmic trading simulation environment where agents can be tested against market data using simulated broker environments. The framework features a distributed training system using RLlib to optimize decision policies across large datasets. It includes a walk-forward validation tool that evaluates trading strategies through windowed performance analysis to prevent overfitting and measure real-world viability. The project

llm-foundry 是一个大型语言模型训练框架，提供了一个用于基础模型预训练和监督微调的系统。它包括一个用于跨多个节点和 GPU 扩展工作负载的分布式训练器、一个用于从云存储加载数据的数据集流式传输管道，以及参数高效的微调实现。 该框架通过使用参数分片和高吞吐量数据流来保持大规模训练期间的稳定性，从而脱颖而出。它结合了低秩自适应（LoRA）以降低计算成本，并使用 8 位浮点精度来提高兼容硬件上的计算速度。 该代码库涵盖了广泛的功能，包括将原始数据转换为压缩格式的数据集工程、通过评估套件进行的模型性能基准测试，以及将模型权重导出为标准化行业格式的能力。它还支持通过装饰器进行自定义组件注册，并提供对位置嵌入方法的控制。

该项目是一个容器化机器学习工作流引擎和编排器，旨在自动化 Kubernetes 集群上机器学习模型的端到端生命周期。它作为一个 MLOps 管道编译器，将领域特定语言转换为用于便携式和可扩展部署的结构化规范。 该平台提供了一个具有隔离命名空间和身份提供商认证的多租户环境。它通过结合基于容器的任务隔离、用于数据传递的强类型工件管理以及用于避免冗余计算的内容寻址结果缓存而脱颖而出。 该系统涵盖了全面的工作流编排，包括并行任务执行、循环运行调度和条件分支逻辑。它进一步支持实验跟踪、工作流指标收集以及可重用管道组件的管理，并能够为 CPU、内存和 GPU 配置特定的硬件资源请求。 该软件通过 Python SDK 分发，可部署在独立、本地或多租户环境中。

这是一个 PyTorch 推荐框架和深度学习推荐模型，旨在生成个性化的内容预测。它作为一个分布式嵌入训练器，通过神经网络架构处理密集和稀疏特征来预测用户偏好。 该项目实现了使用专用 GPU 内核的 CUDA 优化机器学习系统，以加速嵌入查找和聚合。它采用分布式方法将海量稀疏特征表分片到多个 GPU 上，从而支持大规模模型的训练。 该系统利用双塔架构进行特征交互，并支持混合并行，结合了计算集群上的数据并行和模型并行。其功能面包括跨网络节点的分布式训练、GPU 内存优化以及基于检查点的状态恢复。

OpenManus-RL is a reinforcement learning framework and distributed training pipeline designed to train large language models as agents. It serves as an agentic reasoning optimizer and reward model trainer, providing the infrastructure to improve model decision-making through reward-based policy optimization. The project distinguishes itself through a distributed architecture that supports parameter sharding across multiple compute nodes and a coordinated rollout system for collecting interaction trajectories. It incorporates advanced reasoning strategies, such as Tree-of-Thoughts and Monte Ca

Hallo2 is an AI video generation tool and audio-driven portrait animation framework designed to transform static images into speaking videos. It functions as a portrait image animator that synchronizes a single photo with an audio track to produce high-resolution talking head videos. The system includes a distributed animation trainer for fine-tuning deep learning models using custom datasets and distributed computing resources. It employs hierarchical video generation and temporal consistency modeling to produce long-form character animations that remain stable over extended durations. The

Open-Instruct is a distributed training and instruction tuning framework for large language models. It functions as a coordinator for supervised fine-tuning, reinforcement learning from human feedback pipelines, and tool-use training, providing specialized roles for dataset curation and model alignment. The project distinguishes itself through a high-performance training architecture that utilizes actor-based distributed coordination and hybrid sharding to manage large GPU clusters. It implements advanced alignment techniques including direct preference optimization, group relative policy opt

This project is a training pipeline and framework for developing Chinese language models based on the Llama 2 architecture. It functions as a distributed GPU trainer and dataset preprocessing toolkit designed for both the initial pre-training of baseline models and subsequent supervised fine-tuning. The system distinguishes itself through a specialized workflow for Chinese text, incorporating a data curation pipeline that uses similarity hashing for deduplication and a tokenization process that converts raw text into memory-mapped binary files for efficient disk access. It implements a superv