Awesome GitHub RepositoriesTraining Pipelines

Nanochat is a lightweight execution environment designed for training and running language models on standard consumer hardware. It functions as both a neural network training framework and an inference engine, enabling users to perform backpropagation-based training and model execution directly on general-purpose processors without the need for dedicated graphics hardware. The project distinguishes itself through a suite of optimization tools that prioritize efficiency on local machines. By utilizing memory-mapped weight loading and CPU-optimized vector math, it maximizes throughput for inte

This project is a modular research toolkit designed for developing, training, and evaluating deep learning models for object detection, segmentation, and video instance tracking. It provides a flexible training engine that manages complex neural network execution, including distributed training, custom lifecycle hooks, and weight optimization. The framework is built around a hierarchical configuration system that allows users to define architectures, data pipelines, and training hyperparameters through composable, inheritable files. The project distinguishes itself through its highly modular

This project provides an end-to-end framework for adapting large language models to follow user instructions through supervised fine-tuning. It functions as a comprehensive training pipeline that enables the creation of specialized assistant models by minimizing the difference between predicted outputs and target responses within structured instruction datasets. The framework distinguishes itself by integrating synthetic data generation with memory-efficient training techniques. It utilizes powerful language models to iteratively expand small sets of human-written seeds into diverse, high-qua

This is a framework for training and sampling diffusion models to generate high-fidelity images, video, and 4D assets. It provides a modular environment for managing generative AI training pipelines, including the handling of datasets, noise sampling, and loss weighting to stabilize the creation of synthetic content. The project features a modular model configuration system that uses YAML-based assembly to define network submodules and conditioners. It also includes a dedicated toolset for AI image watermarking, allowing for the embedding and detection of invisible markers to verify the origi

This project is a generative speech synthesis engine that converts text into high-fidelity human speech. It utilizes a two-stage autoregressive transformer architecture that separates semantic token prediction from acoustic detail reconstruction to balance linguistic accuracy with audio quality. The system is designed to support multilingual output and conversational AI development, enabling the generation of context-aware speech that maintains flow across multiple dialogue turns. The platform distinguishes itself through a production-ready inference server that employs continuous batching to

Paddle is a deep learning framework designed for building, training, and deploying large-scale machine learning models. It incorporates a distributed training engine for optimizing performance across multiple chips and a model inference engine for transforming trained models into production-ready formats for cross-platform execution. The platform features a heterogeneous hardware abstraction and a standardized software stack that allows models to run across diverse hardware architectures through a common interface. It also includes a scientific computing library capable of solving complex dif

FinRL-Library is a reinforcement learning trading framework and algorithmic trading library used to develop and backtest automated financial trading strategies. It functions as a quantitative trading pipeline and financial market simulator, allowing users to build decision policies that optimize asset trading across various financial markets. The framework features a modular integration system for swapping reinforcement learning algorithms through a consistent API. It utilizes a standardized environment wrapper to encapsulate market dynamics into a state-action-reward interface, facilitating

PaddleDetection is an object detection framework designed for the end-to-end development, training, and deployment of computer vision models. It provides a comprehensive library of modular neural network architectures and pipelines that support object detection, instance segmentation, and multi-object tracking tasks. The project distinguishes itself through a configuration-driven approach that decouples model components like backbones and heads, allowing for the flexible assembly of custom vision workflows. It incorporates advanced techniques such as anchor-free detection logic, joint detecti

This is a PyTorch implementation of a text-to-image model designed for synthesizing high-fidelity images from natural language descriptions. It utilizes a diffusion image generator to transform latent embeddings into visual data through an iterative denoising process. The system employs a two-stage latent mapping process, using a CLIP-based latent prior to map text embeddings to image embeddings before decoding them into pixels. It features a cascading diffusion decoder that produces high-resolution imagery by passing low-resolution outputs through a sequence of models at increasing scales.

Nougat is a neural OCR system and LLM document parser designed to convert images of academic PDF documents into structured markdown text and mathematical formulas. It functions as a PDF to markdown converter that uses deep learning to handle layout and formula recognition. The project provides a document training pipeline for generating datasets and training neural networks to recognize specific academic document styles. This includes utilities for training dataset generation, neural model training, and model checkpoint management to ensure reproducible deployment. The system covers a broad

LanceDB is a vector database and columnar data store designed to function as a versioned dataset manager and vector search engine. It serves as a high-performance backend for indexing and retrieving high-dimensional embeddings, providing the foundation for machine learning data pipelines. The system distinguishes itself through a combination of cloud-native object storage and immutable version tracking, allowing for data time-travel and reproducible AI experiments. It integrates hybrid search capabilities, merging dense vector similarity with BM25 full-text search and SQL-like scalar filters

nnU-Net is a PyTorch-based deep learning framework for the supervised semantic segmentation of 2D and 3D biomedical images. It functions as an automated medical imaging pipeline that generates predicted masks and labels from clinical images. The system distinguishes itself by using dataset-driven auto-configuration to automatically select the optimal network architecture, preprocessing steps, and training hyperparameters based on the specific properties of the input medical dataset. The framework covers a broad range of capabilities including medical dataset preparation, intensity normalizat

Gluon-CV 是一个 MXNet 计算机视觉库，提供了一系列预实现的视觉架构和训练管线。它作为一个深度学习研究工具包和模型库，包含用于图像和视频分析的最先进预训练权重。 该项目包括一个专门的人体姿态估计库和模型压缩工具包。这些工具允许对深度学习模型进行剪枝和量化，以提高推理速度并促进在受限边缘硬件上的部署。 该库涵盖了广泛的视觉功能，包括图像分类、目标检测以及语义和实例分割。它还提供视频分析工具，如动作识别、目标跟踪和单目深度估计。 训练通过自动化管线和分布式多 GPU 工作负载提供支持，以加速模型收敛。

rllm is an asynchronous reinforcement learning framework for training language agents. It provides a unified pipeline that runs the same agent code for both evaluation and training, automatically capturing traces for gradient computation. The framework supports distributed reinforcement learning across multiple GPUs and nodes using pluggable backends, and executes agents in isolated sandboxes—either locally or in the cloud—for safe and scalable rollout collection. It trains agents built with LangGraph, SmolAgents, OpenAI Agents SDK, or custom frameworks without requiring core logic changes. T

OpenChat is a conversational AI agent builder and customer service automation platform that uses large language models to power customer support chatbots across multiple channels. It provides tools for defining AI agent behavior, training on custom knowledge, managing actions, and controlling autopilot responses per channel. The platform enables deploying AI agents on web, phone, email, SMS, and WhatsApp, with a unified inbox for managing conversations across all channels. It includes CRM synchronization, automated workflows, contact segmentation, and analytics for tracking customer satisfact

AdalFlow 是一个自主 AI 代理框架和 LLM 应用库，旨在构建模块化工作流。它作为一个模型无关的接口和 RAG 流水线编排器，允许用户开发 ReAct 代理，利用迭代推理和外部工具执行来解决复杂任务。 该项目通过一个提示词优化系统脱颖而出，该系统使用文本梯度下降自动优化提示词模板和少样本示例。它将模型反馈视为可微分信号，实现了一种 LLM 反向传播形式，从而根据评估指标迭代提高输出质量。 该框架涵盖了广泛的功能面，包括带有语义向量搜索和重排序的检索增强生成、用于可观测性的基于跨度的执行追踪，以及模式驱动的结构化解析。它为众多专有和开源模型提供商提供了统一的通信层，并支持将 Python 函数转换为标准化的工具接口。 该系统使用 Python 实现，并与 MLflow 集成以进行工作流跟踪和分析。