Awesome GitHub RepositoriesText Model Training

fastText is a library and framework for word embedding generation, text vectorization, and supervised text classification. It provides tools to transform raw text into fixed-length vector representations and to train models that assign category labels to sentences or documents. The system utilizes subword-based vectorization and character n-gram embeddings, allowing it to generate meaningful vectors for words that were not present during training. To manage resource usage, it includes a quantized language model implementation that employs product quantization and dimensionality reduction to d

AutoGluon is an automated machine learning framework and multimodal library designed to automate the end-to-end pipeline from data preprocessing to high-accuracy model training and validation. It functions as an automated model trainer for tabular, image, text, and time series data, as well as a tool for time series forecasting and foundation model finetuning. The project is distinguished by its ability to jointly process and fuse different data types, allowing for the construction of multimodal neural networks that integrate images, text, and structured tables. It supports zero-shot inferenc

This project is a Transformer machine translation model and attention-based neural network implemented using the PyTorch deep learning framework. It functions as a text-to-text translation tool designed to convert source sequences into target language text. The implementation focuses on neural machine translation, covering the development of sequence-to-sequence architectures. It includes the full pipeline for translation, from text sequence preprocessing and vocabulary creation to model training and text generation inference. The system incorporates standard transformer components such as a

This project is a toolkit for fine-tuning and managing text-to-image diffusion models. It focuses on low-rank adaptation to create small, portable weight files that customize model styles and behaviors without modifying the entire base model. The project provides specialized utilities for model distillation using singular value decomposition to extract adapters from fully trained models, as well as tools for blending and merging multiple adapters through weight interpolation. It includes capabilities for subject inversion and pivotal tuning to increase the visual fidelity of specific identiti

nlp.js is a JavaScript natural language processing library and development framework used to build natural language understanding engines. It provides a toolkit for creating local machine learning models for intent classification and acts as a multilingual text processor that detects languages and normalizes text across various dialects. The framework distinguishes itself by supporting local execution on both servers and mobile devices, enabling chatbot functionality without an internet connection. It features a specialized system for conversational slot filling to collect mandatory informati

这是一个机器学习框架，用于将多样化的自然语言处理任务视为统一的文本到文本问题。它提供了一个用于预训练和微调大规模 Transformer 模型的工具包，利用一种将输入和输出都格式化为原始文本序列的系统。 该框架的特色在于其分布式训练系统，该系统使用基于网格的策略跨多个 TPU 核心扩展模型权重和训练批次。它通过使用可配置的混合率将多样化的数据集组合成单一训练流来支持多任务学习，从而允许单个模型处理各种语言任务。 该系统涵盖了广泛的功能，包括编码器-解码器架构、用于文本生成的束搜索解码以及迁移学习工作流。它包括用于 NLP 数据集准备、模型性能评估以及导出训练检查点以进行生产服务的实用程序。 该库支持加载各种大小的预训练模型检查点以加速开发。

pycorrector is an open-source toolkit for detecting and correcting spelling and grammar errors in Chinese text. It combines multiple correction approaches, including rule-based methods using Kenlm n-gram language models and confusion sets, as well as deep learning correctors built on BERT, GPT, and T5 models. The toolkit also provides a command-line interface for batch processing Chinese text files with configurable detection and output options. The project distinguishes itself by offering a range of correction strategies that can be mixed and matched. Rule-based correction uses character-lev

This project is a neural machine translation system used to build models that automatically translate text from one language to another. It utilizes sequence-to-sequence modeling to transform variable-length input sequences into corresponding output sequences. The system implements bidirectional recurrent neural network encoding and attention mechanisms to capture contextual information and focus on specific parts of the source text during translation. To manage training and inference, it employs separate computational graphs and supports distributing model layers across multiple GPU devices.

This project is a collection of educational resources and technical guides focused on the development and implementation of large language models. It provides a comprehensive curriculum covering transformer architectures, training methods, and deployment strategies. The materials provide detailed instructions for building autonomous agents using reasoning loops and tool integration, as well as guides for fine-tuning models through supervised learning and preference optimization. It also includes tutorials for constructing retrieval augmented generation pipelines and implementing transformer m

PyText is an extensible PyTorch-based framework for building, training, and deploying custom natural language processing models, including text classifiers, sequence taggers, and intent-slot predictors. It provides a modular toolkit that allows developers to assemble these models using pluggable registries for model architectures, data formats, and tensorizers, all configurable through YAML files without requiring code changes. The framework distinguishes itself through its comprehensive support for the full NLP model lifecycle, from training to production inference. It includes pre-built neu

Torchtune is a PyTorch-native library for fine-tuning, aligning, and quantizing large language models. It provides a config-driven system for instantiating components, orchestrating distributed training, and managing parameter-efficient fine-tuning with quantization support, all through YAML-based configurations and command-line overrides. The library distinguishes itself through its comprehensive post-training workflow orchestration, combining supervised fine-tuning, preference optimization (DPO, PPO, GRPO), knowledge distillation, and quantization-aware training in a single configurable pip

该项目是一个用于文本分类的卷积神经网络的 TensorFlow 实现。它作为一个深度学习文本分类器，通过识别和分析训练集中的学习模式，将预定义标签分配给文本文档。 该模型利用嵌入层向量化、用于特征提取的卷积层和用于处理文本数据的最大池化下采样序列。最终类别概率通过密集层分类系统确定。 该工作流涵盖了端到端的机器学习生命周期，包括监督文本学习、具有可配置嵌入维度和过滤器大小的模型训练，以及使用验证数据集和基于检查点的模型持久化进行的性能评估。

BLIP is a vision-language model framework that combines contrastive, matching, and language modeling objectives to align images with text. Built on a multimodal encoder-decoder architecture, it supports distributed data-parallel training with cosine learning rate scheduling and sliding-window metric tracking for training stability. The framework provides capabilities for image captioning, visual question answering, and cross-modal retrieval, scoring semantic alignment between images and text through learned embeddings. It includes toolkits for fine-tuning pre-trained models on custom datasets

Neuraltalk is an automated image captioning system that generates natural language descriptions for images. It utilizes a deep learning model that integrates a pretrained convolutional neural network for visual feature extraction with a recurrent neural network decoder to produce text sequences. The project provides a full workflow for training and evaluating captioning models, including weight optimization via backpropagation and gradient descent. It includes tools for measuring caption accuracy by comparing generated text against reference descriptions. The system covers data preprocessing

该项目是一个综合性教育计划和深度学习框架，旨在通过 Notebook 和代码示例教授 PyTorch 深度学习实践。它作为一个用于构建、训练和部署神经网络的高级库，充当模型训练编排器，协调 PyTorch 模型、优化器和损失函数。 该项目为计算机视觉、自然语言处理和表格数据预处理提供了专门的工具包。它通过高级训练控制脱颖而出，例如判别式学习率、用于自定义训练逻辑的双向回调系统，以及自动化设备放置和训练循环的高级学习器抽象。 该框架涵盖了广泛的能力面，包括自动化数据流水线构建、模型架构分析以及跨分类、回归和分割任务的性能评估。它还包括用于跨多个 GPU 进行分布式训练的工具、用于内存优化的混合精度训练，以及对医学影像数据的专门支持。 该项目以一系列 Jupyter Notebook 的形式交付。

Kolors 是一个生成式模型实现，用于根据自然语言描述和视觉参考合成逼真的图像。它利用潜在扩散模型框架来生成高保真图像，并在压缩的潜在空间内运行以提高生成效率和质量。 该系统作为一个多语言图像生成器，能够解释多种语言的文本提示词，从而产生语义准确的视觉输出。它包含一个自定义模型训练流水线，使用低秩自适应（LoRA）技术，通过少量图像集教会模型特定的主题或艺术风格。 该项目涵盖了广泛的图像合成和编辑功能，包括文生图和图生图转换。它提供了通过深度图或姿态图进行空间布局控制的工具、用于美学一致性的视觉身份注入，以及基于掩码的修复（Inpainting）以重构或修改特定图像区域。 该实现包括图像质量评估工具，根据人类偏好指标对生成图像的美学和语义质量进行评分。

这是一个多模态模型训练器和机器学习微调工具，为将预训练模型适配到特定任务提供了容器化工作流。它具有无代码 Web 界面和仪表板，无需编写代码即可训练大语言模型和其他机器学习数据集。 该系统的特点是将无代码界面与远程 GPU 编排相结合，允许用户在云基础设施或本地硬件上部署容器化训练环境。它包含一个专门的集成器，用于将训练好的模型权重和配置直接上传到 Hugging Face Hub。 其功能涵盖多种模态，包括用于图像分类和目标检测的计算机视觉、用于文本分类和序列到序列任务的自然语言处理，以及用于表格数据的监督学习。该工具包还支持通过参数高效微调和量化进行专门的语言模型优化。 训练工作流通过配置文件或基于浏览器的界面进行管理，并内置了对数据集列映射和训练进度监控的支持。

Spark NLP 是一个构建在 Apache Spark 分布式计算框架之上的可扩展文本分析和机器学习工具包。它提供了一个多模态机器学习框架和一个用于对标注器进行排序以处理大规模语言数据的分布式流水线系统。该库包含一个用于生成上下文向量嵌入的 Transformer 文本处理器，以及一个用于管理大型语言模型的专用推理引擎。 该项目通过其在统一视觉-语言架构内处理异构数据类型（包括文本、音频和图像）的能力而脱颖而出。它支持高级生成式 AI 功能，如提示工程、具有约束 JSON 输出的结构化实体提取，以及消除网络延迟的本地推理。此外，它还提供跨文本和图像模态的跨语言翻译和零样本分类工具。 该框架涵盖了广泛的功能，包括用于实体识别和情感分析的监督模型训练，以及抽取式问答和文档摘要。它集成了向量数据库支持以进行相似性搜索，并为 GPU 加速和通过集中式注册表进行模型生命周期管理提供了基础设施。 该工具包允许通过公共仓库分发自定义模型和流水线，并支持通过 REST API 部署模型。

本项目是 ALBERT 语言模型架构的实现，提供了一个用于训练和评估基于 Transformer 的文本分类器和相似度模型的框架。它特别包括针对生成中文文本的语义嵌入和表示而优化的预训练资产和工具。 该框架通过将重型语言模型检查点转换为轻量级格式以在移动设备上实现低延迟推理的工具脱颖而出。它利用特定的权重缩减技术，包括跨参数共享和因子分解嵌入参数化，以在较小的内存占用下保持性能。 该系统涵盖了自然语言处理的完整流水线，从原始文本归一化和子词分词，到使用掩码语言建模的自监督预训练。它提供下游任务适配功能，允许对预训练模型进行微调以进行文本相似度分析和监督分类。 该项目包括用于二进制记录数据转换和模型格式转换的实用程序，以确保跨不同机器学习平台的兼容性。

这是一个基于 PyTorch 的场景文本识别框架和工具包。它提供了一个深度学习流水线，用于从自然环境的图像中提取字符和单词，涵盖了从训练数据准备到模型验证的完整过程。 该框架作为衡量文本识别模型准确性和推理速度的标准化基准。它包括用于计算识别准确率和测量每张图像 GPU 处理时间的工具，以评估模型在一致数据集上的性能。 该系统结合了视觉和序列处理阶段，利用卷积特征提取和循环序列建模。它包括用于文本和索引转换的数据工程工具，以及用于在训练期间管理数据集分布的批处理级数据平衡功能。