2 个仓库
Training BERT models on text error correction data to learn and correct character-level mistakes in Chinese text.
Distinct from Text Model Training: Distinct from Text Model Training: specifically trains models for error correction, not general text model training.
Explore 2 awesome GitHub repositories matching artificial intelligence & ml · Error Correction Model Training. Refine with filters or upvote what's useful.
pycorrector is an open-source toolkit for detecting and correcting spelling and grammar errors in Chinese text. It combines multiple correction approaches, including rule-based methods using Kenlm n-gram language models and confusion sets, as well as deep learning correctors built on BERT, GPT, and T5 models. The toolkit also provides a command-line interface for batch processing Chinese text files with configurable detection and output options. The project distinguishes itself by offering a range of correction strategies that can be mixed and matched. Rule-based correction uses character-lev
Provides training scripts for BERT models on paired error-correction data to learn character-level corrections.
Spark NLP 是一个构建在 Apache Spark 分布式计算框架之上的可扩展文本分析和机器学习工具包。它提供了一个多模态机器学习框架和一个用于对标注器进行排序以处理大规模语言数据的分布式流水线系统。该库包含一个用于生成上下文向量嵌入的 Transformer 文本处理器,以及一个用于管理大型语言模型的专用推理引擎。 该项目通过其在统一视觉-语言架构内处理异构数据类型(包括文本、音频和图像)的能力而脱颖而出。它支持高级生成式 AI 功能,如提示工程、具有约束 JSON 输出的结构化实体提取,以及消除网络延迟的本地推理。此外,它还提供跨文本和图像模态的跨语言翻译和零样本分类工具。 该框架涵盖了广泛的功能,包括用于实体识别和情感分析的监督模型训练,以及抽取式问答和文档摘要。它集成了向量数据库支持以进行相似性搜索,并为 GPU 加速和通过集中式注册表进行模型生命周期管理提供了基础设施。 该工具包允许通过公共仓库分发自定义模型和流水线,并支持通过 REST API 部署模型。
Provides tools for training models to detect and correct character-level spelling mistakes in text.