Awesome GitHub RepositoriesError Correction Model Training

pycorrector is an open-source toolkit for detecting and correcting spelling and grammar errors in Chinese text. It combines multiple correction approaches, including rule-based methods using Kenlm n-gram language models and confusion sets, as well as deep learning correctors built on BERT, GPT, and T5 models. The toolkit also provides a command-line interface for batch processing Chinese text files with configurable detection and output options. The project distinguishes itself by offering a range of correction strategies that can be mixed and matched. Rule-based correction uses character-lev

Spark NLP is a toolkit for scalable text analysis and machine learning built on the Apache Spark distributed computing framework. It provides a multimodal machine learning framework and a distributed pipeline system for sequencing annotators to process large-scale linguistic data. The library includes a transformer text processor for generating contextual vector embeddings and a dedicated inference engine for managing large language models. The project distinguishes itself through its ability to process heterogeneous data types, including text, audio, and images, within a unified vision-langu