Awesome GitHub RepositoriesCustom Dictionaries

Tesseract is a neural network-based optical character recognition engine designed to convert scanned images and digital documents into machine-readable, searchable text. It functions as both a command-line utility for automating large-scale digitization workflows and a cross-platform library that can be embedded into desktop, mobile, or server-side applications. By utilizing long short-term memory networks, the engine provides robust text extraction across more than one hundred languages and dozens of scripts. The project distinguishes itself through a sophisticated document layout analysis f

HanLP is a natural language processing library and deep learning framework specifically optimized for the Chinese language, while also functioning as a multilingual text processor. It serves as a toolkit for performing linguistic analysis, semantic understanding, and script conversion. The project distinguishes itself through a dedicated focus on Chinese linguistic structures, including a specialized script converter for transforming text between Simplified Chinese, Traditional Chinese, and Pinyin. It further supports domain-specific model training to improve the recognition of professional t

This project is a Chinese text segmentation library and tokenizer designed to split Chinese sentences into individual words. It serves as a natural language processing tool for splitting characters into words, tagging parts of speech, and extracting keywords using statistical analysis. The library distinguishes itself through support for custom dictionary configuration and vocabulary file management, allowing users to override default segmentation rules for domain-specific accuracy. It also includes a TF-IDF keyword extractor to identify significant words and core topics within documents. Th

Analysis-ik is a Chinese text segmenter and analysis plugin for Lucene-based search engines. It provides a specialized analyzer for splitting Chinese sentences into meaningful words to improve indexing and search accuracy within Elasticsearch and OpenSearch. The project features a dynamic dictionary manager that can load word libraries and stop-word files from remote HTTP endpoints. It monitors metadata headers on these remote files to trigger automatic vocabulary updates without requiring a service restart. The analyzer supports both fine-grained exhaustive and coarse-grained smart segmenta

LanguageTool is a multilingual grammar and style checking engine designed to detect spelling, grammar, and writing errors across multiple languages. It provides automated proofreading capabilities that can be deployed as a self-hosted server or executed as a standalone local desktop application. The project distinguishes itself through a flexible rule development framework, allowing linguistic patterns to be defined via XML or implemented as custom Java classes. It utilizes n-gram frequency modeling for confused word detection and supports neural word embeddings to improve disambiguation betw

Harper is a local English linter and grammar checker designed as an IDE writing assistant. It operates as a language server that provides real-time spelling and grammar analysis for markdown and code comments, processing all linguistic data on the local device to ensure privacy and eliminate cloud dependencies. The tool is specialized for technical documentation, featuring the ability to identify and ignore text within code fences and inline blocks to prevent false positives. It allows for personalized configuration through custom dictionaries and the use of suppression comments to exclude sp

OpenCC is a library and command-line tool for converting text between Simplified Chinese, Traditional Chinese, and Japanese Kanji. It operates at both the individual character and multi-character phrase levels, and applies region-specific vocabulary choices for Mainland China, Taiwan, and Hong Kong during conversion. The conversion engine resolves ambiguous character mappings using semantic and contextual rules, normalizes variant character forms for consistent orthography, and sequences multiple dictionary files into a configurable pipeline. It supports embedding custom conversion rules dire

pkuseg-python is a Chinese word segmentation toolkit and natural language processing library. It provides specialized models for splitting Chinese text into words across various domains, including news, medical, and web content, and includes a tool for assigning grammatical parts of speech tags to segmented words. The library allows for the training of custom segmentation models using annotated datasets and supports the integration of user-defined dictionaries to ensure specialized terminology is recognized correctly. It employs a multi-threaded execution engine to process large volumes of Ch

ansj_seg is a Java NLP toolkit and segmentation library designed for processing Chinese text. It functions as a word segmenter, part-of-speech tagger, and named entity recognizer to divide continuous Chinese characters into meaningful words and tokens. The library utilizes statistical models for text segmentation and provides capabilities for identifying and extracting person names from unstructured documents. It also assigns grammatical categories to tokens to determine their linguistic roles within a sentence. The toolkit supports domain-specific text processing through the use of custom d

FlashText एक डिक्शनरी-आधारित टेक्स्ट प्रोसेसर और कीवर्ड एक्सट्रैक्शन लाइब्रेरी है जिसे उच्च-प्रदर्शन स्ट्रिंग सर्चिंग और प्रोग्रामेटिक सामग्री प्रतिस्थापन के लिए डिज़ाइन किया गया है। यह टेक्स्ट से पूर्व-निर्धारित शब्दों की पहचान करने और उन्हें पुनः प्राप्त करने या डेटा की बड़ी मात्रा में विशिष्ट शब्दों को प्रतिस्थापन स्ट्रिंग्स के साथ बदलने के लिए विशेष यूटिलिटीज प्रदान करती है। यह प्रोजेक्ट मल्टी-पैटर्न मैचिंग को सक्षम करने के लिए Aho-Corasick स्ट्रिंग सर्च कार्यान्वयन का उपयोग करता है। यह दृष्टिकोण सिंगल-पास टेक्स्ट स्कैनिंग और लीनियर-टाइम कॉम्प्लेक्सिटी सर्च की अनुमति देता है, यह सुनिश्चित करता है कि प्रोसेसिंग समय कीवर्ड डिक्शनरी के साइज़ के बजाय इनपुट टेक्स्ट की लंबाई पर निर्भर करता है। यह लाइब्रेरी लक्षित शब्द सूचियों को बनाए रखने के लिए कीवर्ड डिक्शनरी मैनेजमेंट को कवर करती है और ऑटोमेटेड टेक्स्ट रिप्लेसमेंट और कीवर्ड एक्सट्रैक्शन दोनों के लिए वर्कफ़्लो प्रदान करती है। ये क्षमताएं विशाल डेटासेट्स के भीतर पूर्व-निर्धारित शब्दों के व्यवस्थित संशोधन और पुनर्प्राप्ति की अनुमति देती हैं।

This project is a CJK input method framework and configuration set designed for the Rime input engine. It provides a comprehensive system of schemas and dictionary packs to optimize Chinese character entry through pinyin and double-pinyin workflows. The framework is distinguished by its use of Lua-powered extensions that add dynamic utilities, such as inline mathematical calculators, automated timestamps, and text formatting, directly to the input interface. It also features refined word libraries and language models specifically tuned to improve prediction accuracy and first-choice hit rates