BYVoid/OpenCC
OpenCC
OpenCC is a library and command-line tool for converting text between Simplified Chinese, Traditional Chinese, and Japanese Kanji. It operates at both the individual character and multi-character phrase levels, and applies region-specific vocabulary choices for Mainland China, Taiwan, and Hong Kong during conversion.
The conversion engine resolves ambiguous character mappings using semantic and contextual rules, normalizes variant character forms for consistent orthography, and sequences multiple dictionary files into a configurable pipeline. It supports embedding custom conversion rules directly in configuration files and can integrate external C++ segmentation plugins, such as Jieba, to improve phrase boundary detection before applying conversion rules.
OpenCC provides programming language bindings for Python, Node.js, and C++, enabling direct integration into applications and scripts. It also offers a command-line interface for batch conversion with in-place file editing and custom dictionary configurations, and can transform benchmark datasets for evaluating language models across Chinese script variants.
Features
- Chinese Character Simplifiers - Converts text between Simplified Chinese, Traditional Chinese, and Japanese Kanji at both character and phrase levels.
- Script Conversion - Provides command-line conversion between Chinese script variants with custom dictionary support.
- Chinese Script Normalizers - Normalizes Chinese text by converting script variants and standardizing variant characters for NLP pipelines.
- Regional Script Conversions - Applies region-specific word choices for Mainland China, Taiwan, and Hong Kong during script conversion.
- Regional Vocabulary Selectors - Applies region-specific word choices during script conversion to match local usage in Taiwan, Hong Kong, or mainland China.
- Chinese Character Disambiguations - Disambiguates characters with multiple script counterparts using semantic and contextual rules.
- Dual-Level Character and Phrase Mappings - Converts text at both individual character and multi-character phrase levels using separate dictionary layers.
- Dictionary-Chained Conversion Pipelines - Sequences multiple dictionary files into a configurable conversion pipeline for script transformation.
- Chinese Script Normalization Pipelines - Normalizes Chinese text corpora by converting between script variants during NLP preprocessing.
- Scripting Language Integration - Provides Python, Node.js, and C++ bindings for integrating Chinese script conversion into applications.
- Script Ambiguity Resolvers - Disambiguates characters that map to multiple counterparts in the other script using semantic and contextual rules.
- Chinese Language Segmenters - Segments Chinese text into phrases to improve conversion accuracy for multi-character terms and regional expressions.
- Segmentation-Based Converters - Segments Chinese text into phrases before conversion to improve accuracy for multi-character terms.
- Pre-Conversion Segmenters - Segments input text into phrases before conversion to improve accuracy for multi-character terms and regional expressions.
- Terminal Text Conversions - Runs conversion directly from the terminal with in-place file editing and multiple dictionary configurations.
- Language Bindings - Provides Python, Node.js, and C++ bindings for programmatic Chinese script conversion.
- Multi-Language Bindings - Provides C++ library with Python, Node.js, and command-line bindings for Chinese script conversion.
- Dictionary Chaining - Loads and sequences multiple dictionaries to define custom conversion paths between Chinese script variants.
- Inline Conversion Rule Embeddings - Embeds small custom conversion rules directly in configuration files without modifying external dictionaries.
- Plugin Integrations - Integrates external C++ segmentation plugins like Jieba to improve phrase-level conversion accuracy.
- Segmentation Plugin Loadings - Loads an optional Jieba segmentation plugin to improve phrase boundary detection during conversion.
- Segmentation Plugin Integrations - Loads external C++ segmentation plugins like Jieba to improve phrase boundary detection during conversion.
- Variant Character Normalizers - Replaces variant character forms with a standard representative character to ensure consistent orthography.
