baidu/lac
Lac
LAC is a Chinese lexical analysis engine and toolkit designed for joint word segmentation, part-of-speech tagging, and named entity recognition. It functions as a high-performance system that identifies word boundaries and grammatical categories using trained machine learning models.
The project features a lightweight, compiled native runtime that enables on-device natural language processing and embedding into mobile applications. It includes model compression and conversion to optimize for resource-constrained environments and supports multi-threaded parallel execution to increase throughput.
The toolkit provides capabilities for joint lexical analysis, including the ability to assign numerical weights for word importance scoring. Users can refine predictions through custom dictionary overrides or improve accuracy for specific industries using incremental model training on domain-specific datasets.
Features
- Chinese Natural Language Processing - Provides a comprehensive suite for Chinese natural language processing including segmentation, tagging, and entity recognition.
- Joint Lexical Analysis - Provides a high-performance engine for joint word segmentation, part-of-speech tagging, and named entity recognition in Chinese text.
- Chinese NLP Libraries - Ships a toolkit specifically designed for the joint linguistic analysis of Chinese text.
- Lexical Analysis Engines - Implements a high-performance engine that identifies word boundaries and grammatical categories using trained models.
- Model Inference Engines - Implements a high-performance inference engine that performs joint lexical analysis in a single pass.
- Named Entity Recognition - Includes a processor that categorizes specific entities like people, locations, and organizations within text.
- Native Runtime Libraries - Ships a compiled native runtime library for high-performance model execution on mobile and desktop devices.
- Local NLP Inference - Enables local execution of NLP tasks on-device to operate without network connectivity.
- Part-of-Speech Taggers - Assigns grammatical categories and named entity labels to segmented Chinese words using machine learning models.
- Chinese Language Segmenters - Provides specialized tools for tokenizing and segmenting continuous Chinese text streams.
- Mobile Framework Integrations - Offers lightweight runtime libraries for embedding NLP models directly into mobile applications.
- Incremental Model Training - Supports incremental training on custom datasets to adapt the model to specific industry domains.
- Model Compression - Includes model compression and conversion to optimize computation graphs for resource-constrained environments.
- Keyword Importance Scoring - Assigns numerical weights to words to distinguish core meaning from redundant terms in a query.
- Domain-Specific Fine-Tuning - Supports improving lexical analysis accuracy through incremental fine-tuning on domain-specific datasets.
- Lexicon-Driven Segmenters - Combines machine learning models with user-defined lexicons to refine text segmentation and tagging.
- Custom Dictionary Managers - Provides custom dictionary managers to refine model predictions for word boundaries and labels.
- Prediction Overrides - Allows forcing specific word boundaries and labels using user-defined custom lexicons to override model predictions.
- NLP Mobile Runtimes - Provides a lightweight, compiled interface for executing linguistic analysis models locally on mobile devices.
- Model Compression - Includes model compression and conversion to optimize performance for resource-constrained mobile environments.
- C++ Inference Runtimes - Provides a compiled C++ runtime optimized for executing machine learning model inference.
- Parallel Text Processing - Supports multi-threaded parallel execution to distribute text processing tasks across CPU cores.
- Chinese NLP Toolkits - Baidu's open-source tool for segmentation, tagging, and entity recognition.
