baidu/lac
Lac
LAC 是一个中文词法分析引擎和工具包，专为联合分词、词性标注和命名实体识别而设计。它作为一个高性能系统，利用训练好的机器学习模型来识别词边界和语法类别。
该项目具有轻量级的编译原生运行时，支持端侧自然语言处理并嵌入到移动应用程序中。它包含模型压缩和转换功能，以针对资源受限的环境进行优化，并支持多线程并行执行以提高吞吐量。
该工具包提供了联合词法分析功能，包括为词重要性评分分配数值权重。用户可以通过自定义词典覆盖来优化预测，或通过在特定领域数据集上进行增量模型训练来提高特定行业的准确性。
Features
- Chinese Natural Language Processing - Provides a comprehensive suite for Chinese natural language processing including segmentation, tagging, and entity recognition.
- Joint Lexical Analysis - Provides a high-performance engine for joint word segmentation, part-of-speech tagging, and named entity recognition in Chinese text.
- Chinese NLP Libraries - Ships a toolkit specifically designed for the joint linguistic analysis of Chinese text.
- Lexical Analysis Engines - Implements a high-performance engine that identifies word boundaries and grammatical categories using trained models.
- Model Inference Engines - Implements a high-performance inference engine that performs joint lexical analysis in a single pass.
- Named Entity Recognition - Includes a processor that categorizes specific entities like people, locations, and organizations within text.
- Native Runtime Libraries - Ships a compiled native runtime library for high-performance model execution on mobile and desktop devices.
- Local NLP Inference - Enables local execution of NLP tasks on-device to operate without network connectivity.
- Part-of-Speech Taggers - Assigns grammatical categories and named entity labels to segmented Chinese words using machine learning models.
- Chinese Language Segmenters - Provides specialized tools for tokenizing and segmenting continuous Chinese text streams.
- Mobile Framework Integrations - Offers lightweight runtime libraries for embedding NLP models directly into mobile applications.
- Incremental Model Training - Supports incremental training on custom datasets to adapt the model to specific industry domains.
- Model Compression - Includes model compression and conversion to optimize computation graphs for resource-constrained environments.
- Keyword Importance Scoring - Assigns numerical weights to words to distinguish core meaning from redundant terms in a query.
- Domain-Specific Fine-Tuning - Supports improving lexical analysis accuracy through incremental fine-tuning on domain-specific datasets.
- Lexicon-Driven Segmenters - Combines machine learning models with user-defined lexicons to refine text segmentation and tagging.
- Custom Dictionary Managers - Provides custom dictionary managers to refine model predictions for word boundaries and labels.
- Prediction Overrides - Allows forcing specific word boundaries and labels using user-defined custom lexicons to override model predictions.
- NLP Mobile Runtimes - Provides a lightweight, compiled interface for executing linguistic analysis models locally on mobile devices.
- Model Compression - Includes model compression and conversion to optimize performance for resource-constrained mobile environments.
- C++ Inference Runtimes - Provides a compiled C++ runtime optimized for executing machine learning model inference.
- Parallel Text Processing - Supports multi-threaded parallel execution to distribute text processing tasks across CPU cores.
- Chinese NLP Toolkits - Baidu's open-source tool for segmentation, tagging, and entity recognition.
