BELLE is a specialized implementation of Chinese conversational large language models, encompassing a full instruction tuning framework. It provides a pipeline for training, evaluating, and deploying models optimized for natural language understanding and dialogue tasks in the Chinese language.
The project is distinguished by its integrated approach to model refinement, combining the curation of multi-million entry instruction datasets with a distributed training pipeline. This pipeline supports both full fine-tuning and low-rank adaptation to optimize conversational performance.
The system includes a comprehensive evaluation suite that utilizes categorized test benchmarks and automated scoring prompts to assess output quality. For deployment, it provides a quantized runtime that enables these models to run locally and offline on both desktop and mobile devices.