thunlp/UltraChat
UltraChat
UltraChat is a collection of large-scale conversational datasets and instruction-tuning data designed for training and evaluating generative AI models. It provides structured JSON data consisting of complex, multi-round dialogue sequences intended to refine the performance of large language models in chat tasks.
The project focuses on improving reasoning and response quality through a diverse set of interactions across multiple sectors. These datasets are used for supervised fine-tuning and instruction tuning workflows to improve how models follow complex directions and maintain context across sequential turns.
The codebase includes a pipeline for supervised fine-tuning and a cross-sector evaluation framework to measure model coherence and generalization across different topical datasets. It also covers data curation, serialization, and partitioning to ensure broad knowledge coverage during the training process.
Features
- Training Datasets - Provides large-scale conversational training datasets in JSON format for generative AI models.
- LLM Dataset Collection - Provides a large-scale collection of multi-round conversational datasets for training and evaluating generative AI models.
- Conversational AI - Provides a structured JSON training set of complex dialogue sequences for generative AI.
- Conversational Dataset Sourcing - Acquires and structures JSON conversational data across multiple sectors to build training pipelines.
- Instruction Tuning - Curates high-quality prompt and response pairs to improve model instruction-following capabilities.
- Instruction Tuning Datasets - Provides a diverse set of conversational interactions for aligning and refining chat models.
- Language Model Fine-Tuning - Enables fine-tuning of language models with conversational datasets and acceleration tools.
- LLM Fine-Tuning Datasets - Provides diverse, multi-sector conversational interactions specifically designed for supervised fine-tuning to improve model reasoning.
- Supervised Fine-Tuning - Utilizes curated instruction and response pairs to improve the output quality of language models.
- Supervised Fine-Tuning Frameworks - Provides a pipeline for adapting pre-trained language models using curated dialogue datasets.
- Dialogue Dataset Structuring - Organizes conversational data into sequential multi-turn structures to train models on maintaining context.
- Large Language Model Fine-Tuning - Adapts large language models to specific tasks using conversational data and optimization tools.
- Large-Scale Model Training - Facilitates the training of chat models using large-scale, multi-round dialogue sequences.
- Model Evaluation Frameworks - Provides a framework for testing model coherence and reasoning across diverse topical datasets.
- Chat Model Evaluations - Offers structured tests to assess conversational ability and coherence using diverse sector datasets.
- Knowledge Domain Partitioning - Partitions training data into distinct industry and topical sectors to ensure broad knowledge coverage.
- Data Curation - Scales high-quality instructional conversations for chat model enhancement.
- Instruction Tuning - Scales high-quality instructional conversations for chat models.
- Large Language Models - Large-scale high-quality multi-turn dialogue instruction data.
- Natural Language Processing - Listed in the “Natural Language Processing” section of the FunNLP awesome list.
