yizhongw/self-instruct
Self Instruct
Self-instruct is a framework for generating synthetic instruction datasets and fine-tuning large language models to improve their instruction-following capabilities. It provides a pipeline for aligning pretrained models with human intentions through a supervised fine-tuning workflow.
The system utilizes a synthetic data generator that uses a seed set of tasks to prompt a model to create new instructional data. It includes an instruction dataset curator to remove redundant or low-quality entries, maintaining dataset diversity through a filtered task pool.
The framework covers the full alignment lifecycle, including synthetic dataset generation, instruction-based model fine-tuning using instruction-input-output triplets, and instruction-following evaluation to measure performance across diverse user tasks.
Features
- Synthetic Instruction Generators - Creates large collections of instructional tasks by prompting a model with seed examples and filtering them into a task pool.
- Alignment Pipelines - Implements a workflow for filtering synthetic data and training models to align outputs with human instructions.
- Instruction Fine-tuning - Improves model execution of user tasks by training on instruction-input-output triplets.
- Instruction Tuning - Trains language models to follow complex user directions using synthetic datasets of instruction triplets.
- Instruction Tuning Frameworks - Provides a full system for generating synthetic instruction datasets and fine-tuning LLMs for instruction following.
- Alignment Techniques - Aligns pretrained language models with human intentions using a synthetic data generation and supervised fine-tuning workflow.
- Supervised Instruction Fine-Tuning - Structures synthetic data into instruction-input-output triplets for supervised instruction fine-tuning.
- Supervised Fine-Tuning - Trains base models on filtered synthetic instruction sets to improve zero-shot task execution.
- Synthetic Dataset Generation - Creates large collections of instructional tasks by prompting a model to generate new examples from a seed set.
- Sample Generation - Generates diverse training data by sampling from a language model based on a seed set.
- Text Dataset Curators - Provides a process for filtering, formatting, and deduplicating synthetic instruction data to maintain diversity.
- Instruction Following Evaluations - Measures a model's ability to handle diverse user tasks across various styles and formats using representative instructions.
- Instructional Task Pools - Maintains a growing collection of generated instructions to serve as context for subsequent sampling phases.
- Instruction Quality Filtering - Removes low-quality or redundant entries from generated instruction sets to ensure diversity and accuracy.
- Instructional Data Filtering - Ensures training data remains diverse and accurate by removing low-quality or redundant synthetic entries.
- Instructional Redundancy Filtering - Implements heuristic-based filtering to remove redundant or low-quality synthetic entries using similarity metrics.
- Data Expansion - Aligns language models using self-generated instruction data.
- Instruction Tuning - Framework for aligning models using self-generated instructions.
- Instruction Tuning Datasets - Dataset generated using automated self-instruction methods.
- Self-Improvement Methods - Generates synthetic instructions to align models with user intent.
