yizhongw/self-instruct
Self Instruct
Self-instruct este un framework pentru generarea de seturi de date sintetice de instrucțiuni și fine-tuning-ul modelelor de limbaj mari pentru a le îmbunătăți capabilitățile de urmare a instrucțiunilor. Oferă un pipeline pentru alinierea modelelor pre-antrenate cu intențiile umane printr-un flux de lucru de fine-tuning supervizat.
Sistemul utilizează un generator de date sintetice care folosește un set de sarcini de bază (seed) pentru a determina un model să creeze noi date instrucționale. Include un curator de seturi de date de instrucțiuni pentru a elimina intrările redundante sau de calitate scăzută, menținând diversitatea setului de date printr-un pool de sarcini filtrat.
Framework-ul acoperă întregul ciclu de viață al alinierii, inclusiv generarea setului de date sintetice, fine-tuning-ul modelului bazat pe instrucțiuni folosind triplete instrucțiune-input-output și evaluarea urmării instrucțiunilor pentru a măsura performanța în diverse sarcini ale utilizatorului.
Features
- Synthetic Instruction Generators - Creates large collections of instructional tasks by prompting a model with seed examples and filtering them into a task pool.
- Alignment Pipelines - Implements a workflow for filtering synthetic data and training models to align outputs with human instructions.
- Instruction Fine-tuning - Improves model execution of user tasks by training on instruction-input-output triplets.
- Instruction Tuning - Trains language models to follow complex user directions using synthetic datasets of instruction triplets.
- Instruction Tuning Frameworks - Provides a full system for generating synthetic instruction datasets and fine-tuning LLMs for instruction following.
- Alignment Techniques - Aligns pretrained language models with human intentions using a synthetic data generation and supervised fine-tuning workflow.
- Supervised Instruction Fine-Tuning - Structures synthetic data into instruction-input-output triplets for supervised instruction fine-tuning.
- Supervised Fine-Tuning - Trains base models on filtered synthetic instruction sets to improve zero-shot task execution.
- Synthetic Dataset Generation - Creates large collections of instructional tasks by prompting a model to generate new examples from a seed set.
- Sample Generation - Generates diverse training data by sampling from a language model based on a seed set.
- Text Dataset Curators - Provides a process for filtering, formatting, and deduplicating synthetic instruction data to maintain diversity.
- Instruction Following Evaluations - Measures a model's ability to handle diverse user tasks across various styles and formats using representative instructions.
- Instructional Task Pools - Maintains a growing collection of generated instructions to serve as context for subsequent sampling phases.
- Instruction Quality Filtering - Removes low-quality or redundant entries from generated instruction sets to ensure diversity and accuracy.
- Instructional Data Filtering - Ensures training data remains diverse and accurate by removing low-quality or redundant synthetic entries.
- Instructional Redundancy Filtering - Implements heuristic-based filtering to remove redundant or low-quality synthetic entries using similarity metrics.
- Data Expansion - Aligns language models using self-generated instruction data.
- Instruction Tuning - Framework for aligning models using self-generated instructions.
- Instruction Tuning Datasets - Dataset generated using automated self-instruction methods.
- Self-Improvement Methods - Generates synthetic instructions to align models with user intent.
