InternLM/xtuner

Features

• Large Language Model Fine-Tuning - Provides a comprehensive engine for full-parameter and adapter-based fine-tuning of large language models.
• Large-Scale Model Training - Combine sequence parallelism and memory optimization to train models with billions of parameters.
• LLM Fine-Tuning Engines - Provides a specialized engine for the efficient pre-training and fine-tuning of large language models using full-parameter and adapter-based methods.
• Large Language Model Training Frameworks - Provides a comprehensive training engine for large language models with distributed launchers and memory optimization.
• Sequence Parallelism - Processes extended text datasets by distributing long sequences across hardware to overcome VRAM limits.
• Communication-Computation Overlap - Reduces GPU idle time by executing communication tasks concurrently with backward propagation and compute operations.
• Conversation Prompt Templating - Apply prompt formats to training data to ensure models follow specific conversational structures.
• Reward Modeling - Provides frameworks for training reward models and performing preference optimization to align model behavior.
• Direct Preference Optimization - Aligns model behavior with human preferences using the Direct Preference Optimization algorithm without requiring a separate reward model.
• Distributed Training - Executes distributed training jobs across single or multiple GPUs to accelerate model convergence.
• Distributed Training Accelerators - Implements sequence and interleaved pipeline parallelism to accelerate distributed deep learning workloads across multiple hardware units.
• Distributed Training - Parallelizes training using sequence, tensor, and data parallelism for ultra-large models.
• Distributed Training Scaling Utilities - Manages and scales training workloads across distributed GPU systems using standard launch commands.
• MoE Distributed Training - Trains Mixture-of-Experts models using optimized parallelism strategies to maintain high throughput.
• Group Relative Policy Optimization - Implements Group Relative Policy Optimization to refine model behavior by comparing sampled responses against a group average.
• Language Model Fine-Tuning - Provides frameworks for adjusting pre-trained language models to specialize behavior for specific tasks.
• Multimodal Model Trainers - Ships a training pipeline for vision-language models supporting supervised fine-tuning and instruction following.
• Multimodal Training Pipelines - Implements pre-training and supervised fine-tuning pipelines for vision-language models to enable instruction following.
• Distributed Training - Scales training workloads across multiple compute nodes and GPUs using cluster orchestrators.
• Training Hyperparameter Configurations - Define hyperparameters, learning rates, and distributed strategies to control how models converge.
• Interleaved Pipeline Parallelism - Increases throughput by using interleaved pipeline parallelism to reduce idle time and overlap communication.
• Multimodal Fine-Tuning - Optimizes models that process text and images simultaneously to improve cross-modal understanding.
• Vision Model Training - Performs pre-training and supervised fine-tuning for vision-language models to optimize instruction following.
• Model Parallelism - Splits model parameters across multiple GPUs using various parallelism strategies to fit large models.
• Group Relative Policy Optimization Training - Refines language model performance through Group Relative Policy Optimization with relative advantage estimates.
• Memory Optimizations - Prevents out-of-memory errors during large-scale training through memory redundancy reduction stages.
• Preference Optimization - Aligns models with human preferences using direct optimization methods to improve output quality.
• Group Relative Policy Optimization - Implements Group Relative Policy Optimization for refining model behavior through reinforcement learning.
• RLHF Training Pipelines - Provides end-to-end pipelines for model alignment using Direct Preference Optimization and Group Relative Policy Optimization.
• Sequence Parallelism Frameworks - Distributes a single long sequence across multiple GPUs to overcome memory limits for ultra-long context windows.
• Training Memory Management - Reduces GPU memory consumption during large-scale training via redundancy elimination and offloading techniques.
• Training Memory Optimizers - Manages memory and processing speed using redundancy reduction and offloading strategies.
• Attention Algorithm Optimizations - Increases training speed and reduces memory usage for transformer models by applying optimized attention algorithms.
• Variable-Length Attention Implementations - Implements variable-length attention to prevent tokens from attending to unrelated samples in concatenated datasets.
• Long-Context Sequence Parallelism - Distributes extended text lengths across multiple devices using sequence parallelism during preference optimization.
• Flexible Scoring Paradigms - Selects the highest-scoring result among multiple sampled responses using a reward model for quality assessment.
• Preference Pair Generators - Creates high-quality training pairs by using a reward model to filter data or synthesize responses for preference optimization.
• Dataset Preprocessing Tools - Converts preprocessed data in standardized directories into tokenized formats ready for model training.
• Distributed Model State Management - Reduces memory overhead by splitting optimizer states, gradients, and parameters across distributed devices.
• Reinforcement Learning Alignment - Optimizes model behavior using reinforcement learning workflows with colocated or disaggregated architectures.
• Domain-Specific Pre-training Corpora - Enables incremental pre-training using custom JSON-formatted text to adapt models to specific domains.
• RL - Specify loss calculations and importance sampling to refine weights during reinforcement learning.
• Conversational Fine-Tuning Formats - Supports training models using standardized JSON formats with multi-turn conversation structures.
• Context Window Scaling - Adjusts maximum position embeddings to expand the model's input and output context window.
• Expert Load Balancers - Enforces uniform distribution of tokens across experts in MoE models to prevent memory errors and maintain high throughput.
• Training Strategies - Scaling and training ultra-large MoE models using optimized parallelism strategies to maintain high compute throughput.
• Interactive Chat Evaluators - Allows launching interactive chat sessions to verify fine-tuned model responses using prompt templates.
• RL Post-Training - Optimizes model performance on reasoning tasks using reinforcement learning post-training techniques.
• Adapter Weight Merging - Combines learned adapter weights with base model parameters to create a single unified set of weights for deployment.
• Quantized Low-Rank Adapters - Reduces VRAM overhead by utilizing quantized low-rank adaptation (QLoRA) for memory-efficient fine-tuning.
• Variable-Length Attention - Processes sequences of different lengths without padding tokens to eliminate memory waste and increase throughput.
• Interleaved Pipeline Parallelism - Reduces GPU idle time by overlapping communication tasks with backward propagation and computation through interleaved pipeline parallelism.
• Preference Optimization Loss Functions - Compute objective functions for cross-entropy or reinforcement learning to guide model optimization.
• Preference Alignment Datasets - Formats sequences for reward model training by applying mapping functions and token truncation to preference data.
• Reference-Free Model Alignment - Refines model behavior using odds-ratio optimization to penalize rejected samples without requiring a reference model.
• Length-Based Feature Grouping - Sorts training samples by length to create batches of similar-sized sequences, minimizing padding overhead.
• Multimodal Dataset Formatting - Convert image and text conversation data into standardized JSONL files for supervised fine-tuning tasks.
• Training Throughput Optimization - Provides techniques to maximize training speed by optimizing batch sizes and GPU utilization.
• Transformer Fine-Tuning - Executes training for transformer architectures using Fully Sharded Data Parallelism for distributed model states.
• Model Evaluation Metrics - Scores model outputs using judging logic and evaluation metrics for reasoning and instruction following tasks.
• Interactive Chat Evaluators - Allows verification of fine-tuned model performance through interactive chat sessions using specific prompt templates.
• Model Weight Checkpoints - Saves and restores model weights and optimizer states to support automatic training resumption.
• Multimodal Pre-training - Trains vision-language models using pre-training and supervised fine-tuning optimized for instruction following.
• Long-Context Sequence Parallelism - Handles sequence lengths up to 64k using memory-efficient designs and sequence parallelism.
• Length-Based Batch Groupers - Sorts training data by length to create batches of similar-sized sequences and minimize padding overhead.
• Data Packing - Combine multiple short sequences into a single fixed-length input to eliminate padding and increase throughput.
• Dataset Tokenization Tools - Converts raw preprocessed dataset files into tokenized binary formats and handles the addition of special tokens.
• Model Memory Managers - Controls activation memory overhead using sequence parallelism and selective recomputation of specific modules.
• Sequence Packing - Reduce GPU memory waste by packing variable-length data into single sequences to eliminate excessive padding.
• Vision-Language Model Benchmarking - Measures accuracy and reasoning capabilities of vision-language models using standardized multimodal benchmarks.
• Fine-Tuning Frameworks - Flexible toolkit for fine-tuning large-scale models.
• Fine-Tuning Frameworks - Flexible toolkit for fine-tuning large-scale models.

Star history