pytorch/torchtune

Features

• LLM Fine-Tuning - Ships a complete framework for supervised fine-tuning, preference alignment, and knowledge distillation of large language models.
• Distributed Training - Distributes training jobs across several GPUs or nodes to handle larger models or speed up computation.
• Distributed Training Orchestration - Scales model training across multiple GPUs and nodes using PyTorch distributed backends and SLURM.
• Distributed Training Orchestrators - Provides a distributed training orchestrator using PyTorch FSDP2 and torchrun for multi-GPU and multi-node scaling.
• Distributed Training Sharding - Scales training across multiple GPUs using PyTorch FSDP2 and torchrun for process management.
• Model Distillation - Produces compact student models with competitive performance via teacher-student knowledge transfer.
• End-to-End Training Pipelines - Orchestrates the full lifecycle of training, evaluating, quantizing, and generating with a language model.
• LoRA Fine-Tuning Tools - Applies low-rank adapters to a subset of model weights to reduce memory and compute.
• Multi-Node Training Scaling - Distributes large-scale model training across multiple hardware nodes to manage GPU resources and communication.
• Knowledge Distillation - Transfers capabilities from large teacher models to smaller student models for improved efficiency.
• Knowledge Distillation into Skills - Transfers knowledge from larger to smaller models to create compact versions retaining original capability.
• Language Model Fine-Tuning - Applies supervised fine-tuning, knowledge distillation, or reinforcement learning to adapt pretrained models.
• LoRA Fine-Tuning Pipelines - Applies low-rank adapters to a base model to reduce memory use while preserving performance.
• Knowledge Distillation Frameworks - Executes pre-built training recipes combining cross-entropy and KL divergence for single-GPU distillation.
• LoRA Configuration Customization - Selects which attention modules receive LoRA adapters and sets rank and alpha via CLI arguments.
• Distributed Training - Scales fine-tuning across multiple GPUs or nodes using PyTorch FSDP2 and torchrun integration.
• Training Config Customizers - Sets training parameters by editing a YAML config file or overriding fields from the command line.
• Fine-tuning Recipes - Launches pre-built, hackable training scripts for single-device or distributed fine-tuning with memory-efficient techniques.
• Preference Alignment - Fine-tunes models using DPO-style losses to steer outputs toward desirable behaviors like safety and helpfulness.
• Human Preference Alignment - Uses reinforcement learning methods like DPO, PPO, or GRPO to adjust model outputs to match human preferences.
• QLoRA Adapters - Combines 4-bit quantization with low-rank adapters to further cut memory consumption during training.
• Training and Evaluation Pipelines - Defines training, evaluation, and inference parameters through YAML configs with CLI overrides and component swapping.
• Memory Optimization Techniques - Reduces peak GPU memory through activation checkpointing, gradient accumulation, and offloading.
• Teacher-Student Distillation - Transfers knowledge from larger teacher models to smaller student models via token-level probability imitation.
• Model Quantization - Compresses model weights into lower-precision formats to shrink memory footprint and speed up inference.
• Parameter Efficient Fine-Tuning - Updates only a small subset of model weights via LoRA, DoRA, or QLoRA to reduce memory and compute.
• PyTorch Training Frameworks - Delivers a PyTorch-native library with configurable YAML recipes for fine-tuning, aligning, and quantizing large language models.
• Weight Quantization - Reduces numerical precision of model weights to lower memory usage, optionally during training with QLoRA or QAT.
• Post-Training Quantization - Compresses trained model weights to reduce memory footprint and accelerate inference on limited hardware.
• Recipe Configuration Definitions - Defines all parameters for a recipe in a YAML file as the primary entry point for execution.
• YAML-Driven Recipe Configurations - Defines training runs through YAML files that specify all parameters and components as the primary entry point.
• Training Recipes - Executes pre-built, hackable training scripts that orchestrate fine-tuning and evaluation workflows.
• Preference Alignment - Aligns language model outputs with human preferences using DPO, PPO, or GRPO algorithms on preference datasets.
• Component Swapping Overrides - Provides CLI-based component swapping via tilde flags to replace entire config components at launch.
• Config-Field Override Systems - Passes key-value pairs on the command line to change any configuration value at launch time.
• Training Config Override Systems - Enables modifying any training config field at launch via command-line key-value pairs.
• Config-Driven Instantiation - Instantiates Python objects like models and datasets by resolving import paths from YAML config.
• Low-Rank Adaptation - Implements LoRA, a core parameter-efficient fine-tuning method using low-rank matrix decomposition.
• Activation Recomputation Strategies - Recomputes activations during the backward pass instead of storing them to save memory.
• Cross-Sample Attention Masks - Prevents samples packed into the same sequence from attending to each other using document masking.
• Sequence Packing - Packs multiple training samples into single sequences to minimize padding waste and maximize throughput.
• Config-Driven Dataset Builders - Builds custom chat, instruct, or preference datasets declaratively from YAML configurations.
• Direct Preference Optimization - Applies Direct Preference Optimization or Proximal Policy Optimization to align model outputs with human preferences.
• Custom Dataset Integration Methods - Feeds custom instruct, chat, or preference data into training recipes for domain-specific fine-tuning.
• Chat and Instruct Dataset Integrations - Integrates custom instruct, chat, and preference datasets into training pipelines.
• Single-GPU Training - Executes a training recipe on one GPU or CPU, supporting full fine-tuning and parameter-efficient methods.
• Instruction Datasets - Loads instruction-following datasets from local files or Hugging Face and tokenizes them for fine-tuning.
• Prompt Templates - Prepends task-specific system prompts and structures each sample before tokenization during instruction tuning.
• Activation and KV Cache Offloaders - Moves activations to CPU during forward pass and retrieves them for backward to reduce GPU memory.
• Dialogue-Based Fine-Tuning - Loads multi-turn conversational datasets and tokenizes them for supervised fine-tuning.
• LLM Inference Wrappers - Executes trained models to generate predictions or responses using a dedicated recipe script for LLM inference.
• Quantization-Aware Training - Reduces model precision through post-training quantization or quantization-aware training to shrink memory and accelerate inference.
• Classification Fine-Tuning - Applies weight-decomposed low-rank adaptation that separates magnitude and direction for more expressive fine-tuning.
• Fine-Tuned Model Evaluators - Runs structured evaluations using an integrated harness to measure model performance on standard benchmarks.
• Text Generation Interfaces - Produces text completions from fine-tuned models using configurable sampling parameters like temperature and top-k.
• Preference-Based Fine-Tuning - Loads chosen and rejected response pairs and tokenizes them for Direct Preference Optimization.
• Quantized Fine-Tuning - Stores frozen base model weights in 4-bit precision during LoRA fine-tuning to reduce memory usage.
• Distributed Preference Optimizations - Enables distributed DPO training across multiple GPUs with LoRA or full fine-tuning for scalable preference alignment.
• Post-Fine-Tuning Quantizers - Converts fine-tuned models from float precision into quantized weights for reduced memory and faster inference.
• Quantization-Aware Fine-Tuning - Simulates quantization noise during fine-tuning so weights adapt to lower precision before final conversion.
• Quantization Toolkits - Provides a quantization toolkit for reducing model memory footprint and accelerating inference through post-training and quantization-aware training.
• Cross-Family Distillation - Supports distillation between teacher and student models from different model families.
• Distillation Hyperparameter Tuning - Adjusts learning rate and loss ratio to balance teacher imitation against ground-truth learning.
• Teacher-Student Model Swapping - Allows swapping teacher and student models from different families in distillation recipes.
• Mixed-Precision Quantization - Trains models in bfloat16 half-precision to halve memory usage per parameter and speed up computation.
• LLM Performance Evaluators - Runs inference on trained models to measure performance on specific tasks or datasets using dedicated recipe scripts.
• Quantization-Aware Training - Simulates quantization effects during training so the final model maintains accuracy with reduced precision.
• Weights and Biases Training Logging - Logs training metrics, configurations, and model checkpoints to Weights & Biases and Comet for monitoring.
• GPU - Applies activation checkpointing, offloading, and low-bit optimizers to reduce GPU memory during training.
• Preference-Aligned Adapter Training - Combines parameter-efficient fine-tuning with direct preference optimization to align models with reduced memory requirements.
• Weight-Decomposed Low-Rank Adaptation - Applies weight-decomposed low-rank adaptation that separates magnitude and direction for more expressive fine-tuning.
• JSON Preference Dataset Loaders - Reads local JSON files with chosen and rejected conversation pairs for preference fine-tuning.
• Quantized Model Exporters - Provides a dedicated export step to convert QAT-fine-tuned checkpoints into fully quantized model formats.
• Gradient Accumulation Strategies - Simulates a larger batch size by summing gradients over several forward passes before each optimizer update.
• Training Progress Monitors - Logs loss, GPU memory usage, and other metrics during a fine-tuning run, displayed via a progress bar.
• Pre-Distillation Teacher Fine-Tuning - Fine-tunes teacher models on target datasets before distillation to improve knowledge transfer quality.
• Text Dataset Loaders - Loads text-only datasets from Hugging Face for fine-tuning, supporting formats like Alpaca and summarization.
• Training Memory Optimizers - Applies activation checkpointing, offloading, and low-bit optimizers to fit larger models on limited hardware.
• Training Speed Optimizations - Combines packing, compilation, and activation checkpointing to reduce memory and increase training throughput.
• Config Field Removers - Deletes specific configuration parameters at launch time using a tilde flag for component swapping.
• Dataset Loaders - Loads chat datasets directly from Hugging Face repositories for training.
• Preference Dataset Loaders - Fetches preference datasets from Hugging Face repositories for alignment training.
• PyTorch JIT Compilation - Provides torch.compile integration for JIT-compiling PyTorch operations into optimized kernels.
• Chat Dataset Loaders - Loads conversational data from local files or remote URLs for fine-tuning recipes.
• Config Field Interpolations - Reuses values across YAML config fields using interpolation syntax to avoid duplication.
• LLM Evaluation - Measures model performance on a task using a dedicated recipe script for LLM evaluation.
• Fine-Tuning Frameworks - Native PyTorch library for model fine-tuning.
• Model Training - Native PyTorch library designed for fine-tuning language models.
• Model Training and Fine-tuning - PyTorch-native library for fine-tuning LLMs.
• Model Training Frameworks - Native PyTorch library specifically for LLM fine-tuning.
• Fine-Tuning Frameworks - Native PyTorch library for model fine-tuning.

سجل النجوم