huggingface/peft
Peft
This library provides a framework for parameter-efficient fine-tuning, enabling the adaptation of large pretrained models by training only a small subset of parameters. It functions as a distributed model training system and optimization toolkit, designed to reduce the computational and memory requirements typically associated with full model fine-tuning.
The project distinguishes itself through a suite of methods for modular adapter composition, including low-rank matrix decomposition and activation-based scaling. It supports the integration of multiple task-specific adapter modules, allowing users to merge, route, and combine these components into base model architectures. To ensure efficient inference, the library provides capabilities to integrate trained adapter weights directly into the original model.
The framework includes extensive support for memory-optimized training, utilizing techniques such as parameter offloading to system memory, low-bit quantization, and distributed parameter sharding across multiple hardware devices. These features allow for the training of massive models that exceed the memory capacity of individual graphics processing units. The library is distributed as a Python package and includes command-line tools for managing training tasks and authentication.
Features
- Parameter Efficient Fine-Tuning - Provides a library for parameter-efficient fine-tuning of large pretrained models.
- Large Language Model Fine-Tuning Frameworks - Provides a framework for adapting large pretrained models to downstream tasks through parameter-efficient fine-tuning.
- Large Language Model Optimization - Provides a toolkit for optimizing large language models via weight decomposition and activation scaling.
- Parameter Adaptation Techniques - Decomposes large model matrices into smaller low-rank matrices to enable fine-tuning with minimal trainable parameters.
- Distributed Training Frameworks - Functions as a distributed training framework designed for memory-efficient sharding and parameter offloading.
- Model Adapters - Enables modular adapter composition by merging and routing specialized adapter modules into base models.
- Low-Rank Decompositions - Reduces trainable parameter counts by representing large weight updates as the product of two significantly smaller matrices.
- Activation-Based Prompt Tuning - Implements activation-based tuning to adapt model behavior without modifying base weights.
- Distributed Training - Scales model training across multiple hardware devices using distributed parameter sharding.
- Distributed Training Sharding - Enables training of massive models by partitioning parameters and optimizer states across multiple hardware devices.
- Distributed Training Utilities - Splits model parameters and gradients across multiple devices to train massive models exceeding single-GPU memory.
- Weight Merging Utilities - Integrates trained adapter weights directly into the original model to eliminate inference latency.
- Memory Optimization Techniques - Implements memory offloading techniques to move model parameters and optimizer states to system memory during training.
- Weight Offloading - Moves sharded model parameters to system memory to reduce graphics memory consumption during training.
- Model Weight Reconstruction - Integrates trained adapter matrices directly into base model weights to create standalone models for efficient inference.
- Parameter-Efficient Adaptation - Integrates task-specific adapter modules into base architectures using parameter-efficient adaptation methods.
- Training Memory Management - Optimizes memory usage during training through parameter offloading and low-bit quantization.
- Frozen Base Models - Supports fine-tuning by keeping the primary model parameters frozen while training lightweight adapter layers.
- Activation Scaling Adapters - Trains learned vectors to rescale internal model activations for task-specific adaptation.
- Model Quantization - Supports low-bit quantization of model weights during fine-tuning to reduce memory footprint.
- Orthogonal Adaptation Strategies - Applies multiplicative orthogonal weight updates to frozen models to adapt them to new tasks.
- Application Development Tools - Library for parameter-efficient fine-tuning of large language models.
- Large Language Models - Library for parameter-efficient fine-tuning of models.
- Machine Learning - Library for adapting large pretrained models.
- Machine Learning Libraries - Parameter-efficient fine-tuning for large models.
- Model Fine Tuning - Implements state-of-the-art parameter-efficient fine-tuning techniques.
- Model Optimization and Deployment - Parameter-efficient methods for fine-tuning large pre-trained models.
- Model Training - Library for parameter-efficient fine-tuning of large models.
- Model Training and Fine-tuning - Parameter-efficient fine-tuning library.
- Computation and Optimization - Methods for parameter-efficient fine-tuning of pre-trained models.
- Large Language Models (LLMs) - Listed in the “Large Language Models (LLMs)” section of the The Incredible Pytorch awesome list.
- Weight Initialization - Uses gradient-based singular value decomposition to initialize adapter weights for faster convergence.
- Gradient-Based Initializers - Accelerates convergence by initializing adapter weights using gradient-based singular value decomposition.
- Adapter Routing Strategies - Routes and merges multiple trained low-rank modules to solve new tasks.
- Adapter Management - Combines several trained adapter modules using weighted averages to create unified adapter configurations.
- Adapter Dimension Suppressors - Identifies and suppresses specific adapter dimensions that cause catastrophic forgetting to recover original model knowledge.
