hiyouga/ChatGLM-Efficient-TuningArchived
ChatGLM Efficient Tuning
ChatGLM-Efficient-Tuning is a fine-tuning framework and toolkit designed to optimize large language models using parameter-efficient fine-tuning techniques. It provides a pipeline for adjusting model behavior and reducing the memory and compute requirements necessary for training.
The project features a web-based trainer and orchestration interface for configuring and executing the fine-tuning process on a single GPU. It supports quantized training in lower precision formats to enable fine-tuning on hardware with limited memory, as well as reinforcement learning from human feedback for model alignment according to safety and quality standards.
The toolkit covers additional capabilities including adapter management and weight merging to reduce inference latency. It also provides tools for performance evaluation using linguistic metrics and the ability to expose fine-tuned models via a standard API.
Features
- LLM Fine-Tuning Toolsets - Ships a comprehensive toolset for supervised and parameter-efficient fine-tuning of large language models.
- Parameter Efficient Fine-Tuning - Implements parameter-efficient fine-tuning techniques to minimize memory and compute requirements.
- Large Language Model Fine-Tuning - Adapts pre-trained large language models to specific tasks using memory-efficient fine-tuning methods.
- Low-Rank Adaptation - Implements low-rank adaptation to reduce the number of trainable parameters during fine-tuning.
- Quantized Fine-Tuning - Enables training of large language models in lower precision formats to facilitate fine-tuning on limited hardware.
- Parameter-Efficient Training Toolkits - Provides a dedicated toolkit for implementing parameter-efficient fine-tuning and adapter-based learning.
- Quantized Fine-Tuning - Supports fine-tuning models using quantized base weights to minimize GPU memory consumption.
- Reinforcement Learning Alignment - Implements reinforcement learning alignment to adjust model behavior according to human safety and quality standards.
- RLHF Training Pipelines - Provides a training pipeline for aligning model responses with human preferences and safety standards using RLHF.
- RLHF Implementations - Provides a full-stack implementation of reinforcement learning from human feedback for model alignment.
- ChatGLM Adaptations - Provides specialized parameter-efficient fine-tuning tools specifically optimized for ChatGLM models.
- LLM Training Orchestrators - Offers a web-based orchestration interface for configuring and executing LLM fine-tuning processes on a GPU.
- Preference-Based Model Alignments - Uses reward models and policy optimization to tune model responses for safety and helpfulness.
- Fine-Tuned Model Evaluators - Provides tools to evaluate the performance of fine-tuned models using linguistic metrics like BLEU and ROUGE.
- Human Preference Alignment - Implements reinforcement learning from human feedback to align model outputs with safety standards.
- Adapter Weight Merging - Provides utilities to merge adapter weights back into the base model to remove inference latency.
- Mixed-Precision Quantization - Supports reducing the bit-width of model weights to enable training on hardware with limited memory.
- Browser-Based Model Trainers - Includes a browser-based trainer for managing and executing the LLM fine-tuning lifecycle.
- Weight Merging - Combines weights from multiple adapters to support continual learning or high-performance ensembles.
- Training Job Orchestrators - Offers a web interface for monitoring and controlling training jobs on GPU hardware.
- CLI and Web GUI Operation Interfaces - Provides a graphical web interface alongside CLI for controlling and monitoring the fine-tuning process.
- Web Orchestration Interfaces - Features a browser-based interface for configuring and executing training processes on the GPU backend.
- LLM Frameworks - Efficient fine-tuning framework based on PEFT for ChatGLM.
- Natural Language Processing - Listed in the “Natural Language Processing” section of the FunNLP awesome list.
- Training Frameworks - Efficient fine-tuning framework specifically for ChatGLM.
