Swift

Features

Parameter Efficient Fine-Tuning - Provides a comprehensive suite of parameter-efficient fine-tuning methods, including adapters and low-rank approximations.
Distributed Training Accelerators - Accelerates training for large models by distributing workloads across multiple processors using advanced parallelism.
Distributed Training - Scales the training of large models across multiple processors via data and model parallelism.
LLM Fine-Tuning - Serves as a full-featured toolkit for both full-parameter and parameter-efficient fine-tuning of LLMs and multimodal models.
Multimodal Model Trainers - Functions as a training system for models processing mixed modalities including text, image, video, and audio.
Large Language Model Fine-Tuning - Adapts large language and multimodal models to specific tasks using flexible training methods.
Preference-Based Model Alignments - Refines model behavior using preference-based alignment algorithms like DPO and GRPO.
Reinforcement Learning Integrations - Integrates reinforcement learning from human feedback and extensible reward functions to refine model intelligence.
Preference Alignment - Improves model behavior and alignment with human values using preference learning algorithms.
Model Parallelism - Implements data, pipeline, and tensor parallelism to distribute massive model weights and computation across multiple GPUs.
Alignment Toolkits - Offers a dedicated toolkit for optimizing model behavior via RLHF and algorithms like DPO and GRPO.
Multimodal Training - Provides specialized workflows and data packing for training models across text, image, video, and audio modalities.
Weight Quantization - Reduces model memory footprint and hardware requirements through weight quantization.
Attention Memory Optimizations - Manages attention mechanisms and memory allocation to support long-text inputs without exceeding video memory.
Model Compression Suites - Provides utilities for reducing the size and hardware requirements of large models via quantization and compression.
LLM Performance Evaluators - Includes integrated evaluation modules to measure the accuracy and reliability of large language models.
Specialized Model Training - Implements specialized training workflows for creating high-performance embedding models, rerankers, and sequence classifiers.
Data Packing - Optimizes multimodal training throughput by packing diverse data types into sequences to prevent padding waste.
Training Memory Optimizers - Optimizes attention and sequence data handling to reduce video memory consumption during long-text training.
Fine-Tuning Frameworks - PEFT and full-parameter fine-tuning for diverse models.
Fine-Tuning Frameworks - Framework for PEFT and full-parameter fine-tuning.
Training Frameworks - Lightweight framework for model fine-tuning and deployment.

Open-source alternatives to Swift

Similar open-source projects, ranked by how many features they share with Swift.

internlm/xtuner
InternLM/xtuner
5,150View on GitHub
xtuner is a comprehensive training engine for large language models, offering a toolkit for pre-training, supervised fine-tuning, and the optimization of vision-language multimodal models. It serves as a distributed training accelerator and a specialized framework for scaling Mixture-of-Experts models and aligning model behavior through reinforcement learning from human feedback. The project distinguishes itself through advanced memory and compute optimizations, such as sequence parallelism for ultra-long context windows and interleaved pipeline parallelism to reduce GPU idle time. It provide
Pythonagentdeepseek-v3gpt-oss
View on GitHub5,150
pytorch/torchtune
pytorch/torchtune
5,774View on GitHub
Torchtune is a PyTorch-native library for fine-tuning, aligning, and quantizing large language models. It provides a configurable training pipeline orchestrated through YAML recipes, with CLI overrides and component swapping, distributed training via FSDP2, memory optimizations, and parameter-efficient fine-tuning methods like LoRA, DoRA, and QLoRA. The library distinguishes itself through its YAML-driven configuration system that defines all training parameters and instantiates components from config files, with full CLI override capability for any field or component at launch time. It suppo
Python
View on GitHub5,774
yangjianxin1/firefly
yangjianxin1/Firefly
6,642View on GitHub
Firefly is a training framework and inference engine for large language models. It functions as a toolkit for pre-training and fine-tuning various open-weight architectures, providing a system for model alignment and parameter-efficient fine-tuning. The project includes utilities for merging adapter weights back into base models to create standalone files. It also provides a model alignment toolkit to format training data according to specific prompt templates, ensuring conversational consistency across different models. The framework supports distributed model training and preference-based
Pythonalpacaaquilabaichuan
View on GitHub6,642
paddlepaddle/paddlenlp
PaddlePaddle/PaddleNLP
12,953View on GitHub
PaddleNLP is a development library and toolkit for training, fine-tuning, and deploying large and small language models using the PaddlePaddle framework. It provides a comprehensive suite for the entire natural language processing lifecycle, from model development to high-performance inference. The project features a standardized model zoo for loading and managing pre-trained models and tokenizers through a unified interface. It distinguishes itself with a specialized model compression framework that reduces memory footprints via weight precision conversion and lossless size optimization, alo
Python
View on GitHub12,953

See all 30 alternatives to Swift

modelscopeswift

Features

Open-source alternatives to Swift

InternLM/xtuner

pytorch/torchtune

yangjianxin1/Firefly

PaddlePaddle/PaddleNLP

Star history

Open-source alternatives to Swift

InternLM/xtuner

pytorch/torchtune

yangjianxin1/Firefly

PaddlePaddle/PaddleNLP