How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.
This project is a collection of scripts and workflows for training, fine-tuning, and deploying large language models using the Hugging Face Transformers toolkit. It functions as a distributed training framework, a library for natural language processing task implementations, and a system for building retrieval-augmented generation chatbots. The repository includes specialized tools for model optimization, such as a Bayesian hyperparameter optimizer for automatically tuning model settings. It provides implementations for scaling model training across multiple graphics processors using data par
Firefly is a training framework and inference engine for large language models. It functions as a toolkit for pre-training and fine-tuning various open-weight architectures, providing a system for model alignment and parameter-efficient fine-tuning. The project includes utilities for merging adapter weights back into base models to create standalone files. It also provides a model alignment toolkit to format training data according to specific prompt templates, ensuring conversational consistency across different models. The framework supports distributed model training and preference-based
zero_nlp is a distributed framework for training and fine-tuning large language models and multimodal architectures. It provides a specialized toolkit for distributed model parallelism, allowing neural network layers and weights to be partitioned across multiple GPU devices to train models that exceed the memory capacity of a single processor. The project distinguishes itself through a combination of high-throughput data pipelines and parameter-efficient tuning. It utilizes multi-threading and memory mapping to preprocess and stream datasets exceeding 100GB and implements memory-saving adapta
LitGPT is a training and deployment framework for large language models, providing a suite of tools for pretraining, finetuning, quantizing, evaluating, and serving models within a production environment. It includes a dedicated training pipeline for adapting pretrained models to specific tasks, a quantization tool for reducing weight precision, and an inference server for hosting models via web interfaces. The framework supports high-performance model development through custom architecture implementation and the use of predefined recipes to standardize pretraining and finetuning. It enables
The main features of allenai/olmo are: Large Language Model Training Frameworks, Checkpointing Systems, Text Generation Inference Integrations, Pretrained Checkpoint Inference, Two-Stage Language Model Training Frameworks, Language Model Training, 8-Bit Inference Quantizers, 8-Bit Load-Time Quantizers.
Projects with overlapping indexed features include: zyds/transformers-code — This project is a collection of scripts and workflows for training, fine-tuning, and deploying large language models… yangjianxin1/firefly — Firefly is a training framework and inference engine for large language models. It functions as a toolkit for… yuanzhoulvpi2017/zero_nlp — zero_nlp is a distributed framework for training and fine-tuning large language models and multimodal architectures.… lightning-ai/litgpt — LitGPT is a training and deployment framework for large language models, providing a suite of tools for pretraining,… facebookresearch/metaseq — Metaseq is a transformer sequence modeling toolkit designed for training, fine-tuning, and deploying… google/flax — Flax is a deep learning framework and JAX neural network library designed for building complex machine learning…