How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.
SmolLM is a project dedicated to the development of small language models. It focuses on training and fine-tuning compact models that maintain high performance while utilizing fewer parameters. The project emphasizes efficient AI inference and on-device text generation, aiming to enable the deployment of lightweight models on edge devices with limited memory and processing power. It utilizes synthetic data generation to produce artificial datasets that improve the reasoning and training of these AI systems. The system supports a variety of optimization and training capabilities, including we
Torchtune is a PyTorch-native library for fine-tuning, aligning, and quantizing large language models. It provides a config-driven system for instantiating components, orchestrating distributed training, and managing parameter-efficient fine-tuning with quantization support, all through YAML-based configurations and command-line overrides. The library distinguishes itself through its comprehensive post-training workflow orchestration, combining supervised fine-tuning, preference optimization (DPO, PPO, GRPO), knowledge distillation, and quantization-aware training in a single configurable pip
This project is a collection of scripts and workflows for training, fine-tuning, and deploying large language models using the Hugging Face Transformers toolkit. It functions as a distributed training framework, a library for natural language processing task implementations, and a system for building retrieval-augmented generation chatbots. The repository includes specialized tools for model optimization, such as a Bayesian hyperparameter optimizer for automatically tuning model settings. It provides implementations for scaling model training across multiple graphics processors using data par
This is an educational implementation that builds a generative pre-trained transformer (GPT) language model from scratch using PyTorch. The project is structured as a step-by-step tutorial, walking through the construction of a decoder-only transformer architecture and its training loop with clean git commits and an accompanying video lecture for a hands-on learning experience. What sets this implementation apart is its focus on practical reproduction: it provides a workflow to train a 124-million-parameter model from scratch in about one hour on cloud GPU hardware, costing under ten dollars.
GuppyLM is a small educational project that trains a compact language model from scratch and gives it the voice of a fish. It covers the whole path end to end: generating a synthetic conversation dataset, training a tokenizer, training a decoder-only transformer, and chatting with the result. The aim is to make the full lifecycle of a language model easy to follow.
The main features of arman-bd/guppylm are: AI Chatbots, Language Model Training, Chat Model Interactions, From-Scratch Decoder Implementations, In-Browser Model Inference, Small Language Models, Template-Based Generation, Synthetic Dataset Generators.
Projects with overlapping indexed features include: huggingface/smollm — SmolLM is a project dedicated to the development of small language models. It focuses on training and fine-tuning… meta-pytorch/torchtune — Torchtune is a PyTorch-native library for fine-tuning, aligning, and quantizing large language models. It provides a… zyds/transformers-code — This project is a collection of scripts and workflows for training, fine-tuning, and deploying large language models… karpathy/build-nanogpt — This is an educational implementation that builds a generative pre-trained transformer (GPT) language model from… intel-analytics/ipex-llm — ipex-llm is an acceleration library and inference engine designed to optimize the execution and finetuning of large… ml-explore/mlx-examples — This repository provides a collection of reference implementations and code examples for training and deploying…