huggingface/nanotron

Features

• LLM Development Tools - Minimalistic codebase for training large language models.
• Model Training - Framework for 3D-parallelism training of large models.
• Model Training Frameworks - Minimalistic library for 3D-parallelism training.
• Pre Training Models - Listed in the “Pre Training Models” section of the Llm Course awesome list.
• Training and Orchestration - Distributed primitives for efficient large-scale model training.