Awesome GitHub RepositoriesSequence Packing

OpenRLHF is a training framework and alignment library designed for reinforcement learning from human feedback across distributed GPU clusters. It provides tools for aligning large language models and multimodal vision-language models using algorithms such as PPO, GRPO, and DPO. The framework distinguishes itself through a distributed inference engine that overlaps sample rollout with training to increase throughput. It supports scaling to models exceeding 70 billion parameters via parameter sharding and handles long-context sequences through ring-attention sequence parallelism. The project

ERNIE is a development toolkit for training, fine-tuning, and deploying large language models built on the PaddlePaddle deep learning platform. It provides a comprehensive suite of core components, including an inference server for vision and language models, a training and fine-tuning toolkit, and a framework for building retrieval-augmented generation systems using private knowledge bases. The project features multimodal AI models capable of reasoning across text, images, and video to perform complex visual understanding and information extraction. It distinguishes itself through specialize

Torchtune is a PyTorch-native library for fine-tuning, aligning, and quantizing large language models. It provides a config-driven system for instantiating components, orchestrating distributed training, and managing parameter-efficient fine-tuning with quantization support, all through YAML-based configurations and command-line overrides. The library distinguishes itself through its comprehensive post-training workflow orchestration, combining supervised fine-tuning, preference optimization (DPO, PPO, GRPO), knowledge distillation, and quantization-aware training in a single configurable pip

Torchtune is a PyTorch-native library for fine-tuning, aligning, and quantizing large language models. It provides a configurable training pipeline orchestrated through YAML recipes, with CLI overrides and component swapping, distributed training via FSDP2, memory optimizations, and parameter-efficient fine-tuning methods like LoRA, DoRA, and QLoRA. The library distinguishes itself through its YAML-driven configuration system that defines all training parameters and instantiates components from config files, with full CLI override capability for any field or component at launch time. It suppo

This project provides a comprehensive technical guide and framework for engineering large-scale machine learning systems. It covers the full lifecycle of model development, focusing on the infrastructure and computational principles required to build, train, and serve generative AI models across distributed GPU clusters. The repository distinguishes itself by offering deep-dive tutorials and implementation strategies for complex system challenges. It emphasizes high-performance architectural primitives, such as collective communication orchestration, distributed tensor sharding, and static gr

EasyR1 es un sistema de entrenamiento distribuido y framework de aprendizaje por refuerzo para modelos de lenguaje y visión-lenguaje de gran escala. Funciona como un entrenador multimodal y una implementación de un pipeline de Proximal Policy Optimization diseñado para refinar las capacidades de razonamiento y percepción de modelos que procesan tanto texto como imágenes. El sistema se especializa en distribuir cargas de trabajo de aprendizaje por refuerzo a través de múltiples nodos de cómputo para gestionar altos requisitos de memoria. Optimiza el uso del hardware mediante entrenamiento sin padding y fine-tuning para ajustar modelos grandes en las unidades de procesamiento gráfico (GPU) disponibles. El framework cubre el aprendizaje por refuerzo y la orquestación de modelos de recompensa, incluyendo flujos de trabajo de aprendizaje por refuerzo a partir de retroalimentación humana (RLHF). Su superficie técnica incluye paralelismo de datos distribuido, entrenamiento de precisión híbrida y pipelines de entrada multimodal para datos intercalados de texto e imagen. El proyecto incluye utilidades para la recuperación de estado basada en checkpoints y se integra con herramientas de registro externas para rastrear el progreso del entrenamiento y las métricas de rendimiento.