awesome-repositories.com
Blog
awesome-repositories.com

Descoperă cele mai bune repository-uri open source cu căutare AI.

ExploreazăCăutări recomandateAlternative open-sourceSoftware self-hostedBlogHartă site
ProiectDespreCum realizăm clasamentulPresăServer MCP
LegalConfidențialitateTermeni
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
·
huggingface avatar

huggingface/alignment-handbook

0
View on GitHub↗
5,621 stele·492 fork-uri·Python·Apache-2.0·11 vizualizărihuggingface.co/HuggingFaceH4↗

Alignment Handbook

Acest proiect este un framework de aliniere și o suită de pipeline-uri pentru antrenarea modelelor de limbaj folosind fine-tuning supervizat și optimizarea preferințelor. Oferă instrumente pentru executarea antrenării distribuite la scară largă pe mai multe GPU-uri și noduri de calcul, alături de un sistem pentru măsurarea utilității modelului și a calității dialogului prin benchmark-uri single-turn și multi-turn.

Framework-ul include instrumente specializate pentru optimizarea directă a preferințelor (DPO) pentru a rafina comportamentul modelului folosind date pereche, fără a fi nevoie de un model de recompensă separat. De asemenea, suportă alinierea prin AI constituțional și antrenarea modelelor de recompensă pentru a clasifica și puncta răspunsurile pe baza criteriilor de preferință.

Proiectul acoperă capabilități mai largi pentru amestecarea seturilor de date, fine-tuning eficient din punct de vedere al parametrilor prin adaptare low-rank (LoRA) și optimizarea prin eșantionare de respingere. Gestionează ciclul de viață al antrenării prin rețete bazate pe configurație și oferă sisteme pentru streaming-ul metricilor de performanță în timp real către dashboard-uri externe.

Features

  • Preference Alignment - Trains language models to follow human or AI preferences through supervised fine-tuning and preference optimization.
  • Reward Modeling - Provides capabilities for training reward models to distinguish and score different model responses.
  • Preference Alignment - Refines model behavior using human and AI feedback to ensure outputs match specific preference criteria.
  • Direct Preference Optimization - Refines model behavior using paired preference data to align outputs without a separate reward model.
  • Distributed Training Configurations - Includes configurations for scaling model training workloads across multiple GPUs and compute nodes.
  • Large Scale Training Suites - Provides comprehensive frameworks for orchestrating distributed multi-node training of large language models.
  • Distributed Training - Scales large-scale model training across multiple GPUs and compute nodes using distributed orchestration.
  • Alignment Frameworks - Provides a full suite of tools and recipes for training and aligning language models using SFT and preference optimization.
  • Multi-Stage Pipelines - Sequences supervised fine-tuning and preference optimization to iteratively refine model behavior.
  • Supervised Fine-Tuning - Provides a process for training language models on curated instruction-response pairs for task adaptation.
  • Supervised Fine-Tuning Frameworks - Provides a comprehensive framework for aligning pretrained models to specific tasks and behaviors using labeled datasets.
  • Training Recipes - Uses standardized training recipes to define and reproduce exact hyperparameters and model training parameters.
  • Chat Model Evaluations - Includes a system for assessing instruction-tuned chat models on conversational ability and human alignment.
  • Distributed Computing - Coordinates large-scale model alignment tasks across multiple GPUs and compute nodes.
  • Configuration-Driven Pipelines - Provides reproducible training pipelines defined via structured recipe configuration files.
  • Constitutional Alignments - Supports alignment based on structured constitutional principles and recipes.
  • LLM Benchmarking - Evaluates alignment quality and model helpfulness using standardized benchmarks and task configurations.
  • Training Optimization Strategies - Implements acceleration strategies and distributed processing to optimize training across varying hardware setups.
  • Low-Rank Adaptation - Implements low-rank adaptation to reduce memory requirements during large-scale language model optimization.
  • Rejection Sampling Optimizations - Implements rejection sampling to improve the quality of fine-tuned models by selecting the best responses.
  • Parameter Efficient Fine-Tuning - Implements low-rank adaptation (LoRA) to reduce memory requirements during large-scale model optimization.
  • Algorithm Comparisons - Provides a framework for evaluating the performance of different alignment methods across various models.
  • Dataset Blending - Combines multiple datasets with weighted sampling and formats them into chat templates for training.
  • Training Metric Streaming - Streams real-time training and evaluation metrics to external monitoring dashboards.
  • Multi-Turn Dialogue Benchmarking - Measures model helpfulness and quality using single-turn and multi-turn conversational benchmarks.
  • Reinforcement Learning - Directly distills alignment preferences into language models.

Istoric stele

Graficul istoricului de stele pentru huggingface/alignment-handbookGraficul istoricului de stele pentru huggingface/alignment-handbook

Căutare AI

Explorează mai multe repository-uri excelente

Descrie ce ai nevoie în limbaj simplu — AI-ul sortează mii de proiecte open source selectate în funcție de relevanță.

Start searching with AI

Alternative open-source pentru Alignment Handbook

Proiecte open-source similare, clasificate după numărul de funcționalități comune cu Alignment Handbook.
  • thinking-machines-lab/tinker-cookbookAvatar thinking-machines-lab

    thinking-machines-lab/tinker-cookbook

    2,856Vezi pe GitHub↗

    Tinker Cookbook is an open-source framework for fine-tuning large language models, supporting supervised learning, reinforcement learning, and parameter-efficient techniques like LoRA adapters. It provides a complete pipeline for aligning models with human preferences through multi-stage RLHF workflows, from supervised fine-tuning through preference optimization to reinforcement learning. The framework distinguishes itself through recipe-based training orchestration, where fine-tuning workflows are defined as composable recipe files that chain data loading, model configuration, and training l

    Python
    Vezi pe GitHub↗2,856
  • meta-pytorch/torchtuneAvatar meta-pytorch

    meta-pytorch/torchtune

    5,774Vezi pe GitHub↗

    Torchtune is a PyTorch-native library for fine-tuning, aligning, and quantizing large language models. It provides a config-driven system for instantiating components, orchestrating distributed training, and managing parameter-efficient fine-tuning with quantization support, all through YAML-based configurations and command-line overrides. The library distinguishes itself through its comprehensive post-training workflow orchestration, combining supervised fine-tuning, preference optimization (DPO, PPO, GRPO), knowledge distillation, and quantization-aware training in a single configurable pip

    Python
    Vezi pe GitHub↗5,774
  • pytorch/torchtuneAvatar pytorch

    pytorch/torchtune

    5,774Vezi pe GitHub↗

    Torchtune is a PyTorch-native library for fine-tuning, aligning, and quantizing large language models. It provides a configurable training pipeline orchestrated through YAML recipes, with CLI overrides and component swapping, distributed training via FSDP2, memory optimizations, and parameter-efficient fine-tuning methods like LoRA, DoRA, and QLoRA. The library distinguishes itself through its YAML-driven configuration system that defines all training parameters and instantiates components from config files, with full CLI override capability for any field or component at launch time. It suppo

    Python
    Vezi pe GitHub↗5,774
  • inclusionai/arealAvatar inclusionAI

    inclusionAI/AReaL

    3,559Vezi pe GitHub↗

    AReaL is a system for agent orchestration, distributed model training, and parameter-efficient tuning. It provides a framework for developing multi-turn reasoning agents and training large models using reinforcement learning from human feedback. The project implements a toolkit for improving the visual reasoning and geometry problem solving capabilities of vision-language models. It utilizes a memory-efficient tuning system to optimize mathematical and reasoning models across different inference backends. The infrastructure supports large-scale training through tensor, pipeline, and expert p

    Pythonagentllmllm-agent
    Vezi pe GitHub↗3,559
Vezi toate cele 30 alternative pentru Alignment Handbook→

Întrebări frecvente

Ce face huggingface/alignment-handbook?

Acest proiect este un framework de aliniere și o suită de pipeline-uri pentru antrenarea modelelor de limbaj folosind fine-tuning supervizat și optimizarea preferințelor. Oferă instrumente pentru executarea antrenării distribuite la scară largă pe mai multe GPU-uri și noduri de calcul, alături de un sistem pentru măsurarea utilității modelului și a calității dialogului prin benchmark-uri single-turn și multi-turn.

Care sunt principalele funcționalități ale huggingface/alignment-handbook?

Principalele funcționalități ale huggingface/alignment-handbook sunt: Preference Alignment, Reward Modeling, Direct Preference Optimization, Distributed Training Configurations, Large Scale Training Suites, Distributed Training, Alignment Frameworks, Multi-Stage Pipelines.

Care sunt câteva alternative open-source pentru huggingface/alignment-handbook?

Alternativele open-source pentru huggingface/alignment-handbook includ: thinking-machines-lab/tinker-cookbook — Tinker Cookbook is an open-source framework for fine-tuning large language models, supporting supervised learning,… meta-pytorch/torchtune — Torchtune is a PyTorch-native library for fine-tuning, aligning, and quantizing large language models. It provides a… pytorch/torchtune — Torchtune is a PyTorch-native library for fine-tuning, aligning, and quantizing large language models. It provides a… openrlhf/openrlhf — OpenRLHF is a training framework and alignment library designed for reinforcement learning from human feedback across… inclusionai/areal — AReaL is a system for agent orchestration, distributed model training, and parameter-efficient tuning. It provides a… allenai/open-instruct — Open-Instruct is a distributed training and instruction tuning framework for large language models. It functions as a…