awesome-repositories.com
Blog
awesome-repositories.com

Descoperă cele mai bune repository-uri open source cu căutare AI.

ExploreazăCăutări recomandateAlternative open-sourceSoftware self-hostedBlogHartă site
ProiectDespreCum realizăm clasamentulPresăServer MCP
LegalConfidențialitateTermeni
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
·
eric-mitchell avatar

eric-mitchell/direct-preference-optimization

0
View on GitHub↗
2,888 stele·236 fork-uri·Python·Apache-2.0·3 vizualizări

Direct Preference Optimization

Acest proiect este un framework pentru alinierea modelelor de limbaj mari (LLM) cu preferințele umane. Oferă o bibliotecă pentru optimizarea comportamentului modelului prin maparea directă a datelor de preferință către un obiectiv de politică, eliminând necesitatea unui model de recompensă separat.

Framework-ul utilizează un obiectiv matematic de tip closed-form pentru a ajusta ponderile modelului prin maximizarea probabilității răspunsurilor preferate față de cele nepreferate. Include un set de instrumente pentru fine-tuning supervizat, permițând utilizatorilor să stabilească o bază stabilă pe date demonstrative înainte de a aplica tehnici de optimizare bazate pe preferințe.

Biblioteca suportă antrenarea distribuită, permițând scalarea alinierii modelelor pe mai multe acceleratoare hardware prin strategii de sharding și paralelizare a datelor. De asemenea, oferă utilitare pentru pregătirea seturilor de date de preferință personalizate, mapând perechi de prompt-uri și răspunsuri pentru a structura feedback-ul uman pentru pipeline-ul de antrenare.

Features

  • Direct Preference Optimization - Optimizes language models using human feedback data through direct preference optimization algorithms.
  • Preference Alignment Objectives - Adjusts model weights by maximizing the likelihood of preferred responses relative to dispreferred ones using a closed-form mathematical objective.
  • Reward Modeling - Optimizes language models by mapping preference data directly to a policy objective without requiring a separate reward model.
  • Fine-Tuning Toolkits - Provides a collection of training utilities for aligning language models on specific datasets.
  • Gradient-Based Parameter Updates - Updates model parameters by calculating divergence between preferred and rejected outputs to shift probability distributions.
  • Data Parallelism - Splits training datasets across multiple devices and synchronizes gradients to accelerate model alignment.
  • Large Language Models - Optimizes language models using human preference data to ensure generated outputs match desired behaviors.
  • Large-Scale Model Training - Scales the training process across multiple hardware accelerators to reduce computation time for large models.
  • Distributed Training - Scales model training across multiple hardware accelerators using parallelization strategies like data sharding.
  • Large Language Model Training Frameworks - Provides a framework for scaling the training of large language models across multiple hardware accelerators.
  • Supervised Fine-Tuning Workflows - Trains language models on curated datasets to establish a strong baseline performance before applying advanced optimization techniques.
  • Supervised Fine-Tuning - Establishes a stable baseline by training the model on high-quality demonstration data before applying preference-based fine-tuning.
  • Model Fine-Tuning - Trains models on specific datasets to ensure policy data remains in-distribution before applying preference-based learning.

Istoric stele

Graficul istoricului de stele pentru eric-mitchell/direct-preference-optimizationGraficul istoricului de stele pentru eric-mitchell/direct-preference-optimization

Căutare AI

Explorează mai multe repository-uri excelente

Descrie ce ai nevoie în limbaj simplu — AI-ul sortează mii de proiecte open source selectate în funcție de relevanță.

Start searching with AI

Colecții curatoriate care includ Direct Preference Optimization

Colecții selectate manual în care apare Direct Preference Optimization.
  • Framework-uri pentru optimizarea și auto-tuning-ul prompt-urilor
  • Framework-uri pentru alinierea LLM-urilor și RLHF

Alternative open-source pentru Direct Preference Optimization

Proiecte open-source similare, clasificate după numărul de funcționalități comune cu Direct Preference Optimization.
  • internlm/xtunerAvatar InternLM

    InternLM/xtuner

    5,150Vezi pe GitHub↗

    xtuner is a comprehensive training engine for large language models, offering a toolkit for pre-training, supervised fine-tuning, and the optimization of vision-language multimodal models. It serves as a distributed training accelerator and a specialized framework for scaling Mixture-of-Experts models and aligning model behavior through reinforcement learning from human feedback. The project distinguishes itself through advanced memory and compute optimizations, such as sequence parallelism for ultra-long context windows and interleaved pipeline parallelism to reduce GPU idle time. It provide

    Pythonagentdeepseek-v3gpt-oss
    Vezi pe GitHub↗5,150
  • allenai/open-instructAvatar allenai

    allenai/open-instruct

    3,586Vezi pe GitHub↗

    Open-Instruct is a distributed training and instruction tuning framework for large language models. It functions as a coordinator for supervised fine-tuning, reinforcement learning from human feedback pipelines, and tool-use training, providing specialized roles for dataset curation and model alignment. The project distinguishes itself through a high-performance training architecture that utilizes actor-based distributed coordination and hybrid sharding to manage large GPU clusters. It implements advanced alignment techniques including direct preference optimization, group relative policy opt

    Python
    Vezi pe GitHub↗3,586
  • huggingface/alignment-handbookAvatar huggingface

    huggingface/alignment-handbook

    5,621Vezi pe GitHub↗

    This project is an alignment framework and suite of pipelines for training language models using supervised fine-tuning and preference optimization. It provides tools for executing large-scale distributed training across multiple GPUs and compute nodes, alongside a system for measuring model helpfulness and dialogue quality through single-turn and multi-turn benchmarks. The framework includes specialized tools for direct preference optimization to refine model behavior using paired data without a separate reward model. It also supports constitutional AI alignment and the training of reward mo

    Python
    Vezi pe GitHub↗5,621
  • nndl/llm-beginnerAvatar nndl

    nndl/llm-beginner

    6,421Vezi pe GitHub↗

    This project is a collection of educational resources and technical guides focused on the development and implementation of large language models. It provides a comprehensive curriculum covering transformer architectures, training methods, and deployment strategies. The materials provide detailed instructions for building autonomous agents using reasoning loops and tool integration, as well as guides for fine-tuning models through supervised learning and preference optimization. It also includes tutorials for constructing retrieval augmented generation pipelines and implementing transformer m

    Pythonagentfudannlpllm
    Vezi pe GitHub↗6,421
Vezi toate cele 30 alternative pentru Direct Preference Optimization→

Întrebări frecvente

Ce face eric-mitchell/direct-preference-optimization?

Acest proiect este un framework pentru alinierea modelelor de limbaj mari (LLM) cu preferințele umane. Oferă o bibliotecă pentru optimizarea comportamentului modelului prin maparea directă a datelor de preferință către un obiectiv de politică, eliminând necesitatea unui model de recompensă separat.

Care sunt principalele funcționalități ale eric-mitchell/direct-preference-optimization?

Principalele funcționalități ale eric-mitchell/direct-preference-optimization sunt: Direct Preference Optimization, Preference Alignment Objectives, Reward Modeling, Fine-Tuning Toolkits, Gradient-Based Parameter Updates, Data Parallelism, Large Language Models, Large-Scale Model Training.

Care sunt câteva alternative open-source pentru eric-mitchell/direct-preference-optimization?

Alternativele open-source pentru eric-mitchell/direct-preference-optimization includ: internlm/xtuner — xtuner is a comprehensive training engine for large language models, offering a toolkit for pre-training, supervised… allenai/open-instruct — Open-Instruct is a distributed training and instruction tuning framework for large language models. It functions as a… huggingface/alignment-handbook — This project is an alignment framework and suite of pipelines for training language models using supervised… nndl/llm-beginner — This project is a collection of educational resources and technical guides focused on the development and… eleutherai/gpt-neox — gpt-neox is a distributed training system and framework for building large-scale autoregressive language models. It… inclusionai/areal — AReaL is a system for agent orchestration, distributed model training, and parameter-efficient tuning. It provides a…