# vance0124/token-level-direct-preference-optimization

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/vance0124-token-level-direct-preference-optimization).**

155 stars · 15 forks · Python · Apache-2.0

## Links

- GitHub: https://github.com/Vance0124/Token-level-Direct-Preference-Optimization
- awesome-repositories: https://awesome-repositories.com/repository/vance0124-token-level-direct-preference-optimization.md

## Description

This repo contains a reference implementation of the TDPO algorithm for training language models from preference data, as described in the paper Token-level Direct Preference Optimization (ICML 2024). Our implementation is based on DPO, and follows the same usage guidelines.

## Tags

### Part of an Awesome List

- [Reinforcement Learning](https://awesome-repositories.com/f/awesome-lists/ai/reinforcement-learning.md) — Hierarchical reward optimization at the token level.