Projects sharing features with RRHF

The official implementation of Self-Play Fine-Tuning (SPIN)

verl is a distributed training system designed for large language model alignment and reinforcement learning. It provides a framework for executing post-training pipelines, including supervised fine-tuning and reinforcement learning from human feedback, to refine model behavior and agentic capabilities. The system utilizes a hybrid training and inference engine that optimizes memory and communication when switching between model generation and gradient updates. It supports multi-modal reinforcement learning for models processing both image and text data, and implements algorithms such as PPO

This repo provides the source code & data of our paper: Improving Large Language Models via Fine-grained Reinforcement Learning with Minimum Editing Constraint (arXiv 2024)

Welcome to drlzh.ai: a hands-on deep reinforcement learning course where you build the algorithms, not just read about them.

FinRL is a reinforcement learning framework designed for the development, training, and backtesting of automated trading strategies. It functions as a quantitative finance toolkit that integrates deep learning algorithms with financial market simulations to address complex portfolio management and asset allocation tasks. The platform provides an end-to-end pipeline for transforming raw market data into actionable trading models. The project distinguishes itself through a layered, modular architecture that separates data processing, environment simulation, and agent training. This design allow

Fine-Grained RLHF

A modular RL library to fine-tune language models to human preferences

⚡️A Blazing-Fast Python Library for Ranking Evaluation, Comparison, and Fusion 🐍

A library with extensible implementations of DPO, KTO, PPO, ORPO, and other human-aware loss functions (HALOs).

This repository contains the implementation of DISCERN in Python. You can download the manuscript from my website or arXiv.

Reinforcement Learning in PyTorch

This project is a quantized fine-tuning framework for large language models. It implements a low-rank adaptation library and a four-bit quantizer to reduce the GPU memory requirements needed to train large models. The framework utilizes four-bit quantization and low-rank adapters to enable model training on consumer-grade hardware. It further reduces the memory footprint through double quantization and a paged optimizer that offloads states to system RAM. The system supports distributed training across multiple GPUs to handle larger parameter scales and includes utilities for custom dataset

Reinforcement Learning in Go

A set of Deep Reinforcement Learning Agents implemented in Tensorflow.

TensorFlow implementation of Deep Reinforcement Learning papers

trlx is a reinforcement learning library and training framework designed to align large language models using human feedback. It serves as a distributed trainer and compute orchestrator for scaling high-parameter models across multiple GPUs and nodes. The project provides tools for reinforcement learning from human feedback and model alignment. It implements reward-model-based optimization and proximal policy optimization to refine model behavior based on goal-oriented rewards or human-labeled datasets. The framework covers distributed training strategies, including model parallelism, parame

Accelerated RL

|tests| |pypi| |docs| |License|

ROLL is a distributed reinforcement learning framework and model alignment toolkit designed for large language models. It serves as a scalable training pipeline and GPU cluster manager, providing the infrastructure to align model behavior using reinforcement learning algorithms and preference optimization techniques. The project distinguishes itself through an agentic rollout orchestrator that generates and collects multi-turn interaction trajectories between AI agents and simulated environments. It supports specialized alignment methods including Direct Preference Optimization, reinforcement

A short and easy implementation of Quantile Regression DQN | Distributional Reinforcement Learning