hkust-nlp/simpleRL-reason
SimpleRL Reason
simpleRL-reason is a training framework designed to improve mathematical and logical deduction in large language models. It utilizes reinforcement learning and policy optimization to enhance the accuracy and transparency of step-by-step deduction chains.
The project implements a pipeline that establishes baseline capabilities through supervised fine-tuning before applying reinforcement learning to maximize deductive accuracy. It features a reward modeling toolkit that calculates scalar feedback by comparing generated reasoning steps against verified mathematical ground truths.
The framework includes a standardized benchmark for measuring reasoning performance and a web-based interface for visualizing the evolution of logical processes. It further incorporates curated difficulty-based datasets and a dependency-free execution suite to analyze model responses.
Features
- Mathematical Reasoning Training - Implements a comprehensive training methodology for improving model performance on complex mathematical problem-solving tasks.
- Rule-Based Reward Models - Calculates scalar feedback by comparing generated reasoning steps against verified mathematical ground truths.
- Reasoning Training Frameworks - Provides a framework for improving mathematical and logical deduction in LLMs using reinforcement learning and policy optimization.
- Supervised Fine-Tuning Workflows - Implements supervised fine-tuning workflows to establish baseline logical capabilities before applying reinforcement learning.
- Policy Gradient Optimizers - Utilizes gradient-based methods to optimize model policy parameters for improved logical accuracy in deduction.
- Reinforcement Learning - Applies reinforcement learning techniques to refine model outputs based on verified ground truth answers and scalar feedback.
- Reasoning Chain Training - Uses reinforcement learning with rule-based rewards to enable models to learn extended reasoning chains spontaneously.
- Deduction Chains - Structures model outputs as sequential logical deduction chains to improve the transparency and accuracy of mathematical reasoning.
- Reasoning Process Monitors - Provides a web-based interface to visualize and audit the evolution of step-by-step reasoning chains across training stages.
- Difficulty-Based Datasets - Provides curated training and evaluation datasets organized by complexity to facilitate multi-step deductive problem solving.
- Reasoning Evaluations - Provides a testing framework to measure the model's ability to reason and comprehend complex mathematical tasks.
- Reasoning Optimization - Enhances complex reasoning performance through two-stage reinforcement learning and value estimation.
- Mathematical Reasoning Evaluations - Includes a standardized evaluation suite to objectively measure numerical computation and multi-step problem-solving skills.
- Reinforcement Learning Frameworks - Simplified reinforcement learning pipeline for reasoning tasks.
