inclusionAI/AReaL

Features

• Large-Scale Model Training - Provides a scalable infrastructure for training large models using various forms of distributed parallelism.
• Agentic Reasoning Loops - Structures model interactions as a sequence of internal thought steps before final tool execution.
• Agentic Workflow Orchestration - Orchestrates multi-turn reasoning agents that execute tool calls through external executors.
• Reward Modeling - Implements Bradley-Terry reward modeling to align model behavior using pairwise human preference comparisons.
• External Tool Execution - Decouples reasoning from action by routing tool calls to external executors that return results to the agent.
• Hybrid Model Parallelism - Supports large-scale training through a combination of tensor, pipeline, and expert parallelism.
• Reinforcement Learning Alignment - Implements synchronous and asynchronous reinforcement learning algorithms to improve model reasoning and goal alignment.
• Distributed Training - Enables large-scale model training across cloud environments using tensor, pipeline, and expert parallelism.
• Low-Rank Adaptation - Employs low-rank adaptation to reduce training memory by updating only small, decomposed weight matrices.
• Scalable Distributed Pipelines - Offers a scalable environment for training large models across distributed hardware using advanced parallelism.
• Reinforcement Learning Training - Uses both synchronous and asynchronous reinforcement learning algorithms to improve goal alignment and reasoning.
• Asynchronous Algorithms - Implements asynchronous reinforcement learning to decouple experience collection from the training loop for better stability.
• RLHF Training Pipelines - Integrates reward modeling and reinforcement learning to train large language models from human feedback.
• Agent Framework Integrations - Provides adapters for integrating third-party agent frameworks into a unified multi-turn reasoning execution environment.
• Reinforcement Learning Optimization - Utilizes reinforcement learning techniques to train vision-language models for complex visual tasks and geometry problem solving.
• Vision-Language Trainers - Provides a toolkit specifically for training multimodal models to handle visual reasoning and geometry problems.
• Preference-Based Model Alignments - Combines reward modeling and parameter-efficient tuning to align model behavior with human preferences.
• Vision-Language Training - Trains vision-language models to solve complex visual reasoning and geometry problems using RL techniques.
• Parameter Efficient Fine-Tuning - Implements low-rank adaptation to minimize memory usage when training mathematical and reasoning models.
• Reasoning Model Training Suites - Provides asynchronous RL infrastructure to optimize the training of complex reasoning and agentic models.
• Agentic Frameworks - Framework for agentic reinforcement learning.
• Agentic Reinforcement Learning - Framework for agentic reinforcement learning.
• Reasoning Models - Reinforcement learning for reasoning.
• Reinforcement Learning - Library for reinforcement learning research and development.
• Reinforcement Learning Frameworks - Large-scale asynchronous reinforcement learning system for language reasoning.

Star history