RL training environments

IsaacGymEnvs is a GPU-accelerated physics sandbox and robotics policy training suite designed for reinforcement learning. It serves as a vectorized robotic simulator that runs thousands of parallel environments on GPUs to accelerate the training of neural networks. The project provides a sim-to-real transfer framework that utilizes domain randomization and physics variations to ensure policies trained in simulation are robust enough for deployment on real hardware. It distinguishes itself through a high-performance architecture that uses tensor-based state management to handle observations an

Acme is a reinforcement learning framework and execution engine designed for developing and benchmarking learning algorithms. It provides a library of modular components and reference implementations used to construct agents and establish performance baselines. The system enables the scaling of agent architectures from single-stream execution to large distributed environments. This allows for the transition from initial prototyping to distributed execution for training and evaluation. The framework covers reinforcement learning development and agent architecture prototyping, providing the bu

OpenManus-RL is a reinforcement learning framework and distributed training pipeline designed to train large language models as agents. It serves as an agentic reasoning optimizer and reward model trainer, providing the infrastructure to improve model decision-making through reward-based policy optimization. The project distinguishes itself through a distributed architecture that supports parameter sharding across multiple compute nodes and a coordinated rollout system for collecting interaction trajectories. It incorporates advanced reasoning strategies, such as Tree-of-Thoughts and Monte Ca

Gymnasium is a suite of standardized APIs and simulation toolkits used to evaluate agent behavior and benchmark reinforcement learning algorithms. It provides a standardized interface for creating and interacting with simulated environments, enabling the training of reinforcement learning agents through a consistent set of interaction protocols. The project emphasizes experimental reproducibility through a versioned API and a system for tracking changes to environment logic using version suffixes. This ensures that learning results remain consistent and can be replicated across different soft

Tianshou is a reinforcement learning framework designed for developing and testing agents. It provides a system for implementing custom agents by defining policies and parameter update rules to optimize agent behavior. The framework decouples neural network architectures from update logic through policy-based abstractions and separates data pre-processing from gradient updates. It utilizes a collector-driven pipeline to stream experience from environments into structured memory buffers for sampled learning. The system supports vectorized environment execution to run multiple parallel instanc

Malmo is a voxel-based simulation platform designed for artificial intelligence research and the study of autonomous agent behaviors. Built as a sandbox environment using Minecraft, it serves as a framework for multi-agent simulation and reinforcement learning research within a 3D grid of blocks. The project distinguishes itself through a multi-agent simulation framework that coordinates and synchronizes multiple autonomous agents to perform collaborative missions. It provides a standardized interface following reinforcement learning specifications, allowing it to function as an environment f

ElegantRL is a deep reinforcement learning framework and quantitative trading platform designed for automating financial decision making. It provides a system for designing and training agents using massively parallel GPU execution and includes a coordination layer for multi-agent reinforcement learning. Additionally, it features a GPU-based solver for NP-complete and nonconvex mathematical optimization problems. The platform distinguishes itself through GPU-accelerated environments that simulate thousands of parallel market interactions on a single device to accelerate data collection. It in

Gym is a reinforcement learning environment toolkit and agent simulation framework. It provides a standardized API and a universal communication interface that defines how learning agents interact with simulation environments through actions and observations. The project includes a benchmark environment suite and a diverse library of pre-configured simulation worlds, including physics engines and classic control tasks. It enables the creation of custom simulation environments to train agents in specific operational scenarios while ensuring reproducibility across different learning algorithms.

This project is a reinforcement learning toolkit and simulation-based AI trainer for creating intelligent agents within Unity simulations. It provides a multi-agent simulation framework for configuring cooperative or competitive scenarios and includes an environment wrapper that bridges simulations with standard machine learning libraries using gym-style interfaces. The system features a native cross-platform inference engine that executes trained neural network models for real-time decision making without external dependencies. It enables the acceleration of the learning process by running m

Isaac Lab is an open-source framework for training robot policies in physically simulated environments, supporting both single-agent and multi-agent reinforcement learning. It is built on an Omniverse-PhysX simulation backend that models rigid bodies, articulated systems, deformable objects, and sensors, and provides a task-based environment configuration system where each training environment is defined as a modular class specifying observation spaces, action spaces, reward functions, and termination conditions. The framework distinguishes itself through an RL-library abstraction layer that

RLinf is a distributed reinforcement learning orchestrator and embodied AI training framework. It provides the infrastructure to train vision-language-action models and robotic policies using a combination of reinforcement learning and supervised fine-tuning. The system is designed for scaling workloads across GPU clusters, managing the placement of actors, rollout workers, and environment components. It features a specialized robotics data collection pipeline for gathering teleoperated demonstrations and simulation trajectories into standardized replay buffers, alongside a hardware interface

Stable-baselines3 is a reinforcement learning library built on the PyTorch deep learning framework. It provides a collection of reliable, standardized implementations of reinforcement learning algorithms designed for training, testing, and benchmarking agent policies in diverse simulated environments. The library functions as an agent training toolkit that emphasizes modularity and reproducibility. It features a unified environment interface and supports vectorized execution to accelerate data collection across multiple simulation instances. Users can customize neural network architectures, f

Universe is a training and evaluation platform that transforms websites, games, and software into standardized environments for general intelligence agents. It functions as a reinforcement learning wrapper and remote environment orchestrator, providing a consistent interface to wrap diverse software for AI agent interaction. The platform distinguishes itself through a visual observation interface that streams real-time pixel data and transmits keyboard and mouse events to simulate human interaction. It utilizes a bi-directional communication protocol to deliver reward signals and performance

Verifiers is a reinforcement learning environment framework and evaluation toolkit designed to train and evaluate large language models. It provides a standardized system for constructing simulation environments, managing training harnesses, and tracking agent trajectories through multi-turn interactions. The project features a dedicated agent trajectory manager to handle branching rollouts and token sequences, alongside an evaluation toolkit that tests model outputs against defined reward rubrics and datasets. It includes capabilities for reward engineering and the ability to package environ

FinRL is a reinforcement learning framework designed for the development, training, and backtesting of automated trading strategies. It functions as a quantitative finance toolkit that integrates deep learning algorithms with financial market simulations to address complex portfolio management and asset allocation tasks. The platform provides an end-to-end pipeline for transforming raw market data into actionable trading models. The project distinguishes itself through a layered, modular architecture that separates data processing, environment simulation, and agent training. This design allow

Lab is a customizable 3D platform and research testbed designed for training and testing autonomous agents using reinforcement learning. It serves as a spatial AI training simulator where agents can be evaluated through navigation and puzzle-solving tasks. The environment allows for the definition of complex layouts and task behaviors through external scripting, enabling the generation of specific challenges for AI research. It supports both automated training via standard API bindings and manual agent control to validate simulation dynamics. The system utilizes a grid-based spatial represen

PyBoy is a programmable Game Boy emulator and hardware simulation framework written in Python. It functions as an emulation engine that allows users to execute original handheld software while providing a programmatic interface to control, probe, and automate game execution. The project is specifically designed as a reinforcement learning environment, exposing emulator states and controls to facilitate the training of machine learning agents. It distinguishes itself by providing tools for game area mapping and the extraction of simplified 2D screen representations and collision maps to suppor

This project is a transformer post-training toolkit and reinforcement learning library designed to align language model behavior with human preferences. It provides a framework for managing the transition from supervised fine-tuning to reinforcement learning and preference optimization. The library distinguishes itself through a specialized focus on preference optimization and reward modeling, enabling the adjustment of model outputs based on preferred versus rejected examples. It also includes capabilities for training agents within controlled sandbox environments using task suites and verif

Ray is a distributed computing framework designed to scale Python and Java applications across clusters by abstracting task scheduling and resource management. It functions as a resource-aware execution engine that manages task dependencies, placement, and fault tolerance across networked compute nodes. At its core, the system provides a stateful actor model, allowing developers to define classes that run in dedicated processes to maintain and mutate internal state across remote method calls. The framework distinguishes itself through a robust cross-language interoperability layer, enabling f

Dopamine is a reinforcement learning research framework designed for prototyping and testing algorithms across diverse simulated environments. It provides an agent development toolkit that utilizes a flat class hierarchy to facilitate the creation and extension of learning agents. The framework includes a standardization layer via environment wrappers that connect agents to various physics simulations and gaming environments. It also features a high-performance experience replay buffer for storing and sampling transition data to improve training stability, alongside a dedicated hyperparameter

Retro Games in Gym

Habitat-sim is a high-performance 3D simulation platform designed for training and benchmarking embodied AI agents within photorealistic indoor and outdoor environments. It serves as a simulator for AI and robotics, providing a system for generating synthetic data and simulating physical interactions. The project is distinguished by a native C++ core that enables high-throughput simulation and a rendering pipeline using physically based rendering and baked global illumination. It features a navigation system based on pre-computed navigation meshes to ensure collision-free traversal and a rigi

TF-Agents: A reliable, scalable and easy to use TensorFlow library for Contextual Bandits and Reinforcement Learning.

PyGame Learning Environment is a Python framework that provides a standardized interface for training artificial intelligence agents within diverse game environments. It functions as a communication layer that bridges reinforcement learning agents with game state observations, action inputs, and reward signals, allowing for consistent interaction across different game titles. The platform distinguishes itself by offering a headless execution mode that disables graphical rendering pipelines. By bypassing display overhead, this feature accelerates training cycles for automated agents during bac

Unitree RL Gym is an integrated software framework designed for the simulation, training, and deployment of motion control policies for legged robotic platforms. It provides a comprehensive environment for developing locomotion strategies for quadruped and humanoid robots by leveraging reinforcement learning algorithms within high-fidelity physics-based simulations. The platform distinguishes itself by offering an end-to-end workflow that bridges the gap between virtual training and physical execution. It includes specialized tools for mapping raw sensor data into normalized state-space obser

Gym-anytrading is a reinforcement learning toolkit designed to simulate financial market conditions for the development and evaluation of automated trading agents. It provides a standardized framework that models stock and forex market data, allowing researchers to train agents through trial and error within consistent, gym-compatible environments. The platform distinguishes itself through an object-oriented architecture that enables users to define custom trading logic, including unique reward functions, profit calculations, and trade fee policies. By transforming raw financial datasets into

Legged Gym is a high-performance simulation platform and toolkit engineered for training autonomous robotic agents in complex, physics-based environments. It provides a comprehensive framework for developing legged locomotion control policies, enabling robots to learn movement strategies for navigating uneven terrain and managing physical disturbances through reinforcement learning. The platform distinguishes itself by utilizing hardware-accelerated physics and headless execution to maximize computational throughput during training. It incorporates a domain randomization pipeline that injects

The Arcade Learning Environment is a research-focused platform that provides a high-performance emulation engine for Atari 2600 games. It serves as a standardized environment for training and evaluating artificial intelligence agents, bridging native game execution with machine learning libraries through a dedicated Python interface. The project distinguishes itself by offering robust support for parallel and vectorized environment execution, allowing multiple game instances to run simultaneously to accelerate data collection. It includes specialized tools for asynchronous command queuing and

Osim-rl is a research environment designed for the development and evaluation of reinforcement learning agents within physics-based musculoskeletal simulations. It provides a standardized interface that maps physiological state observations to muscle excitation control signals, enabling the study of human movement and biomechanics through iterative policy optimization. The framework distinguishes itself by integrating high-fidelity musculoskeletal modeling with tools for scientific benchmarking and reproducible experimentation. It allows researchers to define custom reward functions and adjus

RL-Swarm is a decentralized reinforcement learning framework designed to coordinate distributed machine learning agents across peer-to-peer networks. It functions as a distributed computing orchestrator that manages multi-agent roles and hardware resources to facilitate large-scale collaborative training. By anchoring training processes in cryptographic verification and decentralized contracts, the system ensures that model development remains transparent, verifiable, and resistant to manipulation. The platform distinguishes itself through a multi-agent expert coordination model that organize

Agentic RL Training at Scale

dm_control is a physics-based simulation framework and robot control simulation toolkit designed for creating and interacting with continuous control tasks. It serves as a suite of reinforcement learning environments and a benchmarking tool for evaluating autonomous agents within virtual physics spaces. The framework provides a collection of environments based on MuJoCo physics bindings to simulate rigid body dynamics and contact forces. It features hardware-accelerated rendering and interactive viewers for the visualization of physics environments and agent behavior. The system supports the

Accelerated RL