thu-ml/tianshou
Tianshou
Tianshou is a reinforcement learning framework designed for developing and testing agents. It provides a system for implementing custom agents by defining policies and parameter update rules to optimize agent behavior.
The framework decouples neural network architectures from update logic through policy-based abstractions and separates data pre-processing from gradient updates. It utilizes a collector-driven pipeline to stream experience from environments into structured memory buffers for sampled learning.
The system supports vectorized environment execution to run multiple parallel instances for data collection. Experiment management is handled through a declarative interface for defining hyperparameters, environment factories, and training pipelines.
Features
- Reinforcement Learning - Provides a comprehensive framework for implementing and developing reinforcement learning agents through standardized policy abstractions.
- Agent Training Tools - Facilitates the execution of training loops that optimize policies through gradient updates and environment interaction.
- Experience Replay Buffers - Ships structured memory buffers for storing environment transitions to support sampled off-policy learning.
- Modular Parameter Update Rules - Separates the data pre-processing step from the gradient update step to allow flexible algorithm implementation.
- Policy Abstractions - Decouples neural network architectures from update logic using standardized policy classes.
- Training Execution Loops - Implements a coordination system to manage the training loop, including data collection and gradient updates.
- Data Collection - Gathers interaction data from simulation environments to be used as training sets for learning agents.
- RL Experience Collection - Provides mechanisms for gathering experience from multiple environments and storing transitions for learning.
- Experience Collection Pipelines - Provides a collector-driven pipeline to stream experience from environments directly into training buffers.
- Reinforcement Learning Configurations - Offers a declarative interface for defining hyperparameters and environment factories to ensure reproducible RL runs.
- Vectorized Environments - Supports running multiple parallel instances of an environment to accelerate data collection and increase sample diversity.
- RL Experiment Configurations - Includes a declarative interface for defining training pipelines, environment factories, and hyperparameters.
- Reinforcement Learning - Elegant deep reinforcement learning library for PyTorch.
