DLR-RM/stable-baselines3
Stable Baselines3
Stable-baselines3 is a reinforcement learning library built on the PyTorch deep learning framework. It provides a collection of reliable, standardized implementations of reinforcement learning algorithms designed for training, testing, and benchmarking agent policies in diverse simulated environments.
The library functions as an agent training toolkit that emphasizes modularity and reproducibility. It features a unified environment interface and supports vectorized execution to accelerate data collection across multiple simulation instances. Users can customize neural network architectures, feature extractors, and policy definitions to suit specific observation and action spaces, while built-in tools for deterministic seeding ensure consistent results across training runs.
Beyond core training, the project includes comprehensive utilities for managing the agent lifecycle. This encompasses memory-efficient experience replay buffering, advanced exploration strategies for continuous control, and automated monitoring of performance metrics. The framework also supports the export and distribution of trained models, facilitating collaboration and deployment across various hardware and runtime environments.
Features
- Reinforcement Learning - Provides a collection of reliable implementations of reinforcement learning algorithms for training and benchmarking agents.
- Agent Training Tools - Functions as a comprehensive toolkit for executing simulations, managing buffers, and monitoring agent learning.
- Deep Learning Frameworks - Built on PyTorch to provide standardized interfaces for creating and training neural network-based policies.
- Reinforcement Learning Environments - Implements a standardized interface for agents to interact with diverse simulation environments for training.
- Deterministic - Ensures reproducible results across training runs by providing tools for deterministic random seeding.
- Policy Architectures - Enables configuration of neural network structures and hidden layers to tailor agent behavior to specific observation spaces.
- Reinforcement Learning Algorithms - Implements standard reinforcement learning algorithms to facilitate training and comparative analysis of policies.
- Reinforcement Learning Training Pipelines - Provides a consistent interface for implementing and training reinforcement learning agents to solve complex tasks.
- Experience Replay Buffers - Implements memory-efficient experience replay buffers to decouple data collection from gradient-based optimization.
- Custom - Supports specialized neural network modules to process raw observations like images or multi-modal data.
- Model Exporters - Provides utilities to export and serialize trained reinforcement learning models for cross-platform deployment and inference.
- Custom Policy Definitions - Allows granular control over actor-critic architectures by supporting custom policy class definitions.
- Multi-Output Action Architectures - The library handles dictionary action spaces to allow for independent or mixed discrete and continuous action outputs within a single policy.
- Reinforcement Learning Algorithm Analyzers - Provides tools for comparing and benchmarking the performance of different reinforcement learning algorithms across environments.
- Vectorized Environments - Accelerates data collection by running multiple simulation instances in parallel across CPU cores.
- Intrinsic Reward Modules - Integrates intrinsic reward mechanisms to improve agent exploration in environments with sparse feedback.
- Reinforcement Learning Environments - Provides a standardized interface for agent-environment interaction across heterogeneous reinforcement learning tasks.
- Agent Performance Evaluators - Assesses agent behavior and policy stability during training using automated callbacks and video recording.
- Training Monitoring Tools - Tracks and logs training metrics and hyperparameters to external visualization tools for analysis.
- Deep Learning - Listed in the “Deep Learning” section of the Awesome Python awesome list.
- Reinforcement Learning - Reliable PyTorch implementations of reinforcement learning algorithms.
- Parallel Execution - Executes multiple simulation instances in parallel to accelerate data collection and improve training efficiency.
- Noise - Enhances training stability in continuous control tasks by utilizing pink noise exploration strategies.
- State-Dependent Exploration - Improves agent performance in complex environments through specialized state-dependent exploration strategies.
- Intrinsic Reward Mechanisms - Augments standard objective functions with auxiliary signals to encourage exploration in sparse-reward environments.
- Performance Benchmarking - Compares reinforcement learning algorithm performance by evaluating agents and tracking metrics across environments.
- Optimizer Configurations - Allows customization of the optimization process by selecting specific optimizer classes and parameters.
- Research and Data Analysis Tools - Provides a platform for research into advanced exploration strategies and continuous control tasks.
- Remote Model Hubs - Facilitates community collaboration by enabling the upload, versioning, and distribution of pre-trained agents.
- Model Exporting - Supports exporting and distributing trained agents for use across different hardware and runtime environments.
- Neural Network Architectures - Supports configuration of specialized policy networks and feature extractors for diverse observation spaces.
- Modular Architectures - Separates feature extraction from decision-making logic to allow flexible neural network configurations.
- Observation Transformers - Processes complex multi-input observation spaces by transforming them into unified feature vectors for agent consumption.
- Training Callbacks - Provides callback mechanisms to inject custom logic into the training loop for monitoring and checkpointing.
