DLR-RM/rl-baselines3-zoo
Rl Baselines3 Zoo
This project is a collection of pretrained reinforcement learning agents and training scripts built on Stable Baselines3 and Gymnasium. It provides a framework for training agents to solve specific tasks, managing experiment reproducibility, and deploying pretrained models.
The system includes a specialized benchmarking suite and optimization tools for tuning agent settings. It utilizes automated search spaces and distributed trials to maximize performance, while employing bootstrap sampling to generate statistically robust performance metrics and confidence intervals.
Broad capabilities cover the full reinforcement learning lifecycle, including experiment tracking with external dashboards, model hub synchronization for sharing trained agents, and environment decoration for data normalization. It also provides visualization utilities for generating reward plots and recording agent behavior videos.
Configuration is managed through external files that decouple hyperparameters from core logic.
Features
- Reinforcement Learning Training - Provides a framework for training reinforcement learning agents to solve specific tasks using Stable Baselines3.
- RL Algorithm Benchmarking Toolkits - Ships a specialized benchmarking suite for evaluating RL agent success using statistically robust metrics.
- Hyperparameter Configurations - Enables defining learning rates and custom policies via external configuration files to tune agent behavior.
- Hyperparameter Optimization Tools - Tunes agent settings through automated search spaces and distributed trials to maximize performance.
- Hyperparameter Search Strategies - Implements search strategies including automated trials and pruning to optimize reinforcement learning agent settings.
- Hyperparameter Optimizers - Provides automated search, tuning, and pruning of agent configurations using distributed trials.
- Distributed Tuning Orchestrators - Runs hyperparameter optimization trials across multiple distributed jobs using a shared database to accelerate search.
- RL Agent Implementation Frameworks - Provides a comprehensive collection of pretrained reinforcement learning agents and training scripts built on Stable Baselines3.
- Weight Serialization - Implements serialization and loading of neural network weights and hyperparameters to reconstruct agents.
- Pretrained Agent Execution - Allows loading a previously trained model and executing it within a target environment to observe behavior.
- Experiment Tracking - Logs training curves and hyperparameters to external dashboards to monitor reinforcement learning progress.
- Reinforcement Learning Performance Visualizers - Generates graphical plots of training rewards and success rates to analyze learning progress.
- Agent Performance Evaluators - Implements tools for assessing agent behavior and policy stability through automated callbacks and video recording.
- RL Experiment Management Frameworks - Provides a framework for logging training curves and syncing trained models with remote repositories.
- Experiment Tracking - Integrates systems for monitoring training progress and recording learning curves to track model quality.
- Experiment Tracking Integrations - Provides interfaces that connect reinforcement learning workflows to external platforms for automated experiment tracking.
- ML Metric Logging - Provides mechanisms to log numerical training metrics and learning curves to external monitoring dashboards.
- Performance Benchmarking - Provides tools for evaluating and comparing reinforcement learning agent performance across various simulation environments.
- Environment Wrappers - Uses environment wrappers to preprocess simulation data through observation normalization and state decoration.
- Observation and Reward Normalization - Standardizes observation and reward scales by wrapping environments to ensure more stable agent convergence.
- Training Callbacks - Supports injecting custom logic into training loops at specific intervals via a callback system.
- YAML Configuration Files - Uses YAML configuration files to decouple agent hyperparameters from the core training logic.
- Agent Performance Visualizers - Renders agent trajectories and records behavior videos to analyze final model performance.
- Bootstrapped Performance Statistics - Implements bootstrap sampling to generate statistically robust confidence intervals and means for agent performance.
- Bootstrapped Performance Benchmarks - Computes confidence intervals and interquartile means using bootstrap sampling for statistically robust performance benchmarks.
