kwai/DouZero
DouZero
DouZero is a deep reinforcement learning framework and training system designed to teach digital agents to master complex card games. It provides the infrastructure to implement high-throughput reinforcement learning pipelines and evaluate the competitive success of game agents.
The system utilizes a distributed actor-learner architecture that separates game simulation actors from GPU training devices to accelerate model convergence. It combines Monte Carlo Tree Search with policy-based value estimation to determine optimal moves through recursive evaluation and random sampling.
The toolkit includes capabilities for self-play simulation loops to generate synthetic training data and asynchronous experience buffers for sampled batch training. It also features benchmarking tools to measure agent performance by comparing win rates against random, rule-based, or human datasets.
Features
- Deep Reinforcement Learning Implementations - Provides a functional implementation of an agent using deep reinforcement learning and neural networks.
- Card Game Frameworks - Provides a framework for developing and evaluating reinforcement learning agents specifically for card game environments.
- Monte Carlo Tree Search - Implements the Monte Carlo Tree Search algorithm to determine optimal moves through random sampling of the game tree.
- Self-Play Data Generation - Generates training data through agents playing against themselves to iteratively improve model performance.
- Reinforcement Learning Training - Provides a general framework for training agents in simulation environments using reinforcement learning.
- Card Game RL Trainers - Acts as a specialized trainer for card game agents using deep reinforcement learning and self-play simulations.
- Distributed Training Systems - Provides high-throughput infrastructure that separates simulation actors from GPU training devices.
- Game Simulation Environments - Implements agent-based game simulation environments specifically designed for reinforcement learning and behavioral research.
- Actor-Learner Patterns - Uses a distributed actor-learner architecture to decouple game simulation from GPU model updates.
- Distributed Simulation Workflows - Accelerates agent training by distributing game simulations across multiple actor processes and GPU devices.
- Asynchronous Experience Collection - Features parallelized collection of environment interactions across multiple CPU cores to accelerate training throughput.
- Agent Performance Evaluators - Provides tools for assessing agent behavior and policy stability by comparing win percentages against baselines.
- Game Episode Evaluators - Evaluates agent performance by running complete game episodes and recording win rates.
- GPU Training Accelerators - Uses parallelization strategies across CPU actors and GPU learners to increase the speed of model training.
- Model Performance Evaluators - Provides tools for quantifying agent success by comparing win rates against random, rule-based, or human baselines.
- State-Value Estimators - Utilizes state-value estimators to predict the win probability of a given game state.
- Card Games - Self-play deep reinforcement learning agent for DouDizhu.
