# morvanzhou/reinforcement-learning-with-tensorflow

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/morvanzhou-reinforcement-learning-with-tensorflow).**

9,464 stars · 4,982 forks · Python · MIT

## Links

- GitHub: https://github.com/MorvanZhou/Reinforcement-learning-with-tensorflow
- Homepage: https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/
- awesome-repositories: https://awesome-repositories.com/repository/morvanzhou-reinforcement-learning-with-tensorflow.md

## Topics

`a3c` `actor-critic` `asynchronous-advantage-actor-critic` `ddpg` `deep-deterministic-policy-gradient` `deep-q-network` `double-dqn` `dqn` `dueling-dqn` `machine-learning` `policy-gradient` `ppo` `prioritized-replay` `proximal-policy-optimization` `q-learning` `reinforcement-learning` `sarsa` `sarsa-lambda` `tensorflow-tutorials` `tutorial`

## Description

This project is an educational repository of reinforcement learning agents and tutorials implemented using TensorFlow. It provides a practical codebase for both model-free and model-based learning agents, designed to demonstrate how AI agents learn through trial and error.

The collection features detailed implementations of various algorithmic approaches, including Deep Q-Networks and Policy Gradient methods. It specifically covers Actor-Critic architectures for continuous and discrete action spaces, alongside Proximal Policy Optimization and Deep Deterministic Policy Gradients.

The framework incorporates several training stabilization techniques, such as experience replay buffers with prioritized sampling, target network synchronization, and asynchronous parallel training. It also includes interfaces for connecting agents to standardized simulation platforms and tools for visualizing agent progress and training costs.

## Tags

### Artificial Intelligence & ML

- [Reinforcement Learning](https://awesome-repositories.com/f/artificial-intelligence-ml/reinforcement-learning.md) — Provides a comprehensive collection of reinforcement learning agents that learn through trial and error. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/intro-RL-methods))
- [Educational Tutorials](https://awesome-repositories.com/f/artificial-intelligence-ml/reinforcement-learning/educational-tutorials.md) — Provides educational tutorials on implementing AI agents that learn from trial and error using TensorFlow.
- [Actor-Critic Architectures](https://awesome-repositories.com/f/artificial-intelligence-ml/actor-critic-architectures.md) — Implements architectures that combine policy-based agents with value-based evaluators to reduce gradient variance.
- [Continuous Control Actors](https://awesome-repositories.com/f/artificial-intelligence-ml/continuous-control-actors.md) — Uses Actor-Critic networks to output precise continuous values for controlling agents in high-dimensional spaces. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/DDPG))
- [Deterministic Policy Gradients](https://awesome-repositories.com/f/artificial-intelligence-ml/continuous-control-actors/deterministic-policy-gradients.md) — Optimizes actions in continuous environments using deterministic policy gradients to overcome discrete action limitations. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/intro-AC))
- [Continuous Control Training](https://awesome-repositories.com/f/artificial-intelligence-ml/continuous-control-training.md) — Develops controllers for high-dimensional, real-valued action spaces found in robotic and physical simulations.
- [Deep Learning Development](https://awesome-repositories.com/f/artificial-intelligence-ml/deep-learning-development.md) — Develops deep neural network agents capable of processing high-dimensional state spaces. ([source](https://cdn.jsdelivr.net/gh/morvanzhou/reinforcement-learning-with-tensorflow@master/README.md))
- [Deep Q-Learning Implementations](https://awesome-repositories.com/f/artificial-intelligence-ml/deep-q-learning-implementations.md) — Implements deep Q-learning algorithms using neural networks and experience replay to solve discrete tasks.
- [Direct Policy Mapping](https://awesome-repositories.com/f/artificial-intelligence-ml/direct-policy-mapping.md) — Maps environment observations directly to action probabilities for continuous distributions without using value functions. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/policy-gradient-softmax1))
- [Experience Replay Buffers](https://awesome-repositories.com/f/artificial-intelligence-ml/experience-replay-buffers.md) — Provides memory structures that store agent transitions to break temporal correlations during training.
- [Noise](https://awesome-repositories.com/f/artificial-intelligence-ml/exploration-strategies/noise.md) — Uses Gaussian noise and decaying variance to facilitate exploration in continuous control environments. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/DDPG))
- [Probabilistic Action Sampling](https://awesome-repositories.com/f/artificial-intelligence-ml/inference-sampling-strategies/reproducible-sampling/probabilistic-generative-sampling/probabilistic-action-sampling.md) — Samples actions from network-generated probability distributions to ensure the agent explores the environment stochastically. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/policy-gradient-softmax2))
- [Policy and Value Function Approximators](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/frameworks/model-construction/neural-network-layers/policy-and-value-function-approximators.md) — Constructs neural network architectures to approximate state-value functions for reinforcement learning agents. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/DQN1))
- [Reinforcement Learning Environments](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/frameworks/reinforcement-learning-environments.md) — Provides simulation environments that model state transitions and reward assignments based on agent actions. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/general-rl))
- [Policy Gradient Methods](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/infrastructure/machine-learning-training/utilities/gradient-optimization-techniques/policy-gradient-methods.md) — Implements gradient-based architectures to optimize action probabilities and maximize expected rewards. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/policy-gradient-softmax2))
- [Reinforcement Learning Training Loops](https://awesome-repositories.com/f/artificial-intelligence-ml/neural-network-training-frameworks/reinforcement-learning-training-loops.md) — Stores state transitions and updates neural networks to optimize action selection through experience replay. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/gym))
- [Policy Gradient Implementations](https://awesome-repositories.com/f/artificial-intelligence-ml/policy-gradient-implementations.md) — Develops agents that directly optimize action probabilities to maximize expected rewards.
- [Deep Deterministic](https://awesome-repositories.com/f/artificial-intelligence-ml/policy-gradient-implementations/deep-deterministic.md) — Implements the Deep Deterministic Policy Gradient algorithm to handle continuous action spaces. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/intro-DDPG))
- [Policy Gradient Optimizers](https://awesome-repositories.com/f/artificial-intelligence-ml/policy-gradient-optimizers.md) — Provides gradient-based methods for updating policy parameters to maximize the agent's expected total reward. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/))
- [Action Probability Optimizations](https://awesome-repositories.com/f/artificial-intelligence-ml/reinforcement-learning-reward-systems/action-probability-optimizations.md) — Adjusts action selection probabilities based on received rewards to increase high-reward behaviors. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/intro-PG))
- [Reward Shaping](https://awesome-repositories.com/f/artificial-intelligence-ml/reinforcement-learning-reward-systems/reward-shaping.md) — Implements reward shaping to modify environment reward signals and guide the agent toward efficient behaviors. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/gym))
- [Neural Action-Value Estimation](https://awesome-repositories.com/f/artificial-intelligence-ml/reinforcement-learning-value-estimators/neural-action-value-estimation.md) — Predicts action-value Q-values using neural networks to replace large tabular lookup tables. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/intro-DQN))
- [State-Action Value Updates](https://awesome-repositories.com/f/artificial-intelligence-ml/reinforcement-learning-value-estimators/state-action-value-updates.md) — Adjusts stored state-action values based on the temporal difference between predicted values and actual rewards. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/tabular-q2))
- [State-Value Estimators](https://awesome-repositories.com/f/artificial-intelligence-ml/state-value-estimators.md) — Calculates the expected return of a state using a critic network to provide feedback for policy improvement. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/DPPO))
- [Advantage Actor-Critic Implementations](https://awesome-repositories.com/f/artificial-intelligence-ml/actor-critic-architectures/advantage-actor-critic-implementations.md) — Implements Asynchronous Advantage Actor-Critic to aggregate updates from parallel agents into a global network. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/A3C))
- [Cost Functions](https://awesome-repositories.com/f/artificial-intelligence-ml/cost-functions.md) — Implements mathematical cost functions to monitor network convergence over the course of training steps. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/DQN3))
- [Deep Q-Learning Frameworks](https://awesome-repositories.com/f/artificial-intelligence-ml/deep-q-learning-frameworks.md) — Employs deep Q-learning frameworks to stabilize value estimates using secondary networks. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/DQN1))
- [Clipped Double Q-Learning](https://awesome-repositories.com/f/artificial-intelligence-ml/deep-q-learning-implementations/clipped-double-q-learning.md) — Uses multiple neural networks to decouple action selection from value estimation, reducing Q-value overestimation. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/double_DQN))
- [Data-Parallel Training](https://awesome-repositories.com/f/artificial-intelligence-ml/distributed-training-frameworks/data-parallel-training.md) — Provides a framework for distributing RL training workloads across multiple processor cores using data-parallel techniques. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/intro-A3C))
- [Dueling Network Architectures](https://awesome-repositories.com/f/artificial-intelligence-ml/dueling-network-architectures.md) — Implements dueling network architectures to decouple state value estimation from action advantage.
- [Simulation Integrations](https://awesome-repositories.com/f/artificial-intelligence-ml/end-to-end-training-pipelines/simulation-integrations.md) — Integrates simulation environments with machine learning workflows to train and test agent behaviors. ([source](https://cdn.jsdelivr.net/gh/morvanzhou/reinforcement-learning-with-tensorflow@master/README.md))
- [Episodic Update Logic](https://awesome-repositories.com/f/artificial-intelligence-ml/episodic-update-logic.md) — Implements episodic update logic to refine behavior guidelines after full episode completions. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/intro-RL-methods))
- [Epsilon-Greedy Exploration Strategies](https://awesome-repositories.com/f/artificial-intelligence-ml/epsilon-greedy-exploration-strategies.md) — Implements epsilon-greedy exploration to balance discovery and reward optimization during training.
- [Global Network Synchronization](https://awesome-repositories.com/f/artificial-intelligence-ml/global-network-synchronization.md) — Implements the synchronization of a global network using gradients from multiple parallel local workers. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/A3C))
- [Asynchronous Gradient Aggregation](https://awesome-repositories.com/f/artificial-intelligence-ml/gradient-computation/gradient-aggregators/asynchronous-gradient-aggregation.md) — Aggregates gradients from multiple parallel agents into a global network to accelerate training.
- [Prioritized Transition Sampling](https://awesome-repositories.com/f/artificial-intelligence-ml/gradient-computation/gradient-based-sampling/reinforcement-learning-sampling/prioritized-transition-sampling.md) — Uses a SumTree structure to prioritize the sampling of high-error transitions for more efficient learning. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/prioritized-replay))
- [Asynchronous Training](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/infrastructure/model-training-and-tuning/distributed-and-scaling-strategies/asynchronous-training-utilities/asynchronous-training.md) — Implements asynchronous training to decouple rollout generation from gradient updates across parallel workers.
- [Training Weight Adjustments](https://awesome-repositories.com/f/artificial-intelligence-ml/model-weight-management/dynamic-weight-updates/training-weight-adjustments.md) — Adjusts neural network weights based on the difference between predicted and target value estimates. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/intro-DQN))
- [Offline Reinforcement Learning](https://awesome-repositories.com/f/artificial-intelligence-ml/offline-reinforcement-learning.md) — Supports offline reinforcement learning by training policies from pre-collected datasets. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/intro-RL-methods))
- [Online Learning](https://awesome-repositories.com/f/artificial-intelligence-ml/online-learning.md) — Implements online learning to update model parameters in real-time during environment interaction. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/intro-RL-methods))
- [OpenAI Gym Integrations](https://awesome-repositories.com/f/artificial-intelligence-ml/openai-gym-integrations.md) — Provides integration with OpenAI Gym to manage state observations and action spaces for RL agents. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/gym))
- [Learning Guides](https://awesome-repositories.com/f/artificial-intelligence-ml/policy-gradient-implementations/learning-guides.md) — Offers guides on optimizing agent behavior by maximizing expected rewards using TensorFlow.
- [REINFORCE Implementations](https://awesome-repositories.com/f/artificial-intelligence-ml/reinforcement-learning-algorithms/reinforce-implementations.md) — Implements the REINFORCE algorithm to adjust action probabilities based on total accumulated episode rewards. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/policy-gradient-softmax1))
- [Parallel Experience Collection](https://awesome-repositories.com/f/artificial-intelligence-ml/reinforcement-learning-algorithms/reinforcement-learning-simulators/parallel-experience-collection.md) — Gathers experience from parallel worker threads to accelerate data collection for training. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/DPPO))
- [Policy Clipping](https://awesome-repositories.com/f/artificial-intelligence-ml/reinforcement-learning-optimizers/policy-clipping.md) — Implements clipped surrogate objectives to prevent overly large policy updates and ensure stable convergence.
- [RL Training Workflows](https://awesome-repositories.com/f/artificial-intelligence-ml/reinforcement-learning-training-pipelines/rl-training-workflows.md) — Implements training workflows that sample random memory batches to update neural network weights. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/DQN3))
- [Dueling Network Architectures](https://awesome-repositories.com/f/artificial-intelligence-ml/reinforcement-learning-value-estimators/dueling-network-architectures.md) — Implements a dueling architecture to decouple state value estimation from action advantage for better convergence. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/dueling-DQN))
- [Eligibility Traces](https://awesome-repositories.com/f/artificial-intelligence-ml/reinforcement-learning-value-estimators/eligibility-traces.md) — Implements Sarsa-lambda eligibility traces to update multiple previous state-action pairs for efficient reward propagation. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/intro-sarsa-lambda))
- [Sarsa Update Implementations](https://awesome-repositories.com/f/artificial-intelligence-ml/reinforcement-learning-value-estimators/sarsa-update-implementations.md) — Implements Sarsa update logic to refine Q-tables based on the state, action, reward, and next action. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/tabular-sarsa2))
- [Tabular Q-Learning](https://awesome-repositories.com/f/artificial-intelligence-ml/tabular-q-learning.md) — Implements tabular Q-learning to derive optimal action-value functions via lookup table updates. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/intro-q-learning))
- [Target Network Synchronization](https://awesome-repositories.com/f/artificial-intelligence-ml/target-network-synchronization.md) — Implements target network synchronization to prevent training divergence in Deep Q-Networks. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/DQN2))
- [Training Stability Techniques](https://awesome-repositories.com/f/artificial-intelligence-ml/training-stability-techniques.md) — Implements training stability techniques including memory buffers and target networks with varied update frequencies. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/intro-DDPG))

### Education & Learning Resources

- [Reinforcement Learning Study Guides](https://awesome-repositories.com/f/education-learning-resources/machine-learning-algorithm-study-guides/reinforcement-learning-study-guides.md) — Provides a comprehensive collection of educational guides and tutorials for implementing reinforcement learning algorithms using TensorFlow.

### Networking & Communication

- [Policy-Value Network Coordination](https://awesome-repositories.com/f/networking-communication/federated-coordination-networks/policy-value-network-coordination.md) — Coordinates separate networks for policy and value estimation to determine actions and evaluate states. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/intro-DDPG))

### User Interface & Experience

- [Epsilon-Greedy Exploration](https://awesome-repositories.com/f/user-interface-experience/selectable-lists/randomized-selection/epsilon-greedy-exploration.md) — Implements epsilon-greedy logic to balance exploration and exploitation during the agent's learning process. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/DQN3))

### Part of an Awesome List

- [Model-Based Reinforcement Learning](https://awesome-repositories.com/f/awesome-lists/ai/model-based-reinforcement-learning.md) — Includes model-based reinforcement learning to predict future states and plan agent actions. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/intro-RL-methods))
- [Proximal Policy Optimization Alignment](https://awesome-repositories.com/f/awesome-lists/ai/training-and-alignment/proximal-policy-optimization-alignment.md) — Implements the PPO algorithm using clipped surrogate objectives to ensure stable policy updates. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/DPPO))

### System Administration & Monitoring

- [Agent Performance Visualizers](https://awesome-repositories.com/f/system-administration-monitoring/agent-observability/agent-performance-visualizers.md) — Includes tools for rendering environment states and plotting cost curves to analyze agent performance and convergence. ([source](https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/gym))