# morvanzhou/pytorch-tutorial

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/morvanzhou-pytorch-tutorial).**

8,458 stars · 3,089 forks · Jupyter Notebook · MIT

## Links

- GitHub: https://github.com/MorvanZhou/PyTorch-Tutorial
- Homepage: https://mofanpy.com/tutorials/machine-learning/torch/
- awesome-repositories: https://awesome-repositories.com/repository/morvanzhou-pytorch-tutorial.md

## Topics

`autoencoder` `batch` `batch-normalization` `classification` `cnn` `dqn` `dropout` `gan` `generative-adversarial-network` `machine-learning` `neural-network` `python` `pytorch` `pytorch-tutorial` `pytorch-tutorials` `regression` `reinforcement-learning` `rnn` `tutorial`

## Description

This project is a collection of PyTorch learning resources and educational guides designed to teach the construction and training of neural networks. It serves as a comprehensive deep learning tutorial covering various model architectures and practical implementation strategies.

The resources provide specific guidance on implementing computer vision tasks, such as image classification and synthetic imagery generation, as well as reinforcement learning agents using value networks and experience replay. It also covers sequential data modeling through recurrent networks and generative modeling using adversarial networks and autoencoders.

The content encompasses the full machine learning workflow, including data engineering, model regularization, and parameter optimization. It further addresses performance acceleration via GPU usage and provides methods for monitoring training progress through high-dimensional feature and latent space visualization.

The project is implemented using Jupyter Notebooks.

## Tags

### Artificial Intelligence & ML

- [Deep Reinforcement Learning Implementations](https://awesome-repositories.com/f/artificial-intelligence-ml/deep-q-learning-implementations/deep-reinforcement-learning-implementations.md) — Provides full implementations of reinforcement learning agents using value networks and experience replay.
- [Neural Network Training](https://awesome-repositories.com/f/artificial-intelligence-ml/neural-network-training.md) — Provides a comprehensive guide to optimizing neural network weights via backpropagation and gradient descent. ([source](https://mofanpy.com/tutorials/machine-learning/torch/CNN))
- [Automatic Differentiation Engines](https://awesome-repositories.com/f/artificial-intelligence-ml/automatic-differentiation-engines.md) — Implements automatic gradient computation using computational graphs for parameter optimization.
- [Convolutional Neural Networks](https://awesome-repositories.com/f/artificial-intelligence-ml/convolutional-neural-networks.md) — Constructs convolutional neural networks using pooling and convolutional layers for image classification. ([source](https://mofanpy.com/tutorials/machine-learning/torch/CNN))
- [Dataset Batch Loading](https://awesome-repositories.com/f/artificial-intelligence-ml/dataset-batch-loading.md) — Implements data loaders that group samples into batches with shuffling for optimized training. ([source](https://mofanpy.com/tutorials/machine-learning/torch/train-on-batch))
- [Deep Q-Learning Implementations](https://awesome-repositories.com/f/artificial-intelligence-ml/deep-q-learning-implementations.md) — Provides a full implementation of Deep Q-Networks using dual networks and experience replay to stabilize action-value targets. ([source](https://mofanpy.com/tutorials/machine-learning/torch/DQN))
- [Epsilon-Greedy Exploration Strategies](https://awesome-repositories.com/f/artificial-intelligence-ml/epsilon-greedy-exploration-strategies.md) — Implements epsilon-greedy strategies to balance exploration and exploitation during the training of RL agents. ([source](https://mofanpy.com/tutorials/machine-learning/torch/DQN))
- [Experience Replay Buffers](https://awesome-repositories.com/f/artificial-intelligence-ml/experience-replay-buffers.md) — Implements memory structures that store agent transitions to break temporal correlations during neural network training. ([source](https://mofanpy.com/tutorials/machine-learning/torch/DQN))
- [Feature Extraction](https://awesome-repositories.com/f/artificial-intelligence-ml/feature-extraction.md) — Uses convolutional and pooling layers to extract hierarchical spatial features from images.
- [Generative Adversarial Networks](https://awesome-repositories.com/f/artificial-intelligence-ml/generative-adversarial-networks.md) — Implements generator and discriminator networks to synthesize realistic data samples.
- [GPU Acceleration](https://awesome-repositories.com/f/artificial-intelligence-ml/gpu-acceleration.md) — Implements hardware acceleration by moving model parameters and input tensors to GPU memory to speed up training. ([source](https://mofanpy.com/tutorials/machine-learning/torch/GPU))
- [Gradient Computation](https://awesome-repositories.com/f/artificial-intelligence-ml/gradient-computation.md) — Teaches automatic differentiation and gradient computation to support model training and parameter optimization. ([source](https://mofanpy.com/tutorials/machine-learning/torch/variable))
- [Dynamic Graph Builders](https://awesome-repositories.com/f/artificial-intelligence-ml/gradient-computation/dynamic-graph-builders.md) — Provides guidance on building computational graphs dynamically to support variable input shapes.
- [Image Classification](https://awesome-repositories.com/f/artificial-intelligence-ml/image-classification.md) — Provides a workflow for categorizing image datasets using convolutional neural networks.
- [Iterative Parameter Optimizations](https://awesome-repositories.com/f/artificial-intelligence-ml/iterative-parameter-optimizations.md) — Implements iterative parameter optimization using gradient descent to minimize cost functions. ([source](https://mofanpy.com/tutorials/machine-learning/torch/intro-gradient-descent))
- [Mathematical Operations](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/infrastructure/model-optimization-and-inference/hardware-and-acceleration/tensor-computing-libraries/tensor-libraries/mathematical-operations.md) — Provides routines for executing matrix multiplications and numerical calculations on tensors. ([source](https://mofanpy.com/tutorials/machine-learning/torch/torch-numpy))
- [Hardware Acceleration](https://awesome-repositories.com/f/artificial-intelligence-ml/model-training/hardware-acceleration.md) — Provides methods for offloading tensor computations to GPUs to accelerate neural network training.
- [Neural Network Construction](https://awesome-repositories.com/f/artificial-intelligence-ml/neural-network-construction.md) — Provides comprehensive guides on designing and building diverse deep learning architectures including convolutional and recurrent networks. ([source](https://mofanpy.com/tutorials/machine-learning/torch/))
- [Recurrent Neural Network Training](https://awesome-repositories.com/f/artificial-intelligence-ml/recurrent-neural-network-training.md) — Provides detailed instructions for building and training recurrent networks to improve sequential prediction accuracy. ([source](https://mofanpy.com/tutorials/machine-learning/torch/RNN-classification))
- [Recurrent Neural Networks](https://awesome-repositories.com/f/artificial-intelligence-ml/recurrent-neural-networks.md) — Provides instructions for constructing recurrent neural networks designed for sequential data and time-series prediction. ([source](https://mofanpy.com/tutorials/machine-learning/torch/RNN-regression))
- [Reinforcement Learning](https://awesome-repositories.com/f/artificial-intelligence-ml/reinforcement-learning.md) — Implements reinforcement learning agents featuring value networks and experience replay.
- [Neural Action-Value Estimation](https://awesome-repositories.com/f/artificial-intelligence-ml/reinforcement-learning-value-estimators/neural-action-value-estimation.md) — Implements neural networks to approximate Q-values for state-action pairs in high-dimensional reinforcement learning environments. ([source](https://mofanpy.com/tutorials/machine-learning/torch/intro-DQN))
- [Sequential Data Models](https://awesome-repositories.com/f/artificial-intelligence-ml/sequential-data-models.md) — Implements recurrent networks and LSTM layers for processing and predicting sequential data.
- [Target Network Synchronization](https://awesome-repositories.com/f/artificial-intelligence-ml/target-network-synchronization.md) — Uses secondary target networks with frozen parameters to decouple target values from estimations and stabilize Q-learning. ([source](https://mofanpy.com/tutorials/machine-learning/torch/intro-DQN))
- [Activation Functions](https://awesome-repositories.com/f/artificial-intelligence-ml/activation-functions.md) — Teaches the use of mathematical functions to introduce non-linearity into neural network models. ([source](https://mofanpy.com/tutorials/machine-learning/torch/intro-activation-function))
- [Autoencoders](https://awesome-repositories.com/f/artificial-intelligence-ml/autoencoders.md) — Implements autoencoder architectures for unsupervised learning through data compression and reconstruction. ([source](https://mofanpy.com/tutorials/machine-learning/torch/autoencoder))
- [Dropout Regularization](https://awesome-repositories.com/f/artificial-intelligence-ml/dropout-regularization.md) — Implements dropout regularization by randomly deactivating neurons to prevent model overfitting. ([source](https://mofanpy.com/tutorials/machine-learning/torch/dropout))
- [Latent Space Compression](https://awesome-repositories.com/f/artificial-intelligence-ml/generative-ai-resources/diffusion-visual-models/generative-ai-models/latent-space-generative-models/latent-space-projections/latent-space-encoders/latent-space-compression.md) — Implements encoder-decoder structures to compress high-dimensional data for feature extraction.
- [Synthetic Media Generators](https://awesome-repositories.com/f/artificial-intelligence-ml/generative-ai-resources/diffusion-visual-models/synthetic-content-generators/synthetic-media-generators.md) — Provides capabilities for generating realistic synthetic imagery and modifying visual traits via latent space manipulation. ([source](https://mofanpy.com/tutorials/machine-learning/torch/intro-GAN))
- [Convolutional Classifiers](https://awesome-repositories.com/f/artificial-intelligence-ml/image-classification/transformer-based-image-classifiers/convolutional-classifiers.md) — Designs image classifiers by combining convolutional and fully connected layers. ([source](https://mofanpy.com/tutorials/machine-learning/torch/intro-CNN))
- [Linear Regression](https://awesome-repositories.com/f/artificial-intelligence-ml/linear-regression.md) — Teaches how to build linear regression models to fit mathematical relationships between input and output data. ([source](https://mofanpy.com/tutorials/machine-learning/torch/regression))
- [Convolution Layers](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/frameworks/model-construction/neural-network-layers/convolution-layers.md) — Implements convolutional layers using sliding filters to extract spatial features from images. ([source](https://mofanpy.com/tutorials/machine-learning/torch/intro-CNN))
- [Model Training Optimizers](https://awesome-repositories.com/f/artificial-intelligence-ml/model-training-optimizers.md) — Guides the use of advanced optimizers and GPU acceleration to improve training convergence.
- [Network Architecture Design](https://awesome-repositories.com/f/artificial-intelligence-ml/neural-network-architectures/network-architecture-design.md) — Demonstrates how to design networks with linear layers and activations for data categorization. ([source](https://mofanpy.com/tutorials/machine-learning/torch/classification))
- [Acceleration Methods](https://awesome-repositories.com/f/artificial-intelligence-ml/optimization-algorithms/gradient-descent-algorithms/acceleration-methods.md) — Demonstrates the use of optimizers like Adam and RMSProp to accelerate training convergence. ([source](https://mofanpy.com/tutorials/machine-learning/torch/intro-speed-up-learning))
- [Weight Decay Regularization](https://awesome-repositories.com/f/artificial-intelligence-ml/overfitting-reduction-techniques/weight-decay-regularization.md) — Applies weight decay and penalty-based methods to constrain weight magnitudes and improve generalization. ([source](https://mofanpy.com/tutorials/machine-learning/torch/intro-overfitting))
- [Pooling Layers](https://awesome-repositories.com/f/artificial-intelligence-ml/pooling-layers.md) — Implements downsampling techniques using max and average pooling to reduce spatial dimensions in images. ([source](https://mofanpy.com/tutorials/machine-learning/torch/intro-CNN))
- [Pre-trained Model Transfer](https://awesome-repositories.com/f/artificial-intelligence-ml/pre-trained-model-transfer.md) — Demonstrates methods for adapting pre-trained model backbones to new tasks by resetting output layers. ([source](https://mofanpy.com/tutorials/machine-learning/torch/intro-feature-representation))
- [Loss Function Calculators](https://awesome-repositories.com/f/artificial-intelligence-ml/prediction-visualization/loss-function-calculators.md) — Implements various loss functions to compute the numerical difference between predicted and target values. ([source](https://mofanpy.com/tutorials/machine-learning/torch/optimizer))
- [Gated Recurrent Units](https://awesome-repositories.com/f/artificial-intelligence-ml/recurrent-neural-networks/gated-recurrent-units.md) — Demonstrates the use of gated units to manage information flow in recurrent neural networks.
- [Sequential Data Classification](https://awesome-repositories.com/f/artificial-intelligence-ml/sequential-data-models/sequential-data-classification.md) — Provides capabilities for analyzing sequences of inputs to produce final categories or sentiment outputs. ([source](https://mofanpy.com/tutorials/machine-learning/torch/intro-RNN))
- [Training Progress Monitors](https://awesome-repositories.com/f/artificial-intelligence-ml/training-progress-monitors.md) — Implements tools for tracking real-time metrics, such as loss and predictions, during the model training process. ([source](https://mofanpy.com/tutorials/machine-learning/torch/classification))

### Education & Learning Resources

- [Deep Learning Education](https://awesome-repositories.com/f/education-learning-resources/deep-learning-education.md) — Serves as a comprehensive educational resource for learning neural network theory and practice using PyTorch.
- [PyTorch Deep Learning Examples](https://awesome-repositories.com/f/education-learning-resources/deep-learning-education/deep-learning-platforms/pytorch-deep-learning-examples.md) — Offers a collection of reference implementations and tutorials for building networks with PyTorch.
- [Deep Learning Tutorials](https://awesome-repositories.com/f/education-learning-resources/deep-learning-tutorials.md) — Provides a detailed guide for implementing convolutional, recurrent, and generative network architectures.
- [Computer Vision Tutorials](https://awesome-repositories.com/f/education-learning-resources/computer-vision-tutorials.md) — Offers practical examples for implementing image classification and synthetic imagery generation.
- [Latent Space Manifold Visualization](https://awesome-repositories.com/f/education-learning-resources/manifold-learning-guides/latent-space-manifold-visualization.md) — Provides graphical representations of latent space distributions to analyze how networks separate data categories. ([source](https://mofanpy.com/tutorials/machine-learning/torch/autoencoder))

### Part of an Awesome List

- [Long Short-Term Memory Networks](https://awesome-repositories.com/f/awesome-lists/ai/neural-network-architectures/long-short-term-memory-networks.md) — Implements LSTM networks with gating mechanisms to manage long-term dependencies in sequences. ([source](https://mofanpy.com/tutorials/machine-learning/torch/intro-LSTM))
- [Off-Policy Optimization](https://awesome-repositories.com/f/awesome-lists/ai/off-policy-optimization.md) — Implements off-policy optimization by sampling random batches from experience replay to update the network. ([source](https://mofanpy.com/tutorials/machine-learning/torch/DQN))
- [Recurrent Neural Networks](https://awesome-repositories.com/f/awesome-lists/ai/recurrent-neural-networks.md) — Guides the implementation of recurrent architectures to process sequential data by maintaining internal states. ([source](https://mofanpy.com/tutorials/machine-learning/torch/intro-RNN))
- [Sequential Classifiers](https://awesome-repositories.com/f/awesome-lists/ai/neural-network-architectures/long-short-term-memory-networks/sequential-classifiers.md) — Demonstrates the use of long short-term memory layers to categorize sequential data into classes. ([source](https://mofanpy.com/tutorials/machine-learning/torch/RNN-classification))
- [Sequence To Sequence Models](https://awesome-repositories.com/f/awesome-lists/ai/sequence-to-sequence-models.md) — Implements sequence-to-sequence models for transforming input sequences into target output sequences. ([source](https://mofanpy.com/tutorials/machine-learning/torch/intro-RNN))
- [Synthetic Data Generation](https://awesome-repositories.com/f/awesome-lists/ai/synthetic-data-generation.md) — Implements techniques for creating synthetic training samples through reconstructive modeling. ([source](https://mofanpy.com/tutorials/machine-learning/torch/intro-autoencoder))

### Data & Databases

- [Image Dataset Imports](https://awesome-repositories.com/f/data-databases/image-dataset-imports.md) — Implements the process of loading raw image directories into tensors for model training. ([source](https://mofanpy.com/tutorials/machine-learning/torch/CNN))
- [Dimensionality Projection Plots](https://awesome-repositories.com/f/data-databases/data-analysis-visualization/visualization-frameworks-libraries/data-visualization/three-dimensional-visualizations/dimensionality-projection-plots.md) — Implements visualizations that project high-dimensional layer outputs into lower-dimensional spaces using T-SNE to identify clusters. ([source](https://mofanpy.com/tutorials/machine-learning/torch/CNN))
- [Nonlinear Transformations](https://awesome-repositories.com/f/data-databases/data-processing-pipelines/data-transformation/array-tensor-manipulation/tensor-transformations/nonlinear-transformations.md) — Demonstrates how to transform input tensors using mathematical operations to enable complex pattern learning. ([source](https://mofanpy.com/tutorials/machine-learning/torch/activation))
- [Dimensionality Reduction](https://awesome-repositories.com/f/data-databases/vector-quantization/high-dimensional-vector-compressors/dimensionality-reduction.md) — Provides techniques for reducing high-dimensional input sets to extract representative features. ([source](https://mofanpy.com/tutorials/machine-learning/torch/intro-autoencoder))

### Graphics & Multimedia

- [Image Sequence Processors](https://awesome-repositories.com/f/graphics-multimedia/image-editing-processing/image-processing/image-sequence-processors.md) — Reshapes image pixels into temporal sequences for processing by recurrent neural networks. ([source](https://mofanpy.com/tutorials/machine-learning/torch/RNN-classification))