# hunkim/deeplearningzerotoall

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/hunkim-deeplearningzerotoall).**

4,494 stars · 2,611 forks · Jupyter Notebook

## Links

- GitHub: https://github.com/hunkim/DeepLearningZeroToAll
- Homepage: https://www.youtube.com/user/hunkims
- awesome-repositories: https://awesome-repositories.com/repository/hunkim-deeplearningzerotoall.md

## Topics

`keras` `lab` `mxnet` `python` `pytorch` `tensorflow` `tutorial`

## Description

DeepLearningZeroToAll is a comprehensive educational resource and implementation collection focused on deep learning and machine learning. It provides a structured learning path using TensorFlow to move from foundational linear models to complex neural network architectures.

The project is distinguished by its practical implementations of various network types, including multilayer perceptrons for logic problems, convolutional neural networks for spatial data and image recognition, and recurrent neural networks using LSTM cells for time-series forecasting and character sequence prediction. It also includes detailed demonstrations of model regularization through batch normalization and dropout techniques.

The repository covers a broad range of capabilities, including supervised machine learning with linear and logistic regression, data engineering for tensor manipulation and scaling, and model optimization through gradient descent and manual backpropagation calculations. It further includes tools for model evaluation, weight persistence, and training observability via cost function visualization and metric logging.

The content is delivered through a series of Jupyter Notebooks.

## Tags

### Artificial Intelligence & ML

- [Convolutional Neural Networks](https://awesome-repositories.com/f/artificial-intelligence-ml/convolutional-neural-networks.md) — Implements convolutional neural networks with max-pooling to recognize handwritten digits and image data. ([source](https://github.com/hunkim/DeepLearningZeroToAll/blob/master/lab-11-1-mnist_cnn.py))
- [Backpropagation](https://awesome-repositories.com/f/artificial-intelligence-ml/backpropagation.md) — Implements manual gradient calculations using the chain rule to demonstrate weight update mechanics. ([source](https://github.com/hunkim/DeepLearningZeroToAll/blob/master/lab-09-5-linear_back_prop.py))
- [Batch Normalization](https://awesome-repositories.com/f/artificial-intelligence-ml/batch-normalization.md) — Implements batch normalization to stabilize training and accelerate convergence in deep networks. ([source](https://github.com/hunkim/DeepLearningZeroToAll/blob/master/lab-10-6-mnist_nn_batchnorm.ipynb))
- [Binary Classifiers](https://awesome-repositories.com/f/artificial-intelligence-ml/binary-classifiers.md) — Builds binary classifiers to solve logic problems using weights and sigmoid activations. ([source](https://github.com/hunkim/DeepLearningZeroToAll/blob/master/lab-09-1-xor.py))
- [Convolutional Layers](https://awesome-repositories.com/f/artificial-intelligence-ml/convolutional-layers.md) — Provides convolutional layers that apply filters to extract spatial features from image data.
- [Convolutional Neural Network Architectures](https://awesome-repositories.com/f/artificial-intelligence-ml/convolutional-neural-network-architectures.md) — Designs CNN structures specifically for image feature extraction and classification. ([source](https://github.com/hunkim/DeepLearningZeroToAll/blob/master/lab-11-0-cnn_basics.ipynb))
- [Cost Functions](https://awesome-repositories.com/f/artificial-intelligence-ml/cost-functions.md) — Implements mathematical cost functions to measure prediction error for minimization via gradient descent. ([source](https://github.com/hunkim/DeepLearningZeroToAll/blob/master/lab-03-3-minimizing_cost_tf_optimizer.py))
- [Dropout Regularization](https://awesome-repositories.com/f/artificial-intelligence-ml/dropout-regularization.md) — Implements dropout layers that randomly deactivate neurons during training to prevent overfitting. ([source](https://github.com/hunkim/DeepLearningZeroToAll/blob/master/lab-10-5-mnist_nn_dropout.py))
- [Gradient-Based Parameter Updates](https://awesome-repositories.com/f/artificial-intelligence-ml/gradient-based-parameter-updates.md) — Implements the fundamental process of adjusting model weights using computed gradients.
- [Gradient Computation](https://awesome-repositories.com/f/artificial-intelligence-ml/gradient-computation.md) — Calculates function gradients manually to support training on multi-feature linear datasets. ([source](https://github.com/hunkim/DeepLearningZeroToAll/blob/master/lab-09-6-multi-linear_back_prop.py))
- [Image Classification](https://awesome-repositories.com/f/artificial-intelligence-ml/image-classification.md) — Builds and trains convolutional neural networks to categorize images and recognize handwritten digits.
- [Linear and Logistic Regression](https://awesome-repositories.com/f/artificial-intelligence-ml/linear-and-logistic-regression.md) — Implements linear and logistic regression models for binary and multi-class classification on structured data.
- [Linear Regression Models](https://awesome-repositories.com/f/artificial-intelligence-ml/linear-regression-models.md) — Implements models that map multiple input feature vectors to a scalar output for continuous value prediction. ([source](https://github.com/hunkim/DeepLearningZeroToAll/blob/master/lab-04-1-multi_variable_linear_regression.py))
- [Logistic Regression Models](https://awesome-repositories.com/f/artificial-intelligence-ml/logistic-regression-models.md) — Builds binary classification models using the sigmoid function and weight optimization via gradient descent. ([source](https://github.com/hunkim/DeepLearningZeroToAll/blob/master/lab-05-1-logistic_regression.py))
- [Linear Regression Implementations](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/algorithms/linear-regression-implementations.md) — Provides educational implementations of linear and binary regression models for predictive classification. ([source](https://github.com/hunkim/DeepLearningZeroToAll/tree/master/chainer))
- [Spatial Model Architectures](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/frameworks/model-construction/neural-network-layers/convolution-layers/deformable-convolutions/spatial-model-architectures.md) — Creates specialized models using convolutional layers to process and analyze spatial data. ([source](https://github.com/hunkim/DeepLearningZeroToAll/tree/master/ipynb))
- [LSTM Sequence Models](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/frameworks/model-construction/neural-network-layers/recurrent-layers/recurrent-model-definitions/lstm-sequence-models.md) — Uses LSTM cells to process time-series data and maintain state across sequential inputs.
- [Multilayer Perceptrons](https://awesome-repositories.com/f/artificial-intelligence-ml/multilayer-perceptrons.md) — Implements multilayer perceptrons with hidden layers for tasks like digit classification. ([source](https://github.com/hunkim/DeepLearningZeroToAll/blob/master/lab-10-2-mnist_nn.py))
- [Convolutional Network Builders](https://awesome-repositories.com/f/artificial-intelligence-ml/neural-network-construction/convolutional-network-builders.md) — Implements the construction of multi-layer convolutional neural networks using TensorFlow for image classification tasks. ([source](https://github.com/hunkim/DeepLearningZeroToAll/blob/master/lab-11-2-mnist_deep_cnn.py))
- [Neural Network Regularization](https://awesome-repositories.com/f/artificial-intelligence-ml/neural-network-regularization.md) — Applies regularization techniques including dropout and batch normalization to prevent overfitting.
- [Non-Linear Classifiers](https://awesome-repositories.com/f/artificial-intelligence-ml/non-linear-classifiers.md) — Builds multi-layer architectures using activation functions to solve complex non-linear classification problems. ([source](https://github.com/hunkim/DeepLearningZeroToAll/blob/master/lab-09-4-xor_tensorboard.py))
- [Gradient Descent Algorithms](https://awesome-repositories.com/f/artificial-intelligence-ml/optimization-algorithms/gradient-descent-algorithms.md) — Implements iterative optimization algorithms that update parameters in the direction of the negative gradient. ([source](https://github.com/hunkim/DeepLearningZeroToAll/blob/master/lab-03-2-minimizing_cost_gradient_update.py))
- [Recurrent Neural Networks](https://awesome-repositories.com/f/artificial-intelligence-ml/recurrent-neural-networks.md) — Implements recurrent neural network architectures for processing sequential data and character sequences. ([source](https://github.com/hunkim/DeepLearningZeroToAll/blob/master/lab-12-1-hello-rnn.py))
- [Spatial Processing Operations](https://awesome-repositories.com/f/artificial-intelligence-ml/spatial-processing-operations.md) — Develops convolutional networks designed for spatial analysis and image recognition tasks. ([source](https://github.com/hunkim/DeepLearningZeroToAll/tree/master/tf2))
- [Backpropagation Training](https://awesome-repositories.com/f/artificial-intelligence-ml/weight-reconstruction/discriminator-weight-updates/backpropagation-training.md) — Demonstrates the backpropagation algorithm by manually calculating gradients and updating weights. ([source](https://github.com/hunkim/DeepLearningZeroToAll/blob/master/lab-09-7-sigmoid_back_prop.py))
- [Class Probability Estimation](https://awesome-repositories.com/f/artificial-intelligence-ml/class-probability-estimation.md) — Implements the calculation of likelihoods for multi-class target categories using probability distributions. ([source](https://github.com/hunkim/DeepLearningZeroToAll/tree/master/numpy))
- [XOR Problem Solvers](https://awesome-repositories.com/f/artificial-intelligence-ml/complex-problem-solving/non-linear-problem-solvers/xor-problem-solvers.md) — Implements a multi-layer perceptron with sigmoid activation specifically to solve the XOR problem. ([source](https://github.com/hunkim/DeepLearningZeroToAll/blob/master/lab-09-2-xor-nn.py))
- [Cost Function Visualizations](https://awesome-repositories.com/f/artificial-intelligence-ml/cost-functions/cost-function-visualizations.md) — Plots the relationship between model parameters and resulting costs to visualize how weights affect error. ([source](https://github.com/hunkim/DeepLearningZeroToAll/blob/master/lab-03-1-minimizing_cost_show_graph.py))
- [Deep Learning Workflow Optimization](https://awesome-repositories.com/f/artificial-intelligence-ml/deep-learning-workflow-optimization.md) — Tunes learning rates and visualizes cost functions to optimize the training workflow of deep models.
- [Error Rate Reduction Techniques](https://awesome-repositories.com/f/artificial-intelligence-ml/optimization-algorithms/adaptive-learning-rate-optimizers/error-rate-reduction-techniques.md) — Optimizes model performance by calculating gradients and using optimizers to lower prediction error rates. ([source](https://github.com/hunkim/DeepLearningZeroToAll/tree/master/ipynb))
- [Accuracy Calculators](https://awesome-repositories.com/f/artificial-intelligence-ml/prediction-visualization/accuracy-calculators.md) — Includes utilities for computing classification accuracy by comparing predicted indices against ground truth labels. ([source](https://github.com/hunkim/DeepLearningZeroToAll/blob/master/lab-07-1-learning_rate_and_evaluation.py))
- [Character-Level Models](https://awesome-repositories.com/f/artificial-intelligence-ml/sequence-modeling/character-level-models.md) — Implements models that predict the next character in a sequence to capture linguistic patterns. ([source](https://github.com/hunkim/DeepLearningZeroToAll/blob/master/lab-12-3-char-seq-softmax-only.py))
- [Sequential and Graph Data Analysis](https://awesome-repositories.com/f/artificial-intelligence-ml/sequential-and-graph-data-analysis.md) — Provides capabilities for analyzing sequential data for applications like stock prediction and translation. ([source](https://github.com/hunkim/DeepLearningZeroToAll/tree/master/ipynb))
- [Sequential Data Models](https://awesome-repositories.com/f/artificial-intelligence-ml/sequential-data-models.md) — Creates frameworks for predicting future values in sequences, including character generation and time-series. ([source](https://github.com/hunkim/DeepLearningZeroToAll/tree/master/tf2))
- [Time Series Forecasting](https://awesome-repositories.com/f/artificial-intelligence-ml/time-series-forecasting.md) — Implements recurrent neural networks to forecast future values in temporal data, such as stock prices. ([source](https://github.com/hunkim/DeepLearningZeroToAll/blob/master/lab-12-5-rnn_stock_prediction.py))
- [Training Progress Monitors](https://awesome-repositories.com/f/artificial-intelligence-ml/training-progress-monitors.md) — Provides tools for tracking real-time metrics and model performance during the training process. ([source](https://github.com/hunkim/DeepLearningZeroToAll/tree/master/keras))
- [Xavier Initializations](https://awesome-repositories.com/f/artificial-intelligence-ml/weight-initialization/xavier-initializations.md) — Implements Xavier weight initialization to stabilize gradient variance during training. ([source](https://github.com/hunkim/DeepLearningZeroToAll/blob/master/lab-10-3-mnist_nn_xavier.py))

### Education & Learning Resources

- [Machine Learning Learning Paths](https://awesome-repositories.com/f/education-learning-resources/front-end-learning-paths/machine-learning-learning-paths.md) — Provides a structured educational path for learning deep learning fundamentals through practical TensorFlow tutorials.
- [Sequential State Architectures](https://awesome-repositories.com/f/education-learning-resources/architecture-reference-implementations/deep-learning-architectures/sequential-state-architectures.md) — Builds architectures that process sequential data by maintaining a hidden state across time steps. ([source](https://github.com/hunkim/DeepLearningZeroToAll/blob/master/lab-12-0-rnn_basics.ipynb))
- [Learning Rate Schedulers](https://awesome-repositories.com/f/education-learning-resources/educational-resources/systems-applied-computing/machine-learning-education/learning-rate-schedulers.md) — Provides mechanisms to adjust the learning rate during training to improve convergence stability. ([source](https://github.com/hunkim/DeepLearningZeroToAll/blob/master/lab-07-1-learning_rate_and_evaluation.py))
- [Softmax Regression](https://awesome-repositories.com/f/education-learning-resources/educational-resources/systems-applied-computing/machine-learning-education/softmax-regression.md) — Implements multi-class classification models using softmax regression theory and practical code. ([source](https://github.com/hunkim/DeepLearningZeroToAll/blob/master/lab-06-2-softmax_zoo_classifier.py))

### Part of an Awesome List

- [LSTM Architectures](https://awesome-repositories.com/f/awesome-lists/ai/recurrent-neural-networks/lstm-architectures.md) — Constructs recurrent architectures utilizing LSTM cells for character-level sequence prediction. ([source](https://github.com/hunkim/DeepLearningZeroToAll/blob/master/lab-12-2-char-seq-rnn.py))
- [Learning and Reference](https://awesome-repositories.com/f/awesome-lists/ai/learning-and-reference.md) — TensorFlow basic tutorials.
- [Deep Learning Lectures](https://awesome-repositories.com/f/awesome-lists/learning/deep-learning-lectures.md) — Foundational deep learning lecture series and code.

### Scientific & Mathematical Computing

- [Softmax Normalization](https://awesome-repositories.com/f/scientific-mathematical-computing/numerical-mathematical-foundations/statistics-probability/probability-distributions/softmax-normalization.md) — Applies softmax normalization to map raw model outputs to probability distributions for multi-class classification.

### System Administration & Monitoring

- [Training Metrics](https://awesome-repositories.com/f/system-administration-monitoring/logging/training-metrics.md) — Logs scalars and histograms during training to track loss and weight distributions over time. ([source](https://github.com/hunkim/DeepLearningZeroToAll/blob/master/lab-09-4-xor_tensorboard.py))