# eriklindernoren/ML-From-Scratch

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/eriklindernoren-ml-from-scratch).**

30,849 stars · 5,186 forks · Python · mit

## Links

- GitHub: https://github.com/eriklindernoren/ML-From-Scratch
- awesome-repositories: https://awesome-repositories.com/repository/eriklindernoren-ml-from-scratch.md

## Topics

`data-mining` `data-science` `deep-learning` `deep-reinforcement-learning` `genetic-algorithm` `machine-learning` `machine-learning-from-scratch`

## Description

This project is an educational toolkit that provides implementations of fundamental machine learning algorithms built from scratch. By avoiding high-level library abstractions, it serves as a pedagogical reference for understanding the mathematical foundations and core mechanics of supervised learning, unsupervised learning, and reinforcement learning models.

The repository distinguishes itself through a modular approach to model construction, allowing users to build custom neural networks by chaining independent functional blocks. It covers a wide range of techniques, including gradient-based weight optimization, backpropagation through time for sequential data, and ensemble-based aggregation methods like boosting and bagging. These implementations rely on vectorized computation to perform linear algebra operations, providing a transparent view into how models learn from data.

The collection encompasses a broad capability surface, ranging from classic statistical methods and decision trees to complex deep learning architectures and clustering algorithms. It includes resources for training agents in dynamic environments, performing dimensionality reduction, and discovering patterns in unlabeled datasets. The project is structured as a comprehensive reference, with documentation and installation instructions provided to help users configure their local environments for experimentation.

## Tags

### Artificial Intelligence & ML

- [Machine Learning Toolkits](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning-toolkits.md) — Provides a collection of fundamental algorithms implemented from scratch to demonstrate core learning mechanics.
- [Supervised Learning](https://awesome-repositories.com/f/artificial-intelligence-ml/supervised-learning.md) — Trains models on labeled datasets to predict outcomes or classify observations. ([source](https://github.com/eriklindernoren/ML-From-Scratch#readme))
- [Clustering Algorithms](https://awesome-repositories.com/f/artificial-intelligence-ml/clustering-algorithms.md) — Partition datasets into clusters by iteratively assigning points to the nearest center and updating positions to minimize variance. ([source](https://github.com/eriklindernoren/ML-From-Scratch/blob/master/mlfromscratch/unsupervised_learning/k_means.py))
- [Deep Learning Architectures](https://awesome-repositories.com/f/artificial-intelligence-ml/deep-learning-architectures.md) — Builds custom deep learning models by stacking layers and activation functions. ([source](https://github.com/eriklindernoren/ML-From-Scratch#readme))
- [Multilayer Perceptrons](https://awesome-repositories.com/f/artificial-intelligence-ml/multilayer-perceptrons.md) — Solve complex classification problems by defining hidden layers and backpropagation logic to learn non-linear mappings from input data. ([source](https://github.com/eriklindernoren/ML-From-Scratch/blob/master/mlfromscratch/examples/multilayer_perceptron.py))
- [Random Forest Ensembles](https://awesome-repositories.com/f/artificial-intelligence-ml/random-forest-ensembles.md) — Improve predictive accuracy and reduce variance by combining the outputs of multiple decision trees trained on random data subsets. ([source](https://github.com/eriklindernoren/ML-From-Scratch/blob/master/mlfromscratch/supervised_learning/random_forest.py))
- [Recurrent Neural Networks](https://awesome-repositories.com/f/artificial-intelligence-ml/recurrent-neural-networks.md) — Implements backpropagation through time to train sequential models by unrolling recurrent structures.
- [Reinforcement Learning](https://awesome-repositories.com/f/artificial-intelligence-ml/reinforcement-learning.md) — Develops agents that learn to make sequences of decisions to maximize cumulative rewards. ([source](https://github.com/eriklindernoren/ML-From-Scratch#readme))
- [Unsupervised Learning](https://awesome-repositories.com/f/artificial-intelligence-ml/unsupervised-learning.md) — Discovers hidden structures and patterns in unlabeled data using clustering and dimensionality reduction. ([source](https://github.com/eriklindernoren/ML-From-Scratch#readme))
- [Ensemble Learning](https://awesome-repositories.com/f/artificial-intelligence-ml/ensemble-learning.md) — Combines multiple weak learners into robust models using boosting and bagging strategies.
- [Expectation-Maximization Models](https://awesome-repositories.com/f/artificial-intelligence-ml/expectation-maximization-models.md) — Group data points by iteratively calculating membership probabilities to optimize the fit of statistical distributions to the data.
- [Gradient Boosting](https://awesome-repositories.com/f/artificial-intelligence-ml/gradient-boosting.md) — Minimize prediction errors by combining multiple decision trees into a single ensemble model through iterative gradient descent. ([source](https://github.com/eriklindernoren/ML-From-Scratch/blob/master/mlfromscratch/supervised_learning/xgboost.py))
- [Logistic Regression Models](https://awesome-repositories.com/f/artificial-intelligence-ml/logistic-regression-models.md) — Predict the likelihood of binary outcomes by applying the sigmoid function and optimizing weights through gradient descent. ([source](https://github.com/eriklindernoren/ML-From-Scratch/blob/master/mlfromscratch/supervised_learning/logistic_regression.py))
- [Neural Network Architectures](https://awesome-repositories.com/f/artificial-intelligence-ml/neural-network-architectures.md) — Constructs custom neural networks by defining specific layers, activation functions, and loss functions. ([source](https://github.com/eriklindernoren/ML-From-Scratch/blob/master/mlfromscratch/deep_learning/neural_network.py))
- [Neural Network Frameworks](https://awesome-repositories.com/f/artificial-intelligence-ml/neural-network-frameworks.md) — Constructs complex neural network architectures by chaining independent functional blocks.
- [Clustering Suites](https://awesome-repositories.com/f/artificial-intelligence-ml/clustering-suites.md) — Provides a collection of density and centroid-based algorithms for grouping unlabeled data.
- [Computer Vision](https://awesome-repositories.com/f/artificial-intelligence-ml/computer-vision.md) — Recognizes visual patterns in images by training convolutional neural networks to extract features. ([source](https://github.com/eriklindernoren/ML-From-Scratch#readme))
- [Decision Trees](https://awesome-repositories.com/f/artificial-intelligence-ml/decision-trees.md) — Builds decision-based models by repeatedly splitting datasets into subsets based on feature thresholds.
- [Deep Learning Frameworks](https://awesome-repositories.com/f/artificial-intelligence-ml/deep-learning-frameworks.md) — Constructs custom neural network architectures to demonstrate backpropagation and gradient descent mechanics.
- [Generative Adversarial Networks](https://awesome-repositories.com/f/artificial-intelligence-ml/generative-adversarial-networks.md) — Creates synthetic images of handwritten digits by training generative adversarial networks. ([source](https://github.com/eriklindernoren/ML-From-Scratch#readme))
- [Naive Bayes Classifiers](https://awesome-repositories.com/f/artificial-intelligence-ml/naive-bayes-classifiers.md) — Classify data points by calculating the probability of class membership based on the statistical distribution of input features. ([source](https://github.com/eriklindernoren/ML-From-Scratch/blob/master/mlfromscratch/supervised_learning/naive_bayes.py))
- [Optimization Algorithms](https://awesome-repositories.com/f/artificial-intelligence-ml/optimization-algorithms.md) — Updates model parameters iteratively by calculating partial derivatives of the loss function.
- [Sequential Learning](https://awesome-repositories.com/f/artificial-intelligence-ml/sequential-learning.md) — Processes time-series data by using recurrent neural networks to capture temporal dependencies. ([source](https://github.com/eriklindernoren/ML-From-Scratch/blob/master/mlfromscratch/examples/recurrent_neural_network.py))
- [Autoencoders](https://awesome-repositories.com/f/artificial-intelligence-ml/autoencoders.md) — Reduces data complexity by training encoder and decoder networks to reconstruct original inputs. ([source](https://github.com/eriklindernoren/ML-From-Scratch/blob/master/mlfromscratch/unsupervised_learning/autoencoder.py))
- [Density-Based Clustering](https://awesome-repositories.com/f/artificial-intelligence-ml/density-based-clustering.md) — Identifies clusters of arbitrary shapes by analyzing local data density. ([source](https://github.com/eriklindernoren/ML-From-Scratch/blob/master/mlfromscratch/unsupervised_learning/dbscan.py))
- [Ensemble Learning Libraries](https://awesome-repositories.com/f/artificial-intelligence-ml/ensemble-learning-libraries.md) — Ships a suite of boosting and bagging implementations to improve predictive performance.
- [Evolutionary Algorithms](https://awesome-repositories.com/f/artificial-intelligence-ml/evolutionary-algorithms.md) — Optimizes neural network architectures and weights using evolutionary strategies. ([source](https://github.com/eriklindernoren/ML-From-Scratch#readme))
- [Perceptron Classifiers](https://awesome-repositories.com/f/artificial-intelligence-ml/perceptron-classifiers.md) — Create simple linear models by iteratively adjusting weights based on prediction errors to separate labeled data points. ([source](https://github.com/eriklindernoren/ML-From-Scratch/blob/master/mlfromscratch/supervised_learning/perceptron.py))
- [Association Rule Learning](https://awesome-repositories.com/f/artificial-intelligence-ml/association-rule-learning.md) — Discovers frequent patterns and relationships within transactional datasets using the Apriori algorithm. ([source](https://github.com/eriklindernoren/ML-From-Scratch#readme))
- [Boltzmann Machines](https://awesome-repositories.com/f/artificial-intelligence-ml/boltzmann-machines.md) — Represents complex data structures by training Boltzmann machines to learn underlying feature patterns. ([source](https://github.com/eriklindernoren/ML-From-Scratch/blob/master/mlfromscratch/unsupervised_learning/restricted_boltzmann_machine.py))
- [Dimensionality Reduction](https://awesome-repositories.com/f/artificial-intelligence-ml/dimensionality-reduction.md) — Identify linear combinations of features that maximize class separation to simplify datasets or improve classification performance. ([source](https://github.com/eriklindernoren/ML-From-Scratch/blob/master/mlfromscratch/supervised_learning/linear_discriminant_analysis.py))
- [Gaussian Mixture Models](https://awesome-repositories.com/f/artificial-intelligence-ml/gaussian-mixture-models.md) — Implements Gaussian mixture models to represent complex data distributions using expectation-maximization. ([source](https://github.com/eriklindernoren/ML-From-Scratch/blob/master/mlfromscratch/unsupervised_learning/gaussian_mixture_model.py))
- [Genetic Algorithms](https://awesome-repositories.com/f/artificial-intelligence-ml/genetic-algorithms.md) — Solves optimization problems by simulating natural selection processes to evolve candidate solutions. ([source](https://github.com/eriklindernoren/ML-From-Scratch#readme))
- [K-Medoids Clustering](https://awesome-repositories.com/f/artificial-intelligence-ml/k-medoids-clustering.md) — Groups data points by minimizing total dissimilarity to representative medoids. ([source](https://github.com/eriklindernoren/ML-From-Scratch/blob/master/mlfromscratch/unsupervised_learning/partitioning_around_medoids.py))
- [Numerical Computing Libraries](https://awesome-repositories.com/f/artificial-intelligence-ml/numerical-computing-libraries.md) — Performs mathematical operations on multidimensional arrays to accelerate linear algebra calculations.

### Education & Learning Resources

- [Educational Examples](https://awesome-repositories.com/f/education-learning-resources/educational-examples.md) — Provides a collection of machine learning models and algorithms built from scratch for educational purposes. ([source](https://github.com/eriklindernoren/ML-From-Scratch#readme))
- [Machine Learning Tutorials](https://awesome-repositories.com/f/education-learning-resources/machine-learning-tutorials.md) — Provides fundamental machine learning algorithm implementations built from scratch for educational purposes.
- [Algorithm References](https://awesome-repositories.com/f/education-learning-resources/algorithm-references.md) — Provides clean, readable code examples for standard statistical and neural network architectures.