trekhleb/homemade-machine-learning

Features

• Multilayer Perceptrons - Provides a from-scratch implementation of multilayer perceptrons with input, hidden, and output layers.
• Machine Learning Education - Serves as an educational resource for learning the mathematical foundations of ML through scratch implementations.
• Backpropagation - Implements the backpropagation algorithm to calculate gradients and update neural network weights.
• Classification - Implements multi-class classification using a one-vs-all approach to categorize inputs.
• K-Means Clustering - Provides a manual implementation of k-means clustering for unsupervised data partitioning.
• Logistic Regression Models - Implements logistic regression for binary classification by fitting data to a logistic curve.
• Machine Learning Implementations - Provides code-based reference implementations of core machine learning algorithms written from first principles in Python.
• Linear Regression Implementations - Provides an educational from-scratch implementation of linear regression using gradient descent.
• Gradient Optimization Techniques - Applies gradient optimization and regularization techniques to improve model convergence and accuracy.
• Forward Propagation Engines - Provides the computational logic for the forward pass in multilayer perceptrons.
• Neural Network Implementations - Builds multilayer perceptrons from scratch, focusing on the low-level mechanics of neural networks.
• Gradient Descent Algorithms - Implements gradient descent as the primary iterative optimization method for minimizing cost functions.
• Supervised Learning - Implements supervised learning workflows including linear and logistic regression for prediction and classification.
• Unsupervised Learning - Implements unsupervised learning workflows for clustering and anomaly detection in unlabeled data.
• Backpropagation Training - Implements the backpropagation algorithm from scratch to enable training of multilayer perceptrons.
• Model Prediction Logic - Provides feedforward propagation to identify the most probable class labels for input data.
• Anomaly Detection - Implements a Gaussian distribution-based system to identify outliers as low-probability observations.
• Feature Scale Normalization - Provides mean normalization utilities to ensure stable convergence during gradient descent training.
• Feature Expansion - Provides utilities for polynomial feature expansion to model non-linear relationships within linear regression.
• Polynomial Feature Mapping - Enables non-linear relationship modeling by transforming independent variables into polynomial features.
• Closed-Form Solvers - Implements a closed-form solver using the normal equation to compute optimal model parameters without iteration.
• Performance Metrics - Provides utilities to compute F1 score, precision, and recall for model performance measurement.
• Regularization Techniques - Implements L2 regularization to penalize large coefficients and prevent model overfitting.
• Clustering and Density Estimation - Implements unsupervised techniques including K-Means clustering and Gaussian density estimation for outlier detection.
• Weight Initialization - Includes utilities for initializing neural network weights to break symmetry during training.
• L2 Regularization - Includes L2 regularization (ridge penalty) to prevent overfitting by penalizing large coefficient magnitudes.
• Educational Notebooks - Provides interactive Jupyter notebooks that interleave Python code with mathematical explanations.
• Supervised Learning Examples - Offers practical examples of supervised learning, including linear/logistic regression and regularization.
• Data Science Projects - Implementations of machine learning algorithms from scratch.
• Educational Resources - Python implementations of algorithms with interactive demos.

Star history