zotroneneis/machine_learning_basics
Machine Learning Basics
This project is a collection of foundational machine learning algorithms and tools implemented from scratch in Python. It serves as a library of core implementations for regression, classification, and clustering models, designed to demonstrate the underlying mathematical structures of these algorithms without relying on high-level machine learning frameworks.
The project focuses on the manual implementation of algorithmic logic, including neural networks with forward propagation and weight updates, as well as various supervised and unsupervised learning models. It utilizes NumPy for vectorization to perform matrix calculations and mathematical operations on large datasets.
The toolkit covers a broad range of capabilities, including dimensionality reduction via Principal Component Analysis and data preprocessing for numerical and image datasets. The algorithmic implementations span linear and Bayesian regression, K-Means clustering, and multiple classification methods such as Support Vector Machines, Decision Trees, and K-Nearest Neighbors.
The project is delivered as a series of Jupyter Notebooks.
Features
- From-Scratch ML Model Implementations - Provides foundational implementations of machine learning models built from first principles without relying on high-level frameworks.
- Machine Learning Education - Serves as an educational resource for learning the mathematics of ML by building algorithms from scratch.
- Centroid-Based Clustering - Implements K-Means clustering by iteratively refining center coordinates based on the mean of assigned points.
- Classification Trees - Implements decision trees specifically for categorizing instances into discrete classes through binary splits.
- Gradient-Based Parameter Updates - Implements weight adjustment logic using gradient descent to minimize prediction error.
- Hyperplane Margin Maximization - Implements the mathematical logic for maximizing margins in Support Vector Machines.
- K-Means Clustering - Implements centroid-based partitioning of unlabeled data into a specified number of clusters using iterative refinement.
- K-Nearest Neighbor Classifiers - Implements supervised learning models that assign classes based on the majority vote of the closest training samples.
- Linear Regression - Implements fundamental statistical methods for modeling relationships between variables using linear equations.
- Bayesian Regressions - Implements probabilistic linear models that treat parameters as distributions to quantify prediction uncertainty.
- Logistic Regression Models - Implements algorithms for predicting binary outcomes using the sigmoid function and weight optimization.
- Machine Learning Implementations - Provides a comprehensive collection of core machine learning algorithms implemented manually in Python.
- Forward Propagation Engines - Implements forward propagation logic using matrix multiplication to calculate layer outputs.
- Neural Network Implementations - Implements an artificial neural network from scratch, demonstrating forward propagation and weight updates.
- Supervised Classification - Implements supervised classification and regression models including Decision Trees and SVMs.
- Support Vector Machines - Implements support vector machines to establish optimal decision boundaries that maximize the margin between classes.
- Unsupervised Learning - Implements unsupervised learning algorithms like K-Means to discover patterns in unlabeled data.
- Educational Python Implementations - Provides algorithmic implementations in pure Python to ensure the internal logic and mathematics are transparent and readable.
- Softmax Regression - Provides an implementation of softmax regression for multi-class classification probability prediction.
- NumPy-Based Linear Algebra Implementations - Uses NumPy for vectorization and matrix manipulations to perform efficient mathematical operations on datasets.
- Perceptrons - Implements a basic linear classifier that learns a weight vector to separate two classes of data.
- Posterior Distribution Estimators - Implements Bayesian linear regression by calculating posterior probabilities for linear coefficients using prior beliefs.
- Dimensionality Reduction - Provides mathematical techniques for reducing high-dimensional data into lower-dimensional spaces based on variance.
- Principal Component Analysis - Implements Principal Component Analysis to reduce dimensionality by transforming variables into uncorrelated components.
