lawlite19/MachineLearning_Python
MachineLearning Python
This is a Python machine learning library featuring a collection of core algorithms implemented from scratch to demonstrate foundational AI concepts. It provides a comprehensive toolkit for supervised learning, unsupervised learning, and neural network development.
The project is distinguished by its custom implementation of a neural network framework, which includes multi-layer perceptrons with backpropagation, gradient descent, and weight regularization. It also includes a specialized anomaly detection toolkit that identifies outliers and rare events using Gaussian probability distributions and F1-score optimization.
The library covers a broad range of capabilities, including supervised classification and regression, k-means clustering, and dimensionality reduction via principal component analysis. Additional functional areas include data preprocessing for numerical normalization, feature engineering through polynomial expansion, and computer vision tasks such as handwritten digit recognition and image color quantization.
Features
- Python Machine Learning Libraries - A comprehensive collection of core machine learning algorithms implemented from scratch in Python.
- Centroid-Based Clustering - Implements unsupervised grouping of data by iteratively updating central points.
- Dimensionality Reduction - Simplifies high-dimensional datasets by projecting data onto principal components that preserve maximum variance.
- Gradient-Based Learning - Implements weight updates in neural networks using gradients computed via backpropagation.
- K-Means Clustering - Groups unlabeled data by iteratively assigning points to the nearest centroid.
- Clustering Algorithms - Groups unlabeled data points into distinct clusters based on feature similarity.
- Linear Regression - Implements linear regression to predict continuous values using gradient descent and feature normalization.
- Logistic Regression Models - Performs binary classification by minimizing a convex cost function through optimization methods.
- Neural Network Frameworks - Implements a neural network framework featuring multi-layer perceptrons with backpropagation and regularization.
- Neural Network Implementations - Implements multi-layer neural networks using forward propagation and gradient descent for training.
- Neural Networks - Implements a neural network framework for creating multi-layer perceptrons with backpropagation.
- Gradient Descent Algorithms - Uses iterative gradient descent to minimize cost functions and optimize model weights.
- Supervised Classification - Provides a full workflow for building and training models to categorize data into distinct classes.
- Supervised Learning - Provides a set of supervised learning models including linear regression, logistic regression, and support vector machines.
- Support Vector Machines - Constructs classifiers that find the maximum margin hyperplane to separate two distinct classes.
- Unsupervised Learning - Implements unsupervised learning algorithms including k-means clustering and principal component analysis.
- Anomaly Detection - Provides statistical anomaly detection for identifying outliers and rare events in datasets.
- Toolkits - Ships a specialized anomaly detection toolkit using Gaussian distributions and F1-score optimization.
- Principal Component Analysis - Implements a full principal component analysis algorithm to reduce dimensionality while maximizing variance.
- Anomaly Detection Metrics - Measures the accuracy of anomaly detection using precision, recall, and F1-score.
- Gradient Checking - Validates backpropagation gradients by comparing analytic results against numerical approximations.
- Decision Threshold Calibration - Optimizes probability thresholds on cross-validation sets to maximize the F1-score for anomaly detection.
- Feature Scale Normalization - Scales numerical features to a consistent range using mean and standard deviation for faster model convergence.
- Probability Density Functions - Calculates probability density values using mean and variance to model normal data distributions.
- Kernel-Based Feature Mapping - Transforms data into higher-dimensional spaces using kernel functions to resolve non-linear patterns.
- Polynomial Feature Mapping - Transforms low-dimensional input features into polynomial combinations to capture non-linear relationships.
- Neural Network Construction - Implements the process of building neural networks through random weight initialization and automated gradient checking.
- Neural Network Regularization - Prevents model overfitting by adding a penalty term based on weight magnitudes to the cost function.
- Digit Recognition - Processes image pixel data to identify and recognize handwritten numeric characters.
- Kernel SVMs - Implements support vector machines that use kernel functions for non-linear separation.
- Weight Decay Regularization - Prevents overfitting by adding a penalty term proportional to the square of the weights to the cost function.
- Weight Initialization - Assigns small random values to weights to break symmetry and allow neurons to learn distinct features.
- Weight Regularization - Prevents model overfitting by adding a penalty term based on the square of the weights to the cost function.
- Gaussian Anomaly Detectors - Implements statistical outlier detection using Gaussian probability density functions.
- One-Vs-All Multi-class Classification - Solves multi-class classification problems by training a binary classifier for each distinct category.
- Non-Linear Classifiers - Uses linear, polynomial, and Gaussian kernels to separate non-linearly distributed datasets.
