# jack-cherish/machine-learning

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/jack-cherish-machine-learning).**

10,333 stars · 5,085 forks · Python

## Links

- GitHub: https://github.com/Jack-Cherish/Machine-Learning
- Homepage: https://cuijiahua.com/blog/ml/
- awesome-repositories: https://awesome-repositories.com/repository/jack-cherish-machine-learning.md

## Topics

`adaboost` `adaboost-algorithm` `decision-tree` `knn` `logistic` `machine-learning` `navie-bayes-algorithm` `python` `python3` `regression` `smo` `svm`

## Description

This project is a collection of supervised and unsupervised machine learning algorithms implemented from scratch using Python. It serves as an educational resource for studying model training, parameter optimization, and the implementation of core predictive models.

The library provides a variety of supervised learning tools, including linear and logistic regression, decision trees, and support vector machines. It also features unsupervised learning capabilities for discovering patterns in unlabeled datasets through clustering algorithms.

Broad capability areas include ensemble learning through bagging and boosting, a text classification workflow with support for Chinese text segmentation, and comprehensive model performance evaluation through error analysis and the visualization of decision boundaries. The project also covers data preprocessing tasks such as feature normalization, vectorization, and the parsing of tabular data.

## Tags

### Artificial Intelligence & ML

- [Machine Learning Implementations](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning-implementations.md) — Provides from-scratch Python implementations of core supervised learning algorithms like linear regression and SVMs. ([source](http://cuijiahua.com/blog/2017/11/ml_8_svm_1.html))
- [Categorical Classifiers](https://awesome-repositories.com/f/artificial-intelligence-ml/categorical-classifiers.md) — Provides multiple algorithms like Naive Bayes and SVMs to classify categorical data. ([source](https://cuijiahua.com/blog/ml/))
- [Classification Models](https://awesome-repositories.com/f/artificial-intelligence-ml/classification-models.md) — Implements multi-class classification capabilities using support vector models to categorize data into multiple distinct classes. ([source](http://cuijiahua.com/blog/2017/11/ml_9_svm_2.html))
- [Clustering Algorithms](https://awesome-repositories.com/f/artificial-intelligence-ml/clustering-algorithms.md) — Implements clustering algorithms to discover patterns and group similar unlabeled data points. ([source](https://cuijiahua.com/blog/ml/))
- [Decision Trees](https://awesome-repositories.com/f/artificial-intelligence-ml/decision-trees.md) — Implements recursive-partitioning decision trees using information gain and squared error minimization.
- [Regression Trees](https://awesome-repositories.com/f/artificial-intelligence-ml/decision-trees/regression-trees.md) — Implements decision trees for predicting continuous numerical outcomes by minimizing squared error. ([source](http://cuijiahua.com/blog/2017/12/ml_13_regtree_1.html))
- [Ensemble Learning](https://awesome-repositories.com/f/artificial-intelligence-ml/ensemble-learning.md) — Implements ensemble methods including bagging and boosting to improve predictive accuracy.
- [Bagging Ensembles](https://awesome-repositories.com/f/artificial-intelligence-ml/ensemble-learning/bagging-ensembles.md) — Implements bagging ensembles to reduce model variance and overfitting through bootstrap sampling.
- [K-Means Clustering](https://awesome-repositories.com/f/artificial-intelligence-ml/k-means-clustering.md) — Implements k-means clustering to group unlabeled data points by minimizing distance between samples and centroids. ([source](https://cuijiahua.com/blog/2021/04/ml-14.html))
- [Clustering Algorithms](https://awesome-repositories.com/f/artificial-intelligence-ml/k-means-clustering/clustering-algorithms.md) — Implements unsupervised learning algorithms like K-Means to discover patterns in unlabeled data.
- [K-Nearest Neighbor Classifiers](https://awesome-repositories.com/f/artificial-intelligence-ml/k-nearest-neighbor-classifiers.md) — Implements a k-nearest neighbors classifier to predict data categories based on majority vote of the closest samples. ([source](http://cuijiahua.com/blog/2017/11/ml_1_knn.html))
- [Linear and Logistic Regression](https://awesome-repositories.com/f/artificial-intelligence-ml/linear-and-logistic-regression.md) — Implements logistic regression algorithms for predicting binary outcomes using a sigmoid function. ([source](http://cuijiahua.com/blog/2017/11/ml_7_logistic_2.html))
- [Linear Regression](https://awesome-repositories.com/f/artificial-intelligence-ml/linear-regression.md) — Implements linear regression to calculate optimal coefficients by minimizing squared errors. ([source](http://cuijiahua.com/blog/2017/11/ml_11_regression_1.html))
- [Linear Regression Models](https://awesome-repositories.com/f/artificial-intelligence-ml/linear-regression-models.md) — Implements linear regression models to predict continuous numerical outcomes. ([source](https://cuijiahua.com/blog/ml/))
- [Logistic Regression Models](https://awesome-repositories.com/f/artificial-intelligence-ml/logistic-regression-models.md) — Implements logistic regression models with support for regularization and class weighting. ([source](http://cuijiahua.com/blog/2017/11/ml_7_logistic_2.html))
- [Recursive Partitioning Trees](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/algorithms/predictive-machine-learning-analytics/regression-ensembles/recursive-partitioning-trees.md) — Implements regression trees that split data into recursive partitions to predict continuous values. ([source](https://github.com/Jack-Cherish/Machine-Learning/blob/master/README-eng.md))
- [Boosting Algorithms](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/infrastructure/machine-learning-training/boosting-algorithms.md) — Implements boosting algorithms like AdaBoost that iteratively train models by re-weighting difficult samples. ([source](http://cuijiahua.com/blog/2017/11/ml_10_adaboost.html))
- [Binary Classification Training](https://awesome-repositories.com/f/artificial-intelligence-ml/model-training/binary-classification-training.md) — Implements binary classification using a logistic function to map input features to a binary outcome probability. ([source](http://cuijiahua.com/blog/2017/11/ml_6_logistic_1.html))
- [Naive Bayes Classifiers](https://awesome-repositories.com/f/artificial-intelligence-ml/naive-bayes-classifiers.md) — Predicts categories using probabilistic models based on the conditional independence of features. ([source](http://cuijiahua.com/blog/2017/11/ml_4_bayes_1.html))
- [SMO Algorithms](https://awesome-repositories.com/f/artificial-intelligence-ml/optimization-algorithms/smo-algorithms.md) — Implements the Sequential Minimal Optimization algorithm to train support vector machines.
- [Supervised Learning](https://awesome-repositories.com/f/artificial-intelligence-ml/supervised-learning.md) — Provides a variety of predictive models for labeled datasets, including regression and classification.
- [Support Vector Machines](https://awesome-repositories.com/f/artificial-intelligence-ml/support-vector-machines.md) — Implements support vector machines to separate data classes by finding an optimal maximizing hyperplane. ([source](http://cuijiahua.com/blog/2017/11/ml_8_svm_1.html))
- [Text Classifiers](https://awesome-repositories.com/f/artificial-intelligence-ml/text-classifiers.md) — Implements text classifiers to predict document categories by calculating class probabilities from trained models. ([source](http://cuijiahua.com/blog/2017/11/ml_5_bayes_2.html))
- [Unsupervised Learning](https://awesome-repositories.com/f/artificial-intelligence-ml/unsupervised-learning.md) — Implements unsupervised learning capabilities for discovering hidden patterns in unlabeled datasets.
- [Automated Feature Selection Tools](https://awesome-repositories.com/f/artificial-intelligence-ml/automated-feature-selection-tools.md) — Implements feature selection by calculating information gain and Shannon entropy to identify discriminative attributes. ([source](http://cuijiahua.com/blog/2017/11/ml_2_decision_tree_1.html))
- [Classifier Accuracy Metrics](https://awesome-repositories.com/f/artificial-intelligence-ml/classifier-accuracy-metrics.md) — Provides tools to measure classifier accuracy by calculating the percentage of incorrect predictions. ([source](http://cuijiahua.com/blog/2017/11/ml_1_knn.html))
- [Dataset Distribution Analysis](https://awesome-repositories.com/f/artificial-intelligence-ml/dataset-quality-analyzers/dataset-distribution-analysis.md) — Provides visual analysis of dataset distributions to evaluate data separability. ([source](http://cuijiahua.com/blog/2017/11/ml_8_svm_1.html))
- [Decision Boundary Visualizations](https://awesome-repositories.com/f/artificial-intelligence-ml/decision-boundary-visualizations.md) — Plots the separating boundaries between classes to evaluate how a classifier partitions feature space. ([source](http://cuijiahua.com/blog/2017/11/ml_6_logistic_1.html))
- [Decision Stumps](https://awesome-repositories.com/f/artificial-intelligence-ml/decision-trees/decision-stumps.md) — Implements single-level decision stumps to serve as weak learners for ensemble algorithms. ([source](http://cuijiahua.com/blog/2017/11/ml_10_adaboost.html))
- [Pruning Techniques](https://awesome-repositories.com/f/artificial-intelligence-ml/decision-trees/pruning-techniques.md) — Implements decision tree pruning to reduce model complexity and prevent overfitting. ([source](http://cuijiahua.com/blog/2017/12/ml_13_regtree_1.html))
- [Tree Visualizers](https://awesome-repositories.com/f/artificial-intelligence-ml/decision-trees/tree-visualizers.md) — Generates visual representations of the paths and splitting logic used by decision tree models. ([source](https://cuijiahua.com/blog/ml/page/2/))
- [Feature Scale Normalization](https://awesome-repositories.com/f/artificial-intelligence-ml/feature-scale-normalization.md) — Scales numeric features by mean and variance to ensure stable model convergence. ([source](http://cuijiahua.com/blog/2017/12/ml_12_regression_2.html))
- [Iterative Parameter Optimizations](https://awesome-repositories.com/f/artificial-intelligence-ml/iterative-parameter-optimizations.md) — Implements iterative parameter optimization using gradient ascent to maximize loss functions.
- [Kernel-Based Feature Mapping](https://awesome-repositories.com/f/artificial-intelligence-ml/kernel-based-feature-mapping.md) — Provides kernel-based feature mapping to project data into high-dimensional spaces for non-linear classification.
- [Regularization Techniques](https://awesome-repositories.com/f/artificial-intelligence-ml/linear-regression-models/regularization-techniques.md) — Implements Ridge and stepwise regression to reduce model complexity and prevent overfitting. ([source](https://cuijiahua.com/blog/ml/))
- [Text Classification](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/language-tools/text-classification.md) — Implements a text classification workflow utilizing Naive Bayes and feature vectorization.
- [Model Evaluation Metrics](https://awesome-repositories.com/f/artificial-intelligence-ml/model-evaluation-metrics.md) — Implements model evaluation metrics including confusion matrices, precision, recall, and ROC curves. ([source](http://cuijiahua.com/blog/2017/11/ml_10_adaboost.html))
- [Model Ensembling](https://awesome-repositories.com/f/artificial-intelligence-ml/model-optimization/compression-techniques/model-ensembling.md) — Implements model ensembling techniques to improve predictive accuracy and robustness. ([source](https://cuijiahua.com/blog/ml/))
- [AdaBoost Implementations](https://awesome-repositories.com/f/artificial-intelligence-ml/model-weight-management/dynamic-weight-updates/training-weight-adjustments/boosting-weight-updates/adaboost-implementations.md) — Implements AdaBoost to improve classifier performance by iteratively training weak learners. ([source](https://github.com/Jack-Cherish/Machine-Learning/blob/master/README-eng.md))
- [Text Tokenization](https://awesome-repositories.com/f/artificial-intelligence-ml/natural-language-processing/text-tokenization.md) — Segments raw text strings into lowercase word lists by removing non-alphanumeric characters. ([source](http://cuijiahua.com/blog/2017/11/ml_5_bayes_2.html))
- [Gradient Ascent Algorithms](https://awesome-repositories.com/f/artificial-intelligence-ml/optimization-algorithms/gradient-ascent-algorithms.md) — Implements gradient ascent to optimize model parameters by maximizing the defined loss function. ([source](http://cuijiahua.com/blog/2017/11/ml_7_logistic_2.html))
- [Weighted Regression](https://awesome-repositories.com/f/artificial-intelligence-ml/regression-analysis/weighted-regression.md) — Performs locally weighted regression using Gaussian kernels to reduce underfitting. ([source](http://cuijiahua.com/blog/2017/11/ml_11_regression_1.html))
- [Regression Scoring Evaluation](https://awesome-repositories.com/f/artificial-intelligence-ml/regression-scoring-evaluation.md) — Evaluates regression performance by computing the total squared difference between real and predicted values. ([source](http://cuijiahua.com/blog/2017/11/ml_11_regression_1.html))
- [Stochastic Gradient Ascent](https://awesome-repositories.com/f/artificial-intelligence-ml/stochastic-gradient-ascent.md) — Implements stochastic gradient ascent to reduce computational complexity during model optimization. ([source](http://cuijiahua.com/blog/2017/11/ml_7_logistic_2.html))
- [Text Feature Extraction](https://awesome-repositories.com/f/artificial-intelligence-ml/text-feature-extraction.md) — Transforms unstructured text into structured numerical features based on word frequency ranking. ([source](http://cuijiahua.com/blog/2017/11/ml_5_bayes_2.html))
- [Vocabulary Generators](https://awesome-repositories.com/f/artificial-intelligence-ml/vocabulary-management/tokenizer-vocabulary-merging/vocabulary-generators.md) — Implements vocabulary building to create master feature lists for text vectorization. ([source](http://cuijiahua.com/blog/2017/11/ml_5_bayes_2.html))

### Education & Learning Resources

- [Machine Learning Educational Resources](https://awesome-repositories.com/f/education-learning-resources/machine-learning-educational-resources.md) — Serves as an educational resource with practical implementations of core machine learning algorithms.
- [Entropy Calculators](https://awesome-repositories.com/f/education-learning-resources/technical-domain-education/technical-academic-domains/theoretical-cs-foundations/information-theory/entropy-calculators.md) — Measures dataset uncertainty by calculating the empirical entropy of class label distributions. ([source](http://cuijiahua.com/blog/2017/11/ml_2_decision_tree_1.html))

### Scientific & Mathematical Computing

- [Ordinary Least Squares](https://awesome-repositories.com/f/scientific-mathematical-computing/ordinary-least-squares.md) — Implements ordinary least squares to find the best-fitting line between input features and target values. ([source](http://cuijiahua.com/blog/2017/12/ml_12_regression_2.html))
- [Kernel-Based Feature Mapping](https://awesome-repositories.com/f/scientific-mathematical-computing/non-linear-state-estimation/kernel-based-feature-mapping.md) — Uses kernel functions to project non-linear data into higher-dimensional spaces for linear separation. ([source](http://cuijiahua.com/blog/2017/11/ml_8_svm_1.html))
- [Ridge Regression](https://awesome-repositories.com/f/scientific-mathematical-computing/ridge-regression.md) — Implements ridge regression with L2 norm penalties to reduce collinearity and prevent model overfitting. ([source](http://cuijiahua.com/blog/2017/12/ml_12_regression_2.html))

### Software Engineering & Architecture

- [Regression Trees](https://awesome-repositories.com/f/software-engineering-architecture/tree-traversal-algorithms/regression-trees.md) — Implements regression trees to estimate numerical outcomes by partitioning data into recursive segments. ([source](https://cdn.jsdelivr.net/gh/jack-cherish/machine-learning@master/README.md))

### Testing & Quality Assurance

- [Model Evaluation](https://awesome-repositories.com/f/testing-quality-assurance/model-testing/model-evaluation.md) — Provides tools for measuring model accuracy through error analysis and decision boundary visualization.

### Data & Databases

- [Feature Vectorizations](https://awesome-repositories.com/f/data-databases/feature-vectorizations.md) — Provides symmetric-matrix feature vectorization for transforming raw text and images into numerical formats.
- [Tabular Data Preprocessing](https://awesome-repositories.com/f/data-databases/tabular-data-frameworks/tabular-predictive-models/tabular-data-preprocessing.md) — Converts raw text files into feature matrices and label vectors for use in classifiers. ([source](http://cuijiahua.com/blog/2017/11/ml_1_knn.html))
- [Chinese Language Segmenters](https://awesome-repositories.com/f/data-databases/text-processing-utilities/text-extraction/text-segmentation/chinese-language-segmenters.md) — Processes Chinese sentences into distinct words to handle the absence of whitespace during tokenization. ([source](http://cuijiahua.com/blog/2017/11/ml_5_bayes_2.html))

### User Interface & Experience

- [Feature Interaction Visualizations](https://awesome-repositories.com/f/user-interface-experience/feature-interaction-visualizations.md) — Provides tools to visualize feature distributions and relationships using scatter plots. ([source](http://cuijiahua.com/blog/2017/11/ml_1_knn.html))