WenDesi/lihang_book_algorithm
Lihang Book Algorithm
This is an educational Python implementation of every algorithm from Li Hang's textbook on statistical learning methods. The project provides a comprehensive collection of supervised learning algorithms covering classification, regression, and sequence modeling techniques, implemented from scratch for learning and reference purposes.
The repository covers a broad range of foundational machine learning methods, including decision trees built using the ID3 algorithm with information gain, ensemble boosting through AdaBoost that combines threshold-based weak learners, and probabilistic sequence modeling with hidden Markov models using forward-backward, Baum-Welch, and Viterbi algorithms. It also includes maximum entropy and softmax regression models for multi-class classification, logistic regression for binary outcomes, naive Bayes classifiers for text and high-dimensional data, k-nearest neighbors classification by majority vote, binary perceptron classifiers, and support vector machines that find maximum-margin separating hyperplanes.
Additional capabilities include weight decay regularization to prevent overfitting, numerical techniques for handling high-dimensional probability computations, and a C++ extension that accelerates computationally intensive training loops. The implementations are designed to be runnable and testable, with the code organized to mirror the textbook's chapter structure for easy cross-referencing.
