joelgrus/data-science-from-scratch

Features

• Data Science Algorithms - Implements fundamental machine learning and data science algorithms using basic programming primitives.
• Machine Learning Implementations - Provides code-based implementations of core supervised and unsupervised machine learning algorithms using basic programming primitives.
• Python Machine Learning Libraries - Implements foundational machine learning algorithms and data science tools from scratch using Python.
• Decision Trees - Constructs tree-based models using splits and leaves to classify data attributes.
• Feed-Forward Neural Networks - Implements the fundamental architecture of neural networks where data flows in a single direction from input to output.
• Linear Regression - Implements statistical methods for modeling relationships between variables using linear equations and gradient descent.
• Distributed Data Processing Engines - Implements distributed data processing systems using map-reduce techniques to handle large datasets.
• MapReduce Processing Engines - Provides a framework for executing batch computations by mapping and reducing records into aggregated results.
• Linear Algebra Libraries - Provides foundational matrix and vector operation primitives for mathematical modeling and scientific computing.
• Linear Algebra Routines - Implements fundamental linear algebra operations, including matrix and vector calculations required for scientific computing.
• Matrix Manipulations - Extracts specific rows and columns or generates identity matrices from custom grids.
• Linear Algebra - Executes fundamental vector and matrix calculations, such as dot products, for mathematical modeling.
• Statistical Analysis Libraries - Provides a toolkit for applying probability models and statistical inference to derive insights from datasets.
• Statistical Metric Calculators - Computes probability and hypothesis tests to analyze data distributions and trends.
• Statistical Analysis Libraries - Provides a toolkit for computing probability distributions, hypothesis tests, and central tendency metrics.
• Hypothesis Testing - Provides statistical procedures and tools for executing hypothesis tests to determine the significance of observations.
• Gradient Computation - Provides tools for calculating function gradients to support model training and optimization via backpropagation.
• Numerical Gradient Approximations - Implements numerical gradient approximations using finite difference methods to estimate function slopes.
• Iterative Parameter Optimizations - Provides logic for repeatedly updating model weights using loss functions and gradient descent to fit data.
• K-Nearest Neighbor Classifiers - Implements a supervised learning model that assigns classes based on the majority vote of the closest training samples.
• Linear Model Performance Metrics - Measures linear fit quality using squared errors and the R-squared coefficient.
• Logistic Regression Models - Implements algorithms for predicting binary outcomes using the sigmoid function and weight optimization.
• Naive Bayes Classifiers - Implements probabilistic classification models based on statistical feature distributions to categorize text.
• Natural Language Processing - Provides foundational techniques for analyzing and processing human language data.
• Natural Language Processing Implementations - Provides reference implementations for text analysis, n-gram generation, and text classification.
• N-Gram Generators - Implements tools for generating contiguous word sequences based on n-gram sampling from source text.
• Gradient Descent Algorithms - Implements iterative optimization algorithms that update model parameters by moving in the direction of the negative gradient.
• Perceptrons - Implements basic linear classifiers that learn a weight vector to separate two classes of data.
• Performance Metrics - Provides tools for calculating essential statistical performance indicators like accuracy, precision, and recall.
• Popularity-Based Recommendations - Suggests items based on global frequency while filtering out items already owned.
• Recommender Systems - Implements algorithms designed to predict user preferences and suggest relevant items based on historical patterns.
• User-to-User Similarity - Identifies similar user profiles using cosine similarity to suggest items based on peer preferences.
• Data Visualization - Provides tools to render numeric datasets into visual formats like charts and graphs.
• Distribution Histograms - Generates frequency histograms to visualize the distribution of numeric data.
• Graph Libraries - Implements graph algorithms and data structures for network analysis and centrality calculations.
• Standard Deviation Calculators - Computes the standard deviation of a dataset to measure value dispersion.
• Central Tendency Measures - Computes mean, median, and mode to identify the center of a distribution.
• Correlation Matrices - Generates correlation matrices to identify linear relationships between multiple data vectors.
• Discrete Event Simulators - Generates outcomes for Bernoulli and Binomial trials to model discrete random events.
• Inverse Cumulative Distribution Functions - Finds the value for a given probability in a normal distribution using binary search.
• Network Graph Analysis - Provides capabilities to study connections between entities in a network to identify clusters and influence.
• Normal Distribution Probability Estimation - Predicts the likelihood that a value falls within a specific range of a Gaussian distribution.
• Normal Distribution Bounds - Calculates boundaries that contain a specified probability for a normal distribution.
• Shortest Path Algorithms - Implements algorithms to calculate the most efficient path between nodes in a graph.
• Distribution Function Calculators - Implements functions for computing cumulative distribution functions and their inverses for uniform and normal spreads.
• P-Value Calculations - Determines the probability of observing extreme values in either direction under a normal distribution.
• Parameter Estimation - Infers population parameters, such as mean and standard deviation, from sample data using binomial distributions.
• Quantile and Percentile Calculators - Provides mathematical tools for dividing sorted data into equal groups via quartiles and percentiles.
• Dot Product Computation - Computes dot products and Euclidean distances to analyze geometric relationships between vectors.
• Vector Operations - Calculates element-wise addition, subtraction, and means for numeric vectors.
• Betweenness Centrality - Implements betweenness centrality to measure node influence via shortest path counts.
• Closeness Centrality - Implements closeness centrality to determine node importance based on path lengths.

Star history