# fastai/numerical-linear-algebra **Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/fastai-numerical-linear-algebra).** 10,703 stars · 2,670 forks · Jupyter Notebook ## Links - GitHub: https://github.com/fastai/numerical-linear-algebra - Homepage: http://www.fast.ai/2017/07/17/num-lin-alg/ - awesome-repositories: https://awesome-repositories.com/repository/fastai-numerical-linear-algebra.md ## Topics `algorithms` `data-science` `deep-learning` `linear-algebra` `machine-learning` `numpy` `python` ## Description This project is a comprehensive library for numerical linear algebra and scientific computing, designed to provide optimized routines for matrix decomposition, statistical modeling, and high-performance data analysis. It serves as both a toolkit for solving complex linear systems and an educational resource for understanding the fundamental algorithms behind matrix factorizations and numerical solvers. The library distinguishes itself through a focus on randomized numerical linear algebra, utilizing probabilistic algorithms and approximate methods to perform dimensionality reduction and matrix decomposition with increased speed and memory efficiency. By incorporating techniques such as random sketching and truncated singular value decomposition, it enables the processing of large-scale datasets while preserving essential structural properties. To achieve high performance, the framework employs a variety of computational strategies, including hardware acceleration, parallel processing, and just-in-time compilation. It optimizes numerical stability and execution speed through rigorous floating-point analysis, block-based matrix operations, and memory layout configurations that improve data locality. These capabilities extend to statistical modeling, supporting linear regression solvers, regularization, and automatic differentiation for predictive analysis. The project is implemented as a collection of Jupyter Notebooks, providing an interactive environment for exploring and executing these numerical procedures. ## Tags ### Artificial Intelligence & ML - [Randomized SVD](https://awesome-repositories.com/f/artificial-intelligence-ml/dimensionality-reduction-techniques/randomized-svd.md) — The library approximates matrix decomposition by focusing on the largest singular values to reduce total computational time and memory usage. ([source](http://nbviewer.jupyter.org/github/fastai/numerical-linear-algebra/blob/master/nbs/2.%20Topic%20Modeling%20with%20NMF%20and%20SVD.ipynb)) - [Statistical Modeling Routines](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/algorithms/regression-models/statistical-modeling-routines.md) — The library constructs predictive models using linear regression and regularization techniques to analyze historical data and forecast future outcomes effectively. ([source](https://nbviewer.jupyter.org/github/fastai/numerical-linear-algebra/blob/master/nbs/0.%20Course%20Logistics.ipynb)) - [Hardware Acceleration](https://awesome-repositories.com/f/artificial-intelligence-ml/hardware-acceleration.md) — Offloads intensive matrix operations to graphics hardware using parallel processing to accelerate computational performance. - [Regression Solvers](https://awesome-repositories.com/f/artificial-intelligence-ml/linear-regression/regression-solvers.md) — The library computes regression coefficients using matrix factorization methods like Cholesky or SVD to solve systems of linear equations with high numerical efficiency. ([source](http://nbviewer.jupyter.org/github/fastai/numerical-linear-algebra/blob/master/nbs/6.%20How%20to%20Implement%20Linear%20Regression.ipynb)) - [Regularization Techniques](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/algorithms/regression-models/regularization-techniques.md) — The library adds penalty terms to training processes to prevent overfitting and ensure models remain simple enough to generalize well to new, unseen data. ([source](http://nbviewer.jupyter.org/github/fastai/numerical-linear-algebra/blob/master/nbs/5.%20Health%20Outcomes%20with%20Linear%20Regression.ipynb)) - [Robust Regression](https://awesome-repositories.com/f/artificial-intelligence-ml/regression-analysis/robust-regression.md) — The library applies robust loss functions that transition from quadratic to linear behavior to reduce the negative impact of noisy or extreme data points on predictions. ([source](http://nbviewer.jupyter.org/github/fastai/numerical-linear-algebra/blob/master/nbs/5.%20Health%20Outcomes%20with%20Linear%20Regression.ipynb)) - [Dimensionality Reduction](https://awesome-repositories.com/f/artificial-intelligence-ml/dimensionality-reduction.md) — Simplifies large datasets using dimensionality reduction techniques while preserving essential structural properties. ([source](https://nbviewer.jupyter.org/github/fastai/numerical-linear-algebra/blob/master/nbs/Project_ideas.txt)) - [Linear Regression](https://awesome-repositories.com/f/artificial-intelligence-ml/linear-regression.md) — The library fits linear models to data samples by minimizing the squared difference between predicted values and actual observations to establish clear trends. ([source](http://nbviewer.jupyter.org/github/fastai/numerical-linear-algebra/blob/master/nbs/5.%20Health%20Outcomes%20with%20Linear%20Regression.ipynb)) - [Tensor Computing Libraries](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/infrastructure/model-optimization-and-inference/hardware-and-acceleration/tensor-computing-libraries.md) — Executes matrix operations on graphics hardware using parallel processing to improve performance. ([source](http://nbviewer.jupyter.org/github/fastai/numerical-linear-algebra/blob/master/nbs/2.%20Topic%20Modeling%20with%20NMF%20and%20SVD.ipynb)) - [Representation Routines](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/infrastructure/model-optimization-and-inference/hardware-and-acceleration/tensor-computing-libraries/tensor-operations/sparse-tensor-representations/representation-routines.md) — Provides optimized sparse matrix representations to reduce memory usage and computational overhead for large datasets. - [Gradient Descent Algorithms](https://awesome-repositories.com/f/artificial-intelligence-ml/optimization-algorithms/gradient-descent-algorithms.md) — Updates model weights iteratively using gradient descent methods to minimize loss functions. ([source](http://nbviewer.jupyter.org/github/fastai/numerical-linear-algebra/blob/master/nbs/2.%20Topic%20Modeling%20with%20NMF%20and%20SVD.ipynb)) - [Feature Expansion](https://awesome-repositories.com/f/artificial-intelligence-ml/linear-regression/feature-expansion.md) — The library expands input datasets by calculating combinations of variables to capture complex non-linear relationships that standard linear models might otherwise miss. ([source](http://nbviewer.jupyter.org/github/fastai/numerical-linear-algebra/blob/master/nbs/5.%20Health%20Outcomes%20with%20Linear%20Regression.ipynb)) - [Numerical Stability Analysis](https://awesome-repositories.com/f/artificial-intelligence-ml/numerical-stability-techniques/numerical-stability-analysis.md) — Compares the precision and orthogonality of different factorization algorithms to minimize rounding errors. ([source](http://nbviewer.jupyter.org/github/fastai/numerical-linear-algebra/blob/master/nbs/8.%20Implementing%20QR%20Factorization.ipynb)) ### Scientific & Mathematical Computing - [Computational Libraries](https://awesome-repositories.com/f/scientific-mathematical-computing/high-performance-execution-environments/scientific-computing-platforms/computational-libraries.md) — Provides a comprehensive library of optimized mathematical routines for high-performance numerical tasks. - [Large-Scale Matrix Factorization](https://awesome-repositories.com/f/scientific-mathematical-computing/large-scale-matrix-factorization.md) — Decomposes massive datasets using randomized algorithms and truncated methods to extract hidden structures efficiently. - [Matrix Factorization Toolkits](https://awesome-repositories.com/f/scientific-mathematical-computing/matrix-factorization-toolkits.md) — Provides a toolkit for computing SVD, QR, and LU decompositions to solve complex linear systems. - [Linear Algebra](https://awesome-repositories.com/f/scientific-mathematical-computing/numerical-mathematical-foundations/linear-algebra.md) — Provides optimized routines for matrix decomposition including LU, QR, and SVD factorizations. ([source](https://nbviewer.jupyter.org/github/fastai/numerical-linear-algebra/blob/master/nbs/0.%20Course%20Logistics.ipynb)) - [Numerical Libraries](https://awesome-repositories.com/f/scientific-mathematical-computing/numerical-mathematical-foundations/numerical-libraries.md) — Offers a collection of algorithms for matrix factorization and high-performance numerical computation. - [High-Performance Scientific Computing](https://awesome-repositories.com/f/scientific-mathematical-computing/high-performance-execution-environments/scientific-computing-platforms/high-performance-scientific-computing.md) — Optimizes mathematical operations through vectorization and parallelization to achieve high-performance scientific computing. - [Scientific Computing](https://awesome-repositories.com/f/scientific-mathematical-computing/high-performance-execution-environments/scientific-computing-platforms/scientific-computing.md) — Acts as a scientific computing framework for building statistical models and optimizing matrix operations. - [Randomized Algorithms](https://awesome-repositories.com/f/scientific-mathematical-computing/linear-algebra-routines/randomized-algorithms.md) — Utilizes randomized numerical linear algebra to perform approximate matrix decomposition with increased speed and efficiency. - [Block-Based Matrix Decompositions](https://awesome-repositories.com/f/scientific-mathematical-computing/block-based-matrix-decompositions.md) — Implements block-based matrix operations to improve cache locality and minimize data movement during large-scale factorizations. - [Eigenvalue Approximation Methods](https://awesome-repositories.com/f/scientific-mathematical-computing/eigenvalue-approximation-methods.md) — Calculates eigenvalues and eigenvectors using iterative methods to solve ranking and decomposition problems. ([source](https://nbviewer.jupyter.org/github/fastai/numerical-linear-algebra/blob/master/nbs/0.%20Course%20Logistics.ipynb)) - [Vectorized Array Operations](https://awesome-repositories.com/f/scientific-mathematical-computing/high-performance-execution-environments/scientific-computing-platforms/scientific-computing/vectorized-array-operations.md) — Utilizes vectorized array operations to perform simultaneous arithmetic on multiple data elements for high-performance execution. - [Linear Algebra Routines](https://awesome-repositories.com/f/scientific-mathematical-computing/linear-algebra-routines.md) — Implements LU factorization to split matrices into triangular components for solving linear systems. ([source](https://nbviewer.jupyter.org/github/fastai/numerical-linear-algebra/blob/master/nbs/3.%20Background%20Removal%20with%20Robust%20PCA.ipynb)) - [Iterative Solvers](https://awesome-repositories.com/f/scientific-mathematical-computing/numerical-mathematical-foundations/optimization-solvers/iterative-solvers.md) — Implements iterative solvers for solving complex linear systems with high precision. ([source](https://nbviewer.jupyter.org/github/fastai/numerical-linear-algebra/blob/master/nbs/Project_ideas.txt)) - [Principal Component Analysis](https://awesome-repositories.com/f/scientific-mathematical-computing/linear-algebra-routines/principal-component-analysis.md) — Factors matrices into low-rank and sparse components to separate redundant information from data. ([source](https://nbviewer.jupyter.org/github/fastai/numerical-linear-algebra/blob/master/nbs/3.%20Background%20Removal%20with%20Robust%20PCA.ipynb)) - [QR Factorization](https://awesome-repositories.com/f/scientific-mathematical-computing/linear-algebra-routines/qr-factorization.md) — Decomposes matrices into orthogonal and upper triangular components to simplify complex linear algebra calculations. ([source](http://nbviewer.jupyter.org/github/fastai/numerical-linear-algebra/blob/master/nbs/8.%20Implementing%20QR%20Factorization.ipynb)) - [Stability Analysis](https://awesome-repositories.com/f/scientific-mathematical-computing/numerical-mathematical-foundations/arithmetic-number-types/floating-point-handlers/stability-analysis.md) — Performs rigorous floating-point stability analysis to ensure precision and minimize rounding errors in numerical computations. - [Sparse Data Processing](https://awesome-repositories.com/f/scientific-mathematical-computing/sparse-data-processing.md) — Manages and manipulates large-scale sparse datasets by focusing on non-zero elements to improve computational speed. - [Parallel Processing](https://awesome-repositories.com/f/scientific-mathematical-computing/high-performance-execution-environments/high-performance-and-parallel-computing/parallel-processing.md) — Distributes heavy matrix operations across multiple processor cores to reduce total execution time. ([source](http://nbviewer.jupyter.org/github/fastai/numerical-linear-algebra/blob/master/nbs/1.%20Why%20are%20we%20here.ipynb)) - [Approximate Algorithms](https://awesome-repositories.com/f/scientific-mathematical-computing/numerical-mathematical-foundations/algorithms-and-complexity/approximate-algorithms.md) — Uses probabilistic methods to trade off precision for significant gains in computation speed and memory efficiency. ([source](http://nbviewer.jupyter.org/github/fastai/numerical-linear-algebra/blob/master/nbs/1.%20Why%20are%20we%20here.ipynb)) - [Automatic Differentiation](https://awesome-repositories.com/f/scientific-mathematical-computing/numerical-mathematical-foundations/automatic-differentiation.md) — Computes derivatives of complex mathematical functions automatically to facilitate optimization. ([source](http://nbviewer.jupyter.org/github/fastai/numerical-linear-algebra/blob/master/nbs/2.%20Topic%20Modeling%20with%20NMF%20and%20SVD.ipynb)) - [Numerical Accuracy Management](https://awesome-repositories.com/f/scientific-mathematical-computing/numerical-mathematical-foundations/numerical-libraries/numerical-accuracy-management.md) — Analyzes floating point arithmetic and algorithm stability to manage the impact of rounding errors. ([source](http://nbviewer.jupyter.org/github/fastai/numerical-linear-algebra/blob/master/nbs/1.%20Why%20are%20we%20here.ipynb)) ### Education & Learning Resources - [Educational Resources](https://awesome-repositories.com/f/education-learning-resources/technical-domain-education/technical-academic-domains/theoretical-cs-foundations/computer-science-fundamentals/mathematical-foundations/linear-algebra/educational-resources.md) — Serves as an educational resource for understanding fundamental algorithms behind matrix factorizations and numerical solvers. - [Analysis Routines](https://awesome-repositories.com/f/education-learning-resources/matrix-operations/analysis-routines.md) — Applies matrix operations and decompositions to extract patterns from complex datasets for signal reconstruction and analysis. ([source](https://cdn.jsdelivr.net/gh/fastai/numerical-linear-algebra@master/README.md)) ### Programming Languages & Runtimes - [Arithmetic Broadcasting](https://awesome-repositories.com/f/programming-languages-runtimes/language-features-paradigms/language-features/array-operations/arithmetic-broadcasting.md) — Implements array broadcasting to automatically expand dimensions during arithmetic operations between arrays of different shapes. ([source](http://nbviewer.jupyter.org/github/fastai/numerical-linear-algebra/blob/master/nbs/4.%20Compressed%20Sensing%20of%20CT%20Scans%20with%20Robust%20Regression.ipynb)) - [Management Utilities](https://awesome-repositories.com/f/programming-languages-runtimes/programming-utilities/data-structure-type-helpers/data-structures/specialized-memory-formats/sparse-data-structures/management-utilities.md) — Provides specialized management utilities for processing sparse matrices efficiently in large-scale systems. ([source](http://nbviewer.jupyter.org/github/fastai/numerical-linear-algebra/blob/master/nbs/1.%20Why%20are%20we%20here.ipynb)) ### Security & Cryptography - [Random Sketching](https://awesome-repositories.com/f/security-cryptography/secure-randomizers/random-sketching.md) — Reduces the dimensionality of large datasets by projecting them onto a smaller random matrix. ([source](http://nbviewer.jupyter.org/github/fastai/numerical-linear-algebra/blob/master/nbs/6.%20How%20to%20Implement%20Linear%20Regression.ipynb)) ### Data & Databases - [Matrix Computation Optimizers](https://awesome-repositories.com/f/data-databases/batch-processing/batch-matrix-multiplication-utilities/matrix-multiplication-utilities/matrix-computation-optimizers.md) — Improves performance and numerical stability using block matrix multiplication and advanced solvers. ([source](https://nbviewer.jupyter.org/github/fastai/numerical-linear-algebra/blob/master/nbs/3.%20Background%20Removal%20with%20Robust%20PCA.ipynb)) - [Data Locality Optimizers](https://awesome-repositories.com/f/data-databases/query-optimizations/data-layout-optimizers/data-locality-optimizers.md) — Structures computations to minimize data movement between memory hierarchies for quicker retrieval. ([source](http://nbviewer.jupyter.org/github/fastai/numerical-linear-algebra/blob/master/nbs/1.%20Why%20are%20we%20here.ipynb)) ### Software Engineering & Architecture - [Just-in-Time Compilers](https://awesome-repositories.com/f/software-engineering-architecture/function-execution-engines/just-in-time-compilers.md) — Transforms high-level code into optimized machine instructions using just-in-time compilation. ([source](http://nbviewer.jupyter.org/github/fastai/numerical-linear-algebra/blob/master/nbs/5.%20Health%20Outcomes%20with%20Linear%20Regression.ipynb)) - [Memory Layout Optimizations](https://awesome-repositories.com/f/software-engineering-architecture/memory-layout-optimizations.md) — Configures matrix storage as row-major or column-major to align data access patterns with algorithm execution. ([source](http://nbviewer.jupyter.org/github/fastai/numerical-linear-algebra/blob/master/nbs/5.%20Health%20Outcomes%20with%20Linear%20Regression.ipynb)) - [Computational Optimizers](https://awesome-repositories.com/f/software-engineering-architecture/performance-reliability/performance-optimization/computational-efficiency/computational-graph-optimizers/computational-optimizers.md) — Accelerates matrix operations through vectorization, parallelization, and just-in-time compilation. ([source](https://nbviewer.jupyter.org/github/fastai/numerical-linear-algebra/blob/master/nbs/0.%20Course%20Logistics.ipynb))