Open-source alternatives to Tinygrad

This project is an educational platform and research toolkit designed to teach deep learning through a combination of mathematical theory, visual diagrams, and executable code. It provides a comprehensive environment for building, training, and evaluating neural networks, grounding complex concepts in interactive computational notebooks that allow for hands-on experimentation. The framework distinguishes itself by interleaving theoretical foundations—including linear algebra, calculus, and probability—with practical implementations across multiple industry-standard libraries. It supports flex

This repository serves as a comprehensive collection of reference implementations for the PyTorch machine learning library. It provides practical examples for building, training, and deploying deep learning models, functioning as a toolkit for developers to explore neural network architectures and training workflows. The project distinguishes itself by offering concrete demonstrations of complex machine learning operations, ranging from computer vision tasks like object detection and depth estimation to the training of large-scale transformer models. These examples illustrate how to implement

Einops is a tensor manipulation library that provides a framework-agnostic interface for reshaping, Einstein summation, and multi-dimensional array operations. It serves as an abstraction layer that works across NumPy, PyTorch, TensorFlow, and JAX, allowing for tensor transformations without changing the API. The library distinguishes itself through a declarative notation system that uses readable string patterns to describe tensor rearrangements and reductions. This approach includes an extended Einstein summation interface that supports multi-letter axis names and a named dimension mapping

Torch7 is a scientific computing environment and tensor computation library used for deep learning research and numerical analysis. It functions as a Lua-based framework for training neural networks and learning agents, providing a toolkit for implementing architectures and training through reinforcement learning algorithms. The project is distinguished by its tight integration with C, utilizing a binding layer to map high-level scripting to low-level C structures for direct memory access. It supports hardware-accelerated computation by offloading linear algebra and convolution operations to

CNTK is a deep learning toolkit used for the design, construction, and training of neural networks. It defines model architectures as computational graphs and optimizes network parameters using an automatic differentiation engine and stochastic gradient descent. The project emphasizes large scale model distribution, spreading training workloads across multiple hardware nodes and GPUs. It features specialized support for dynamic sequence handling, allowing filters to be convolved across both spatial and dynamic sequence axes to process data of variable lengths. The toolkit provides hardware-a

This project is a deep learning framework designed for constructing, training, and deploying neural networks across diverse hardware environments. It functions as a high-performance tensor computation library that provides both imperative and symbolic programming interfaces, allowing developers to balance flexible, step-by-step model building with the efficiency of compiled computation graphs. The framework distinguishes itself through a hybrid execution engine that integrates declarative graph compilation with imperative runtime logic. It supports scalable, distributed training across multip

This project is a comprehensive collection of educational examples and reference implementations for building vision and language models using PyTorch. It serves as a deep learning tutorial covering the end-to-end process of developing neural networks, from initial architecture definition to final production deployment. The repository provides detailed guides on implementing a wide range of domain-specific models, including convolutional neural networks for object detection and segmentation, as well as transformer and recurrent architectures for natural language processing. It emphasizes gene

This project serves as a comprehensive educational resource and technical guide for mastering deep learning through the PyTorch framework. It provides structured tutorials and practical code examples designed to teach core machine learning principles, ranging from fundamental tensor operations to the construction of complex neural network architectures. The repository distinguishes itself by bridging the gap between theoretical concepts and hands-on implementation. It covers the development of generative applications, such as image synthesis and style transfer, while offering guidance on opti

Deeplearning4j is a JVM-based deep learning framework and tensor computing library. It provides a computational graph engine for defining and executing deep learning workflows and mathematical operations within the Java Virtual Machine. The project includes a dedicated importer for loading and running pretrained models exported from Keras, TensorFlow, and ONNX formats. Its tensor computing capabilities are driven by a modular native C++ math core to execute high-performance linear algebra operations. The framework covers neural network training, deep learning model inference, and the constru

This project is a high-performance numerical computing library designed for large-scale scientific and machine learning workloads. It functions as an automatic differentiation framework and a just-in-time compilation engine, transforming high-level Python code into optimized machine instructions. By enforcing pure functional programming patterns and immutable array semantics, the library ensures that mathematical functions remain compatible with automated graph transformations and symbolic differentiation. The platform distinguishes itself through its distributed array computing capabilities,

This project is a comprehensive deep learning framework and educational platform designed for constructing, training, and evaluating neural network architectures. It provides a modular environment for building models through tensor operations and automatic differentiation, supporting a wide range of tasks from image classification and object detection to sequential data processing. Beyond its core technical capabilities, the project distinguishes itself by integrating professional career development resources directly into its learning ecosystem. It offers structured guidance, resume reviews,

JAX is a hardware-accelerated array library and automatic differentiation system for numerical computing. It provides a framework compatible with NumPy that extends array operations with a just-in-time compiler to transform Python functions into optimized kernels for execution on GPU and TPU accelerators. The system differentiates itself through the use of an XLA-based compiler and a single program multiple data sharding model. These capabilities allow the library to distribute large-scale computations across multiple hardware accelerators using both automatic parallelization and manual shard

xtensor is a C++ multidimensional array library for numerical computing that provides N-dimensional containers with an interface mirroring the NumPy API. It utilizes a lazy evaluation expression engine to defer numerical computations until assignment, which minimizes memory allocations and intermediate copies. The library features a foreign memory array adaptor that allows it to wrap external buffers, such as NumPy arrays, to perform numerical operations in-place without duplicating data. It further optimizes performance through lazy broadcasting and a system that manages the lifetime of temp

This project is a structured learning curriculum and technical reference for mastering deep learning with TensorFlow. It provides a comprehensive guide for building, training, and deploying neural networks, combining theoretical fundamentals with practical implementation examples. The repository distinguishes itself by covering the end-to-end machine learning workflow, from low-level tensor mathematics and linear algebra to the creation of complex model architectures. It includes specific guidance on developing data pipelines for diverse data types, such as images, text, and time-series seque

DeepSpeed is a distributed deep learning optimization library and framework designed for the training and inference of massive AI models. It serves as a model parallelism orchestrator and a toolkit for scaling large language models across multiple GPUs and compute nodes. The project distinguishes itself through 3D parallelism orchestration, which combines data, pipeline, and tensor parallelism. It utilizes ZeRO-based memory partitioning to eliminate redundant storage and employs CPU-offload memory management to move weights and optimizer states to system RAM. Additionally, it provides special

This project is a Rust interface for the PyTorch C++ library, serving as a deep learning framework and tensor computing library. It functions as a C++ API wrapper that enables the manipulation of multi-dimensional arrays and the execution of neural network architectures across CPU and GPU hardware accelerators. The library provides a TorchScript inference engine to load and execute just-in-time compiled models. It also supports Rust and Python interoperability, allowing for the creation of Python extensions that share tensor data through a common interface. The system covers deep learning mo

This project is an educational resource and pedagogical framework designed to teach the fundamental mechanics of neural networks and gradient-based optimization. It provides a series of tutorials and code examples that guide users through building deep learning models from scratch, focusing on the implementation of core mathematical primitives and the underlying logic of backpropagation. The project distinguishes itself by providing a custom automatic differentiation engine that tracks mathematical operations in a dynamic computational graph. By implementing reverse-mode automatic differentia

Theano is a Python mathematical expression compiler and symbolic math library used as a deep learning backend. It functions as a tensors computation framework that translates mathematical formulas into optimized C or CUDA code for high-performance computing. The system manages the definition and evaluation of complex math formulas using multi-dimensional arrays. It employs a symbolic expression graph and a lazy evaluation engine to optimize mathematical expressions before they are compiled into executable code. The framework provides automatic differentiation for calculating gradients of mat

PyTorch Lightning is a deep learning research framework that provides a structured environment for organizing machine learning code. It functions as a unified trainer orchestrator, centralizing the execution flow by managing the interaction between hardware resources, data loaders, and model components. By decoupling model architecture from training logic, the framework enables researchers to maintain clean, modular codebases that remain portable across different environments. The framework distinguishes itself through a hardware-agnostic abstraction layer that scales deep learning workloads

This project is a collection of PyTorch learning resources and educational guides designed to teach the construction and training of neural networks. It serves as a comprehensive deep learning tutorial covering various model architectures and practical implementation strategies. The resources provide specific guidance on implementing computer vision tasks, such as image classification and synthetic imagery generation, as well as reinforcement learning agents using value networks and experience replay. It also covers sequential data modeling through recurrent networks and generative modeling u

This project is a collection of educational examples and code for implementing deep learning architectures using the PyTorch framework. It serves as a tutorial and implementation guide for building various neural network architectures for machine learning tasks. The project provides practical implementations for computer vision, including image classification and neural style transfer, as well as natural language processing examples for building sequence models and language predictors. It also covers generative models using adversarial and variational networks to synthesize or transform visua

Zig is a general-purpose systems programming language designed for high-performance applications that require manual memory management and direct control over hardware resources. It prioritizes predictable execution by enforcing explicit control flow and requiring functions to accept explicit memory allocators, ensuring that all heap operations and logic paths remain visible to the developer. The language distinguishes itself through a powerful compile-time metaprogramming engine that allows for arbitrary code execution during the build process, enabling advanced reflection and the generation

V is a statically typed, compiled programming language designed for high-performance systems development. It prioritizes memory safety and execution speed by enforcing strict type checking and immutable defaults, while generating native machine code for multiple hardware architectures. The language is built around an integrated toolchain that includes a compiler, package manager, formatter, and testing utilities within a single executable, facilitating rapid development cycles. What distinguishes V is its focus on developer productivity and interoperability. It provides a direct interface for

This repository serves as a structured educational resource for machine learning and deep learning, providing a library of executable scripts and notebooks. It is designed to help users master the practical application of data processing, model evaluation, and neural network construction through annotated code samples and guided tutorials. The collection focuses on translating theoretical mathematical concepts into functional code, offering proven patterns for common tasks such as classification and regression. By providing curated examples of layer construction and training loops, the reposi

LeetCUDA is a collection of high-performance GPU kernel libraries focusing on memory optimization, activation functions, and attention mechanisms. It serves as a reference library for CUDA kernel implementations, ranging from basic element-wise operations to complex neural network components, and provides Python bindings to integrate these kernels into deep learning workflows. The project is distinguished by its focus on low-level hardware optimizations. This includes the use of tensor cores for half-precision matrix multiplication, asynchronous data pipelining with double buffering, and shar

micrograd is a scalar autograd engine and minimal neural network library. It implements a system for reverse-mode automatic differentiation over a dynamic graph of scalar operations to calculate gradients. The project includes a computation graph visualizer that generates representations of data flow and gradient propagation. It provides a set of tools for constructing and training multi-layer perceptrons using an API modeled after PyTorch. The library covers the fundamentals of backpropagation and neural network construction, specifically for binary classification tasks. This includes the i