Awesome GitHub RepositoriesPositional Encoding Techniques

This project is an educational platform and research toolkit designed to teach deep learning through a combination of mathematical theory, visual diagrams, and executable code. It provides a comprehensive environment for building, training, and evaluating neural networks, grounding complex concepts in interactive computational notebooks that allow for hands-on experimentation. The framework distinguishes itself by interleaving theoretical foundations—including linear algebra, calculus, and probability—with practical implementations across multiple industry-standard libraries. It supports flex

Implementation of Denoising Diffusion Probabilistic Model in Pytorch

This project is a comprehensive educational curriculum and structured learning path covering the full lifecycle of large language models. It provides a guided progression through the theory, architecture, training, and deployment of these models. The curriculum includes specialized guides on transformer architecture, model training tutorials, and frameworks for designing autonomous agents. It also provides dedicated resources for studying model safety and ethics. The material covers a wide range of technical capabilities, including distributed training strategies, parameter-efficient fine-tu

The Annotated Transformer is an educational resource that provides annotated code implementations of the Transformer architecture for sequence-to-sequence tasks, built with PyTorch. It serves as a learning tool for understanding attention mechanisms, multi-head parallel attention, and scaled dot-product attention through executable examples that walk through each component of the model. The project covers the full Transformer pipeline, including stacked encoder-decoder layers with residual connections and layer normalization, sinusoidal positional encoding for order-aware representation, and

PlugNPlay-Modules is a collection of reusable PyTorch computer vision modules and deep learning architectural components. It provides a library of standardized building blocks for constructing neural networks, focusing on attention mechanisms, signal processing layers, and feature fusion modules. The project is distinguished by its extensive variety of attention primitives, covering spatial, channel, and temporal weighting, as well as specialized variants like deformable, frequency-enhanced, and linear-complexity attention. It also implements advanced signal processing tools within the neural

Dieses Projekt ist eine PyTorch-Implementierung eines aufmerksamkeitsbasierten (Attention-based) neuronalen Netzes, das für Deep-Learning-Aufgaben von Sequenz zu Sequenz entwickelt wurde. Es dient als Bibliothek zur Konstruktion von Deep-Learning-Sequenzmodellen, die Encoder- und Decoder-Strukturen nutzen, um natürliche Sprache und sequenzielle Daten zu verarbeiten. Die Implementierung konzentriert sich auf einen Multi-Head-Attention-Mechanismus, um diverse Beziehungen zwischen Tokens ohne Rekurrenz zu erfassen. Sie beinhaltet sinusoidale positionale Kodierung zur Wahrung der Sequenzreihenfolge sowie punktweise Feed-Forward-Netze zur unabhängigen Transformation von Token-Positionen. Die Architektur integriert schichtbasierte Normalisierung, um das Training zu stabilisieren und die Konvergenz zu beschleunigen. Sie bietet die notwendigen Komponenten für das Design neuronaler Netzwerkarchitekturen in den Bereichen Natural Language Processing und Sequenz-zu-Sequenz-Lernen.

Tiny Universe is an educational monorepo that delivers multiple independent implementations of core AI subsystems as self-contained Jupyter notebooks. It provides from-scratch constructions of foundational architectures including a complete Transformer model built from the original paper specification, a denoising diffusion probabilistic model for image generation, and a ReAct-style autonomous agent framework that equips an LLM with tools for planning and multi-step task execution. The project distinguishes itself by covering the full lifecycle of modern AI systems through hands-on implementa

DeiT ist ein PyTorch-Vision-Transformer-Framework, das für die Bildklassifizierung entwickelt wurde. Es implementiert eine Transformer-basierte Architektur, die Bilder als Sequenzen abgeflachter Patches unter Verwendung von Self-Attention-Layern und positionsbewusster Sequenzmodellierung anstelle von konvolutiven Filtern verarbeitet. Das Projekt konzentriert sich auf dateneffizientes Training durch ein Knowledge-Distillation-Framework. Dieses System ermöglicht es einem Studentenmodell, die Soft-Labels eines leistungsstarken Lehrermodells nachzuahmen, um Genauigkeit und Generalisierung zu verbessern, insbesondere beim Training mit kleineren Datensätzen. Die Bibliothek deckt den gesamten Entwicklungslebenszyklus ab, einschließlich Training zur Bildklassifizierung, Optimierung der Cross-Entropy-Loss-Funktion und Bereitstellung vortrainierter Gewichte für die Inferenz. Sie enthält zudem ein Benchmarking-Tool zur Bewertung der Modellleistung und -genauigkeit anhand von Standarddatensätzen.

Neuraloperator is a library for learning mappings between infinite-dimensional function spaces, serving as a tool to accelerate physics simulations and partial differential equation solving. It implements resolution-invariant models and spectral neural networks that can produce consistent predictions regardless of the input grid resolution or spatial discretization. The framework incorporates physics-informed neural networks that enforce physical constraints and differential equations through specialized loss functions. It utilizes Fourier transforms and spectral projections to process multid