Awesome GitHub RepositoriesModel Graph Optimizers

InsightFace is a comprehensive deep learning framework designed for face recognition, biometric identity verification, and feature extraction. It provides a specialized engine for one-to-one verification and one-to-many identification tasks, utilizing convolutional neural networks to transform raw image pixels into high-dimensional vector embeddings. The project includes a complete toolkit for detecting, aligning, and processing facial data to ensure consistent identity discrimination. Beyond core recognition, the platform distinguishes itself through an extensive model management and optimiz

Modular is a unified machine learning development platform designed for building, compiling, and deploying high-performance neural network models. It provides a comprehensive execution engine that supports both local and production-grade inference, enabling developers to manage the entire model lifecycle from initial architecture definition to scalable, containerized service deployment. The platform distinguishes itself through a hardware-agnostic runtime that abstracts diverse silicon architectures, allowing models to execute efficiently across varied compute environments. It includes a spec

This repository serves as a comprehensive collection of reference implementations for the PyTorch machine learning library. It provides practical examples for building, training, and deploying deep learning models, functioning as a toolkit for developers to explore neural network architectures and training workflows. The project distinguishes itself by offering concrete demonstrations of complex machine learning operations, ranging from computer vision tasks like object detection and depth estimation to the training of large-scale transformer models. These examples illustrate how to implement

Paddle is a deep learning framework designed for building, training, and deploying neural networks. It provides a platform for constructing models using tensor-based computations and supports both dynamic and static execution graphs to facilitate research and production workflows. The platform functions as a distributed machine learning system, enabling the scaling of training workloads across multiple nodes and hardware clusters. It includes a comprehensive toolkit for model deployment and optimization, allowing users to convert external model formats, compress trained models for resource-co

ONNX is an open-source standard for machine learning interoperability that provides a unified format for representing neural network models. By defining a common set of operators and a standardized file structure, it enables models to be shared, exported, and executed consistently across different training frameworks and software ecosystems. The project functions as an intermediate representation layer that decouples model development from deployment. It utilizes a language-neutral binary serialization format to store model structures and weights, ensuring that computational graphs remain por

This project is a cross-platform machine learning inference engine designed to execute pre-trained models across diverse operating systems and hardware environments. It functions as a standardized execution framework that manages the entire lifecycle of model inference, from loading and graph optimization to hardware-accelerated execution and generative sequence management. The runtime distinguishes itself through a highly modular architecture that decouples model logic from hardware-specific kernels. By utilizing an execution provider abstraction, it enables developers to offload computation

TensorRT ist eine Deep-Learning-Inferenz-Engine und ein Software Development Kit zur Optimierung und Bereitstellung neuronaler Netze für die Hochleistungsausführung auf NVIDIA GPUs. Es fungiert als GPU-Beschleunigungs-Framework, das Latenzzeiten reduziert und den Durchsatz für trainierte Modelle während der Produktion erhöht. Das Toolkit importiert Modelle aus dem Open Neural Network Exchange Format und transformiert diese in optimierte Engines. Es nutzt graphbasierte Modelloptimierung, Layer-Fusion-Kernel-Generierung und präzisionsbasierte Quantisierung, um Fließkomma-Gewichte in Formate mit geringerer Präzision zu konvertieren. Das Framework bietet Funktionen für die hardware-spezifische Engine-Serialisierung und unterstützt die Erweiterung der Inferenzfähigkeiten durch benutzerdefinierte Plugins für spezialisierte Schichten neuronaler Netze.

EffectiveTensorflow is a deep learning tutorial suite and learning resource designed for building models within the TensorFlow framework. It serves as a practical implementation guide and development manual for creating neural network architectures. The project provides curated instructions for prototyping custom operations and implementing conditional logic for recurrent and deep learning structures. It focuses on the transition from imperative prototyping to the optimization of symbolic execution graphs for hardware accelerators. The resource covers numerical stability management to preven

This project is a comprehensive collection of educational examples and reference implementations for building vision and language models using PyTorch. It serves as a deep learning tutorial covering the end-to-end process of developing neural networks, from initial architecture definition to final production deployment. The repository provides detailed guides on implementing a wide range of domain-specific models, including convolutional neural networks for object detection and segmentation, as well as transformer and recurrent architectures for natural language processing. It emphasizes gene

Paddle-Lite is a deep learning inference engine and edge computing runtime designed to execute trained models on mobile and edge devices. It provides a hardware-accelerated inference framework and a decoupled runtime with a minimal binary footprint to operate in resource-constrained environments without third-party dependencies. The project includes a model quantization tool for reducing precision and size via static and dynamic quantization, as well as a computation graph optimizer. These tools reduce latency and memory usage by fusing operators and pruning the model intermediate representat

ANE is an open-source framework for training neural networks directly on Apple's Neural Engine hardware, bypassing Apple's public Core ML toolchain through reverse-engineered private APIs. It provides low-level control over the ANE, enabling developers to compile custom compute graphs into binary kernels, partition transformer model layers into hardware-compatible subgraphs, and share GPU-allocated memory with the Neural Engine via zero-copy IOSurface buffers. The framework distinguishes itself by offering direct access to hardware performance counters and power telemetry for benchmarking thr

coremltools ist ein Konvertierungs-Toolkit und Übersetzer, der darauf ausgelegt ist, Machine-Learning-Modelle aus verschiedenen Frameworks in das Core ML-Format für die Ausführung auf Apple-Hardware zu transformieren. Es bietet eine Suite von Tools für die Migration von Gewichten und Architekturen aus externen Bibliotheken in ein bereitstellbares Modellformat. Das Projekt enthält ein Optimierungstool und eine programmatische Schnittstelle zur Bearbeitung von Modellgraphen und zur Modifikation von Metadaten, um die Leistung auf der Zielhardware zu verbessern. Es verfügt zudem über eine Validierungssuite, mit der Modellspezifikationen und die Kompatibilität von Operationen geprüft werden, um eine korrekte Ausführung innerhalb der Runtime sicherzustellen. Das Toolkit deckt ein breites Spektrum an Deployment-Funktionen ab, einschließlich der Bearbeitung von Modellgraphen, der Metadatenkonfiguration und der Kompatibilitätsprüfung gegen formale Formatspezifikationen.

Dieses Projekt ist eine umfassende Lehrressource und ein Kurs zum Aufbau neuronaler Netze mit PyTorch. Es deckt die grundlegenden Bausteine des Deep Learning ab, einschließlich Tensor-Manipulation, automatischer Differenzierung und der Konstruktion modularer Komponenten für neuronale Netze. Das Repository dient als technischer Leitfaden für verschiedene spezialisierte Bereiche. Es bietet Implementierungsdetails für Computer-Vision-Aufgaben wie Bildklassifizierung, Objekterkennung und semantische Segmentierung sowie Workflows für die Verarbeitung natürlicher Sprache (NLP) mit Transformern, rekurrenten Netzen und generativen Modellen. Zudem enthält es eine Referenz für generative KI, mit Fokus auf die Synthese von Bildern mittels Diffusionsmodellen und adversarialen Netzwerken. Das Material erstreckt sich auf Modelloptimierung und Deployment-Pipelines. Es behandelt Techniken zur Reduzierung der Modellgröße und zur Erhöhung der Inferenzgeschwindigkeit durch Quantisierung und den Export von Modellen in Formate wie ONNX und TensorRT. Weitere Kompetenzbereiche umfassen Data Engineering für paralleles Laden, Modellevaluierung mittels benutzerdefinierter Metriken und das Deployment von Open-Source Large Language Models. Das Projekt wird primär als eine Reihe von Jupyter Notebooks bereitgestellt.

onnxsim ist ein Optimierer für Deep-Learning-Graphen und ein Modell-Vereinfacher, der darauf ausgelegt ist, die Komplexität von ONNX-Berechnungsgraphen zu reduzieren. Er fungiert als Modell-Kompressor, der komplexe Operator-Sequenzen durch vereinfachte konstante Ausgaben ersetzt, um den operativen Overhead zu verringern. Das Projekt erreicht die Vereinfachung durch Constant-Folding-Inference, bei der Teilgraphen aus konstanten Operatoren durch vorab berechnete konstante Tensoren ersetzt werden. Es nutzt musterbasiertes Graph-Rewriting und statische Analyse von Berechnungsgraphen, um redundante Knoten oder nicht erreichbare Operationen zu identifizieren und zu entfernen. Das Tool deckt umfassende Modelloptimierungsfunktionen ab, einschließlich der Eliminierung von Operator-Redundanz und der Entfernung unnötiger Reshape- oder Identity-Knoten. Diese Prozesse straffen den Ausführungsfluss und reduzieren den Speicherbedarf des Modells.

ExecuTorch is a lightweight C++ runtime for deploying PyTorch models on mobile, embedded, and edge hardware. It provides an ahead-of-time compilation pipeline that exports, quantizes, and lowers model graphs into compact serialized programs, then executes them through a minimal runtime with hardware acceleration and on-device large language model inference capabilities. The project distinguishes itself through a hardware accelerator delegate system that partitions model subgraphs and offloads computation to specialized backends including NPUs, GPUs, and DSPs from Apple, Arm, Intel, MediaTek,

oneDNN is a library for deep learning acceleration that provides optimized building blocks for neural network training and inference. It manages tensor computation across CPU and GPU hardware, enabling the execution of high-performance primitives for model training and neural network inference optimization. The project distinguishes itself through hardware-specific kernel optimization and the use of just-in-time compilation to target specific processor instruction sets. It supports quantized neural network execution using both static and dynamic quantization to reduce memory usage and increas

FastDeploy is a high-performance deployment framework for large language models, vision models, and multimodal models. It provides the infrastructure to launch model services that process combined image, video, and text inputs, exposing these capabilities through a standardized, OpenAI-compatible API for chat and text completions. The project distinguishes itself through advanced inference pipeline engineering and GPU optimization. It employs speculative decoding, tensor parallelism, and a disaggregated execution model that separates prefill and decode phases across different hardware resourc

Minigo is a TensorFlow-based reinforcement learning engine designed to master the game of Go. It functions as a comprehensive system for training neural networks to predict board policies and game outcomes, utilizing a model trainer to generate self-play data and optimize weights. The project is distinguished by its ability to perform large-scale game simulations using Kubernetes to distribute worker nodes across CPU, GPU, and TPU hardware. It employs a Monte Carlo Tree Search implementation to identify optimal moves and supports specialized hardware acceleration, including inference on Edge

This project is a machine learning interoperability tool designed to translate models from various training frameworks into the standardized open neural network exchange format. It functions as a model deployment pipeline that enables consistent execution across diverse inference engines and hardware environments. The tool utilizes graph-based translation and an operator mapping layer to convert framework-specific mathematical functions into a common intermediate representation. It distinguishes itself through a pluggable converter architecture, which allows developers to register custom tran