Awesome GitHub RepositoriesModel Compilers

TVM is a machine learning compiler framework designed to convert deep learning models from various frameworks into optimized machine code. It functions as a cross-platform deployment engine that transforms high-level model definitions into efficient, hardware-specific binaries for diverse computing architectures. The system utilizes a multi-level compilation pipeline that decouples algorithm logic from hardware implementation through tensor-operator abstractions. It employs a graph-level intermediate representation to perform cross-operator optimizations and memory planning before lowering co

TensorRT-LLM is a platform and toolkit designed for compiling, optimizing, and serving transformer-based models on accelerated hardware. It functions as a framework that transforms machine learning models into efficient execution graphs, providing an engine to refine these models for specific hardware to maximize throughput and minimize latency during text generation. The project distinguishes itself through advanced execution strategies that manage the entire inference pipeline. It utilizes kernel-level fusion and static graph execution to optimize mathematical operations and computational f

This project provides a comprehensive technical guide and framework for engineering large-scale machine learning systems. It covers the full lifecycle of model development, focusing on the infrastructure and computational principles required to build, train, and serve generative AI models across distributed GPU clusters. The repository distinguishes itself by offering deep-dive tutorials and implementation strategies for complex system challenges. It emphasizes high-performance architectural primitives, such as collective communication orchestration, distributed tensor sharding, and static gr

PlaidML ist ein Deep-Learning-Compiler-Framework und eine plattformübergreifende Runtime, die darauf ausgelegt ist, Machine-Learning-Modelle auf einer Vielzahl von Hardware-Zielen auszuführen. Es fungiert als hardwareunabhängige Tensor-Engine, die Tensor-Modelle in ausführbaren Code übersetzt, wodurch Deep-Learning-Netzwerke auf verschiedenen Rechengeräten ausgeführt werden können, ohne spezifische Treiberabhängigkeiten zu benötigen. Das System ermöglicht die Ausführung von Modellen auf benutzerdefinierter oder eingeschränkter Hardware durch die Verwendung von JSON-Spezifikationen zur Definition der Gerätehardware. Es verwendet eine domänenspezifische Sprache zur Beschreibung von Tensor-Berechnungen und bietet eine mittlere Schicht, um verschiedene Machine-Learning-Frameworks mit seinem hardwareorientierten Compiler zu integrieren. Die Engine unterstützt eine Reihe von Tensor-Operationen, einschließlich Tensor-Kontraktionen mit Index-Constraints, mehrdimensionalen Faltungen mit konfigurierbaren Strides und Padding sowie elementweisen Operationen unter Verwendung von Broadcasting. Zudem enthält sie eine Test-Suite, um Ausführungsgeschwindigkeit und Effizienz über verschiedene Hardwarekomponenten und Umgebungen hinweg zu benchmarken.

IREE is an MLIR-based compiler toolchain and runtime designed to translate machine learning models from various frameworks into optimized binaries for execution across diverse hardware targets. It provides a unified pipeline to ingest models from PyTorch, TensorFlow, JAX, and ONNX, lowering them into a common intermediate representation for deployment on CPUs, GPUs, and bare-metal embedded systems. The project distinguishes itself through a bytecode virtual machine and a hardware abstraction layer that decouple high-level model logic from specific hardware instruction sets. It supports sophis

This project is a collection of technical guides and manuals for the Apache TVM compiler stack translated into Simplified Chinese. It provides translated documentation focusing on deep learning compilation and the transformation of machine learning models into optimized executable code. The documentation covers the use of hardware backend guides for deploying models across CPUs, GPUs, and specialized accelerators. It also includes references for intermediate representations and graph-level optimizations used to compile tensor programs.

zml is a machine learning model compiler and cross-platform inference engine that transforms model descriptions into optimized executable binaries for specific hardware accelerators. It functions as a model deployment toolkit and hardware-agnostic orchestrator, utilizing a tensor-based architecture definition to provide strong type checking during the compilation process. The project distinguishes itself through the ability to shard tensors and distribute large-scale AI workloads across a logical mesh of multiple devices. It further supports the remote model lifecycle by authenticating and do