Awesome GitHub RepositoriesModel Compilers

TVM is a machine learning compiler framework designed to convert deep learning models from various frameworks into optimized machine code. It functions as a cross-platform deployment engine that transforms high-level model definitions into efficient, hardware-specific binaries for diverse computing architectures. The system utilizes a multi-level compilation pipeline that decouples algorithm logic from hardware implementation through tensor-operator abstractions. It employs a graph-level intermediate representation to perform cross-operator optimizations and memory planning before lowering co

TensorRT-LLM is a platform and toolkit designed for compiling, optimizing, and serving transformer-based models on accelerated hardware. It functions as a framework that transforms machine learning models into efficient execution graphs, providing an engine to refine these models for specific hardware to maximize throughput and minimize latency during text generation. The project distinguishes itself through advanced execution strategies that manage the entire inference pipeline. It utilizes kernel-level fusion and static graph execution to optimize mathematical operations and computational f

This project provides a comprehensive technical guide and framework for engineering large-scale machine learning systems. It covers the full lifecycle of model development, focusing on the infrastructure and computational principles required to build, train, and serve generative AI models across distributed GPU clusters. The repository distinguishes itself by offering deep-dive tutorials and implementation strategies for complex system challenges. It emphasizes high-performance architectural primitives, such as collective communication orchestration, distributed tensor sharding, and static gr

PlaidML 是一个深度学习编译器框架和跨平台运行时，旨在在各种硬件目标上执行机器学习模型。它作为一个硬件无关的张量引擎，将张量模型翻译为可执行代码，从而允许深度学习网络在不同计算设备上运行，而无需特定的驱动程序依赖。 该系统通过使用 JSON 规范定义设备硬件，实现了在自定义或受限硬件上执行模型。它采用领域特定语言来描述张量计算，并提供中间层将各种机器学习框架与其硬件目标编译器集成。 该引擎支持一系列张量操作，包括带索引约束的张量收缩、带可配置步长和填充的多维卷积，以及使用广播的逐元素操作。它还包含一个测试套件，用于基准测试不同硬件组件和环境下的执行速度和效率。

IREE is an MLIR-based compiler toolchain and runtime designed to translate machine learning models from various frameworks into optimized binaries for execution across diverse hardware targets. It provides a unified pipeline to ingest models from PyTorch, TensorFlow, JAX, and ONNX, lowering them into a common intermediate representation for deployment on CPUs, GPUs, and bare-metal embedded systems. The project distinguishes itself through a bytecode virtual machine and a hardware abstraction layer that decouple high-level model logic from specific hardware instruction sets. It supports sophis

This project is a collection of technical guides and manuals for the Apache TVM compiler stack translated into Simplified Chinese. It provides translated documentation focusing on deep learning compilation and the transformation of machine learning models into optimized executable code. The documentation covers the use of hardware backend guides for deploying models across CPUs, GPUs, and specialized accelerators. It also includes references for intermediate representations and graph-level optimizations used to compile tensor programs.

zml is a machine learning model compiler and cross-platform inference engine that transforms model descriptions into optimized executable binaries for specific hardware accelerators. It functions as a model deployment toolkit and hardware-agnostic orchestrator, utilizing a tensor-based architecture definition to provide strong type checking during the compilation process. The project distinguishes itself through the ability to shard tensors and distribute large-scale AI workloads across a logical mesh of multiple devices. It further supports the remote model lifecycle by authenticating and do