Awesome GitHub RepositoriesModel Compilers

TVM is a machine learning compiler framework designed to convert deep learning models from various frameworks into optimized machine code. It functions as a cross-platform deployment engine that transforms high-level model definitions into efficient, hardware-specific binaries for diverse computing architectures. The system utilizes a multi-level compilation pipeline that decouples algorithm logic from hardware implementation through tensor-operator abstractions. It employs a graph-level intermediate representation to perform cross-operator optimizations and memory planning before lowering co

TensorRT-LLM is a platform and toolkit designed for compiling, optimizing, and serving transformer-based models on accelerated hardware. It functions as a framework that transforms machine learning models into efficient execution graphs, providing an engine to refine these models for specific hardware to maximize throughput and minimize latency during text generation. The project distinguishes itself through advanced execution strategies that manage the entire inference pipeline. It utilizes kernel-level fusion and static graph execution to optimize mathematical operations and computational f

This project provides a comprehensive technical guide and framework for engineering large-scale machine learning systems. It covers the full lifecycle of model development, focusing on the infrastructure and computational principles required to build, train, and serve generative AI models across distributed GPU clusters. The repository distinguishes itself by offering deep-dive tutorials and implementation strategies for complex system challenges. It emphasizes high-performance architectural primitives, such as collective communication orchestration, distributed tensor sharding, and static gr

PlaidML este un framework de compilare pentru deep learning și un runtime cross-platform conceput pentru a executa modele de machine learning pe o mare varietate de ținte hardware. Funcționează ca un motor de tensori agnostic față de hardware, care traduce modelele de tensori în cod executabil, permițând rețelelor de deep learning să ruleze pe diferite dispozitive de calcul fără a necesita dependențe specifice de drivere. Sistemul permite execuția modelelor pe hardware personalizat sau limitat prin utilizarea specificațiilor JSON pentru a defini hardware-ul dispozitivului. Utilizează un limbaj specific domeniului (DSL) pentru a descrie calculele tensoriale și oferă un strat intermediar pentru a integra diverse framework-uri de machine learning cu compilatorul său orientat pe hardware. Motorul suportă o gamă de operațiuni tensoriale, inclusiv contracții tensoriale cu constrângeri de index, convoluții multidimensionale cu stride-uri și padding configurabile, precum și operațiuni elementwise folosind broadcasting. Include, de asemenea, o suită de testare pentru a evalua viteza și eficiența execuției pe diferite componente hardware și medii.

IREE is an MLIR-based compiler toolchain and runtime designed to translate machine learning models from various frameworks into optimized binaries for execution across diverse hardware targets. It provides a unified pipeline to ingest models from PyTorch, TensorFlow, JAX, and ONNX, lowering them into a common intermediate representation for deployment on CPUs, GPUs, and bare-metal embedded systems. The project distinguishes itself through a bytecode virtual machine and a hardware abstraction layer that decouple high-level model logic from specific hardware instruction sets. It supports sophis

This project is a collection of technical guides and manuals for the Apache TVM compiler stack translated into Simplified Chinese. It provides translated documentation focusing on deep learning compilation and the transformation of machine learning models into optimized executable code. The documentation covers the use of hardware backend guides for deploying models across CPUs, GPUs, and specialized accelerators. It also includes references for intermediate representations and graph-level optimizations used to compile tensor programs.

zml is a machine learning model compiler and cross-platform inference engine that transforms model descriptions into optimized executable binaries for specific hardware accelerators. It functions as a model deployment toolkit and hardware-agnostic orchestrator, utilizing a tensor-based architecture definition to provide strong type checking during the compilation process. The project distinguishes itself through the ability to shard tensors and distribute large-scale AI workloads across a logical mesh of multiple devices. It further supports the remote model lifecycle by authenticating and do