Awesome GitHub RepositoriesHardware-Specific Model Optimizations

AISystem is a comprehensive AI full-stack infrastructure project covering the entire pipeline from AI chip architecture to high-level training frameworks. It encompasses the development of AI compiler frameworks, inference engines, and distributed training orchestrators designed to coordinate workloads across a heterogeneous compute stack of CPUs, GPUs, and NPUs. The project focuses on the deep integration of software and hardware, employing software-hardware co-design to align tensor layouts with physical memory structures. It provides specialized capabilities for accelerating Transformer mo

DiffusionBee is a Stable Diffusion desktop client for macOS that functions as an AI image generator and editor. It allows for the local generation of images from text prompts and the management of diffusion models without requiring external cloud services or technical setup. The application includes a local diffusion model manager for importing and switching between custom trained model files to achieve specific artistic styles. It also features a system for tracking generation history and uploading assets to a public gallery. The software covers several image synthesis and manipulation work

TensorRT 是一个深度学习推理引擎和软件开发工具包，旨在优化和部署神经网络，以在 NVIDIA GPU 上实现高性能执行。它作为一个 GPU 加速框架，在生产部署期间减少延迟并提高训练模型的吞吐量。 该工具包从 Open Neural Network Exchange 格式导入模型，并将它们转换为优化的引擎。它利用基于图的模型优化、层融合内核生成和基于精度的量化，将浮点权重转换为低精度格式。 该框架提供了硬件特定引擎序列化的功能，并支持通过自定义插件扩展推理能力，以用于专门的神经网络层。

YOLOv10 is a PyTorch computer vision library and real-time vision framework designed for locating and identifying multiple objects in images and video streams. It functions as an end-to-end object detector that optimizes for high-speed deployment and detection precision. The project is distinguished by an NMS-free detection architecture that predicts a single bounding box per object, eliminating the need for non-maximum suppression post-processing to reduce inference latency. It further optimizes for edge hardware through scalable weights and a quantization-friendly structure that facilitates

This project is an AI-powered IDE extension and LLM coding assistant that provides a conversational interface for generating, refactoring, and debugging code. It functions as an AI agent framework and a Model Context Protocol client, connecting AI models to external data sources and tools to automate complex development tasks. The system is distinguished by its use of autonomous AI agents capable of multi-step task execution, including the ability to read files, modify code, and run terminal commands iteratively. It supports recursive agent orchestration through subagent delegation and employ

AI-on-the-edge-device is an edge AI meter digitizer and computer vision image processor designed to convert images of analog and digital utility meters into numeric values. It functions as an IoT gateway that runs neural network inference locally on hardware to monitor water, power, and gas readings. The system is distinguished by its ability to handle both analog pointers and digital digits through custom-trained neural networks. It includes specialized tools for image alignment, region-of-interest extraction, and hardware-level lighting control to minimize glare on glass surfaces. To mainta

Nebullvm is an AI inference accelerator, GPU resource orchestrator, and performance optimization library for large language models. It functions as an optimization layer designed to lower operational costs by aligning model execution with underlying hardware architectures. The system maximizes cluster efficiency through real-time dynamic partitioning and elastic quotas for shared hardware resources. It employs alignment methods and techniques to reduce the hardware and data requirements necessary for tuning large language models. The project covers broad capability areas including AI infrast

This project is a vision language model framework and vision-to-text pipeline designed for deploying and optimizing models that process both images and text. It provides an on-device inference engine and a vision language model framework to run quantized models locally on mobile and desktop hardware accelerators. The framework features a model quantization toolkit to reduce weight precision for lower memory footprints and increased execution speed on specialized silicon. It also includes an efficient vision encoder utilizing a hybrid encoding system to compress image tokens, which reduces pro

这是一个端到端的推荐流水线和框架，专为构建生成式推荐引擎而设计。它利用大语言模型和向量搜索来促进个性化物品发现，并通过分词和语义对齐生成物品推荐。 该系统集成了生成式推荐引擎、多目标排序系统和向量搜索检索流水线。这些组件允许使用推理框架对物品进行推理，并应用重排序算法来优化输出结果的多样性。 该框架涵盖了推荐系统开发的完整生命周期，包括候选检索优化、用户偏好建模，以及用于部署的离线-在线流水线拆分。它还结合了硬件感知序列建模，以在不同硬件规模下保持性能。

Corenet is a deep learning training framework and computer vision model library designed for developing neural networks across vision, text, and audio modalities. It functions as a distributed training orchestrator for scaling workloads across multiple compute nodes and provides a multimodal data pipeline for processing image, text, and video data. The project includes a model conversion toolkit for transforming weights and architectures between different machine learning frameworks. It also provides tools for optimizing model performance on Apple Silicon and reducing response latency in gene

YOLOv6 是一个专为工业目标检测设计的单阶段深度学习框架。它作为一个计算机视觉模型训练器，用于识别和定位图像中的对象，以及一个使用掩码描绘精确对象边界的实例分割工具。 该项目包括一个专门的移动端推理优化器和模型量化工具包。这些组件专注于减小模型大小和分辨率以提高在 ARM 架构芯片组上的执行速度，并将模型转换为低精度格式以减小文件大小。 该框架涵盖了广泛的功能，包括自定义模型训练、实时实例分割以及用于跨平台执行的模型运行时转换。它还支持边缘设备推理优化，以保持在各种硬件运行时上的性能。

coremltools 是一个转换工具包和翻译器，旨在将各种框架的机器学习模型转换为 Core ML 格式，以便在 Apple 硬件上执行。它提供了一套工具，用于将权重和架构从外部库迁移到可部署的模型格式中。 该项目包含一个优化工具和一个用于编辑模型图及修改元数据的编程接口，以提高在目标硬件上的性能。它还具有一个验证套件，用于检查模型规范和操作兼容性，以确保在运行时正确执行。 该工具包涵盖了广泛的部署功能，包括模型图编辑、元数据配置以及针对正式格式规范的兼容性验证。

本项目是一套 MLOps 架构指南和框架，旨在设计并将深度学习系统部署到生产环境。它为模型推理部署、机器学习流水线编排以及生产级机器学习架构的构建提供了结构化的方法。 该项目的特色在于专注于分布式深度学习和边缘 AI 优化。它涵盖了在多个 GPU 上并行化模型训练以处理大规模数据集的方法，并应用了量化和蒸馏等技术来减小嵌入式硬件上的模型体积。 其功能范围还扩展到了监控和可观测性，包括跟踪模型性能、数据漂移和实验指标。此外，它还解决了数据工作流编排、通过对象存储进行数据集版本控制，以及使用自适应批处理和容器化编排来管理高并发推理请求的问题。

Efficient-AI-Backbones 是一个轻量级神经网络库和计算机视觉模型库。它提供了一系列经过优化的深度学习骨干网络，旨在最大限度地减少人工智能任务的计算开销和内存占用。 该项目实现了 GhostNet 和 MLP 等专用架构，以降低处理需求。它具有模块化的骨干网络设计，并分发预训练权重，以加速视觉模型的开发和部署。 该库涵盖了高效神经网络设计和边缘设备 AI 优化。其功能包括实现轻量级架构，以及为计算机视觉应用部署预训练骨干网络。

MODNet 是一个深度学习图像分割和人像抠图模型，旨在将人物主体从背景中分离出来。它仅使用标准 RGB 输入即可为图像和视频生成高质量的 Alpha 遮罩，消除了对手动三元图（trimap）引导的需求。 该框架针对实时推理进行了优化，并提供了将预训练模型权重导出为特定格式以部署在目标硬件上的实用工具。 该项目涵盖了人像分离的完整工作流，包括在标记数据集上进行监督式抠图模型训练、实时视频背景移除以及边缘硬件部署。

ExecuTorch is a lightweight C++ runtime for deploying PyTorch models on mobile, embedded, and edge hardware. It provides an ahead-of-time compilation pipeline that exports, quantizes, and lowers model graphs into compact serialized programs, then executes them through a minimal runtime with hardware acceleration and on-device large language model inference capabilities. The project distinguishes itself through a hardware accelerator delegate system that partitions model subgraphs and offloads computation to specialized backends including NPUs, GPUs, and DSPs from Apple, Arm, Intel, MediaTek,

该项目是一套专为 Blender 设计的优化插件，旨在处理用于实时渲染和虚拟形象平台的网格、骨架和纹理。它作为网格和骨架优化器，简化 3D 模型并提供专门的工具，用于从 MMD、Mixamo 和 DAZ 等格式导入和清理资产。 该工具包具有专门的流水线，用于为 VRChat 和其他社交 VR 环境准备角色。它包括一个纹理图集生成器，可将多张图像合并为一张以减少绘制调用（draw calls），以及基于字典的翻译服务，用于将实体名称从日语转换为英语。 该插件涵盖了广泛的技术能力，包括保留形状键（shape keys）的多边形减面、通过骨骼合并和层级清理进行的骨架简化，以及用于口型同步和眼球追踪的面部动画设置。它还提供几何操作工具，用于调整面法线并将对象变换直接烘焙到网格数据中。 该项目是作为基于 Python 的 Blender 插件实现的。

This project is a containerized local AI infrastructure stack designed to deploy large language models and vector databases on private hardware. It functions as an orchestration platform that combines AI runners, knowledge graphs, and a visual workflow builder for creating agentic chatflows and automating tasks via tool integration. The platform distinguishes itself through a low-code approach to agent orchestration, utilizing a visual interface to design complex sequences and connect agents to external tools and search engines. It includes a dedicated local observability stack to track promp

DeepCamera is an open-source AI video surveillance and network video recorder platform powered by local vision language models and hardware-accelerated processing. It integrates live feeds from network cameras, webcams, and mobile devices to monitor physical spaces while running local edge vision inference without relying on cloud servers. The platform incorporates privacy-preserving video anonymization that converts raw video frames into abstract depth maps in real time, retaining motion tracking while protecting personal identity. Its modular architecture supports pluggable AI scripts and

Neural Compressor is a deep learning model compression toolkit and AI inference acceleration engine. It functions as an automated model quantization tool and hardware-aware model compiler designed to reduce the memory footprint of neural networks and decrease execution latency. The project provides specialized frameworks for optimizing large language models, utilizing weight-only quantization and hardware-specific kernels to improve the operational efficiency of generative AI workloads. It maps neural network operators to specialized CPU and GPU vector instructions to accelerate model executi