Awesome GitHub RepositoriesEdge Hardware Optimizations

AISystem is a comprehensive AI full-stack infrastructure project covering the entire pipeline from AI chip architecture to high-level training frameworks. It encompasses the development of AI compiler frameworks, inference engines, and distributed training orchestrators designed to coordinate workloads across a heterogeneous compute stack of CPUs, GPUs, and NPUs. The project focuses on the deep integration of software and hardware, employing software-hardware co-design to align tensor layouts with physical memory structures. It provides specialized capabilities for accelerating Transformer mo

YOLOv10 is a PyTorch computer vision library and real-time vision framework designed for locating and identifying multiple objects in images and video streams. It functions as an end-to-end object detector that optimizes for high-speed deployment and detection precision. The project is distinguished by an NMS-free detection architecture that predicts a single bounding box per object, eliminating the need for non-maximum suppression post-processing to reduce inference latency. It further optimizes for edge hardware through scalable weights and a quantization-friendly structure that facilitates

AI-on-the-edge-device is an edge AI meter digitizer and computer vision image processor designed to convert images of analog and digital utility meters into numeric values. It functions as an IoT gateway that runs neural network inference locally on hardware to monitor water, power, and gas readings. The system is distinguished by its ability to handle both analog pointers and digital digits through custom-trained neural networks. It includes specialized tools for image alignment, region-of-interest extraction, and hardware-level lighting control to minimize glare on glass surfaces. To mainta

YOLOv6 是一个专为工业目标检测设计的单阶段深度学习框架。它作为一个计算机视觉模型训练器，用于识别和定位图像中的对象，以及一个使用掩码描绘精确对象边界的实例分割工具。 该项目包括一个专门的移动端推理优化器和模型量化工具包。这些组件专注于减小模型大小和分辨率以提高在 ARM 架构芯片组上的执行速度，并将模型转换为低精度格式以减小文件大小。 该框架涵盖了广泛的功能，包括自定义模型训练、实时实例分割以及用于跨平台执行的模型运行时转换。它还支持边缘设备推理优化，以保持在各种硬件运行时上的性能。

本项目是一套 MLOps 架构指南和框架，旨在设计并将深度学习系统部署到生产环境。它为模型推理部署、机器学习流水线编排以及生产级机器学习架构的构建提供了结构化的方法。 该项目的特色在于专注于分布式深度学习和边缘 AI 优化。它涵盖了在多个 GPU 上并行化模型训练以处理大规模数据集的方法，并应用了量化和蒸馏等技术来减小嵌入式硬件上的模型体积。 其功能范围还扩展到了监控和可观测性，包括跟踪模型性能、数据漂移和实验指标。此外，它还解决了数据工作流编排、通过对象存储进行数据集版本控制，以及使用自适应批处理和容器化编排来管理高并发推理请求的问题。

Efficient-AI-Backbones 是一个轻量级神经网络库和计算机视觉模型库。它提供了一系列经过优化的深度学习骨干网络，旨在最大限度地减少人工智能任务的计算开销和内存占用。 该项目实现了 GhostNet 和 MLP 等专用架构，以降低处理需求。它具有模块化的骨干网络设计，并分发预训练权重，以加速视觉模型的开发和部署。 该库涵盖了高效神经网络设计和边缘设备 AI 优化。其功能包括实现轻量级架构，以及为计算机视觉应用部署预训练骨干网络。

MODNet 是一个深度学习图像分割和人像抠图模型，旨在将人物主体从背景中分离出来。它仅使用标准 RGB 输入即可为图像和视频生成高质量的 Alpha 遮罩，消除了对手动三元图（trimap）引导的需求。 该框架针对实时推理进行了优化，并提供了将预训练模型权重导出为特定格式以部署在目标硬件上的实用工具。 该项目涵盖了人像分离的完整工作流，包括在标记数据集上进行监督式抠图模型训练、实时视频背景移除以及边缘硬件部署。

ExecuTorch is a lightweight C++ runtime for deploying PyTorch models on mobile, embedded, and edge hardware. It provides an ahead-of-time compilation pipeline that exports, quantizes, and lowers model graphs into compact serialized programs, then executes them through a minimal runtime with hardware acceleration and on-device large language model inference capabilities. The project distinguishes itself through a hardware accelerator delegate system that partitions model subgraphs and offloads computation to specialized backends including NPUs, GPUs, and DSPs from Apple, Arm, Intel, MediaTek,

LiteRT is a runtime and API for executing machine learning and generative AI models on mobile, desktop, and IoT hardware. It consists of an inference engine and a specialized environment for running quantized large language and diffusion models locally on edge hardware. The system includes an ahead-of-time model compiler that translates models into hardware-specific bytecode to reduce startup latency and memory overhead. It provides a unified interface for Neural Processing Units with automatic fallback routing to CPUs or GPUs when specific subgraph support is unavailable. An edge model conve