Awesome GitHub RepositoriesML Performance Profilers

This repository serves as a comprehensive research platform and toolkit for advancing machine learning, quantum computing, and large-scale scientific data analysis. It provides foundational frameworks for developing complex algorithmic systems, offering the necessary infrastructure for distributed training, computational graph execution, and high-performance model development. The project distinguishes itself by integrating specialized research domains with robust, privacy-preserving methodologies. It supports diverse scientific discovery through tools for quantum simulation, physics-informed

AISystem is a comprehensive AI full-stack infrastructure project covering the entire pipeline from AI chip architecture to high-level training frameworks. It encompasses the development of AI compiler frameworks, inference engines, and distributed training orchestrators designed to coordinate workloads across a heterogeneous compute stack of CPUs, GPUs, and NPUs. The project focuses on the deep integration of software and hardware, employing software-hardware co-design to align tensor layouts with physical memory structures. It provides specialized capabilities for accelerating Transformer mo

DiffusionBee is a Stable Diffusion desktop client for macOS that functions as an AI image generator and editor. It allows for the local generation of images from text prompts and the management of diffusion models without requiring external cloud services or technical setup. The application includes a local diffusion model manager for importing and switching between custom trained model files to achieve specific artistic styles. It also features a system for tracking generation history and uploading assets to a public gallery. The software covers several image synthesis and manipulation work

TensorRT 是一个深度学习推理引擎和软件开发工具包，旨在优化和部署神经网络，以在 NVIDIA GPU 上实现高性能执行。它作为一个 GPU 加速框架，在生产部署期间减少延迟并提高训练模型的吞吐量。 该工具包从 Open Neural Network Exchange 格式导入模型，并将它们转换为优化的引擎。它利用基于图的模型优化、层融合内核生成和基于精度的量化，将浮点权重转换为低精度格式。 该框架提供了硬件特定引擎序列化的功能，并支持通过自定义插件扩展推理能力，以用于专门的神经网络层。

YOLOv10 is a PyTorch computer vision library and real-time vision framework designed for locating and identifying multiple objects in images and video streams. It functions as an end-to-end object detector that optimizes for high-speed deployment and detection precision. The project is distinguished by an NMS-free detection architecture that predicts a single bounding box per object, eliminating the need for non-maximum suppression post-processing to reduce inference latency. It further optimizes for edge hardware through scalable weights and a quantization-friendly structure that facilitates

This project is an AI-powered IDE extension and LLM coding assistant that provides a conversational interface for generating, refactoring, and debugging code. It functions as an AI agent framework and a Model Context Protocol client, connecting AI models to external data sources and tools to automate complex development tasks. The system is distinguished by its use of autonomous AI agents capable of multi-step task execution, including the ability to read files, modify code, and run terminal commands iteratively. It supports recursive agent orchestration through subagent delegation and employ

AI-on-the-edge-device is an edge AI meter digitizer and computer vision image processor designed to convert images of analog and digital utility meters into numeric values. It functions as an IoT gateway that runs neural network inference locally on hardware to monitor water, power, and gas readings. The system is distinguished by its ability to handle both analog pointers and digital digits through custom-trained neural networks. It includes specialized tools for image alignment, region-of-interest extraction, and hardware-level lighting control to minimize glare on glass surfaces. To mainta

Nebullvm is an AI inference accelerator, GPU resource orchestrator, and performance optimization library for large language models. It functions as an optimization layer designed to lower operational costs by aligning model execution with underlying hardware architectures. The system maximizes cluster efficiency through real-time dynamic partitioning and elastic quotas for shared hardware resources. It employs alignment methods and techniques to reduce the hardware and data requirements necessary for tuning large language models. The project covers broad capability areas including AI infrast

This project is a vision language model framework and vision-to-text pipeline designed for deploying and optimizing models that process both images and text. It provides an on-device inference engine and a vision language model framework to run quantized models locally on mobile and desktop hardware accelerators. The framework features a model quantization toolkit to reduce weight precision for lower memory footprints and increased execution speed on specialized silicon. It also includes an efficient vision encoder utilizing a hybrid encoding system to compress image tokens, which reduces pro

TensorBoard is a visualization toolkit for tracking and analyzing machine learning model training progress and performance using TensorFlow event logs. It provides a monitoring dashboard for plotting scalar metrics, tensor distributions, and training curves, and includes specialized tools for visualizing neural network computational graphs and projecting high-dimensional embeddings. The project enables side-by-side comparison of multiple training runs to analyze the impact of hyperparameters on model outcomes. It also features a high-dimensional embedding projector and a graph visualizer for

这是一个端到端的推荐流水线和框架，专为构建生成式推荐引擎而设计。它利用大语言模型和向量搜索来促进个性化物品发现，并通过分词和语义对齐生成物品推荐。 该系统集成了生成式推荐引擎、多目标排序系统和向量搜索检索流水线。这些组件允许使用推理框架对物品进行推理，并应用重排序算法来优化输出结果的多样性。 该框架涵盖了推荐系统开发的完整生命周期，包括候选检索优化、用户偏好建模，以及用于部署的离线-在线流水线拆分。它还结合了硬件感知序列建模，以在不同硬件规模下保持性能。

Corenet is a deep learning training framework and computer vision model library designed for developing neural networks across vision, text, and audio modalities. It functions as a distributed training orchestrator for scaling workloads across multiple compute nodes and provides a multimodal data pipeline for processing image, text, and video data. The project includes a model conversion toolkit for transforming weights and architectures between different machine learning frameworks. It also provides tools for optimizing model performance on Apple Silicon and reducing response latency in gene

YOLOv6 是一个专为工业目标检测设计的单阶段深度学习框架。它作为一个计算机视觉模型训练器，用于识别和定位图像中的对象，以及一个使用掩码描绘精确对象边界的实例分割工具。 该项目包括一个专门的移动端推理优化器和模型量化工具包。这些组件专注于减小模型大小和分辨率以提高在 ARM 架构芯片组上的执行速度，并将模型转换为低精度格式以减小文件大小。 该框架涵盖了广泛的功能，包括自定义模型训练、实时实例分割以及用于跨平台执行的模型运行时转换。它还支持边缘设备推理优化，以保持在各种硬件运行时上的性能。

coremltools 是一个转换工具包和翻译器，旨在将各种框架的机器学习模型转换为 Core ML 格式，以便在 Apple 硬件上执行。它提供了一套工具，用于将权重和架构从外部库迁移到可部署的模型格式中。 该项目包含一个优化工具和一个用于编辑模型图及修改元数据的编程接口，以提高在目标硬件上的性能。它还具有一个验证套件，用于检查模型规范和操作兼容性，以确保在运行时正确执行。 该工具包涵盖了广泛的部署功能，包括模型图编辑、元数据配置以及针对正式格式规范的兼容性验证。

This project provides a comprehensive technical guide and framework for engineering large-scale machine learning systems. It covers the full lifecycle of model development, focusing on the infrastructure and computational principles required to build, train, and serve generative AI models across distributed GPU clusters. The repository distinguishes itself by offering deep-dive tutorials and implementation strategies for complex system challenges. It emphasizes high-performance architectural primitives, such as collective communication orchestration, distributed tensor sharding, and static gr

cuml is a GPU-accelerated machine learning library and framework that uses CUDA to accelerate tabular data preprocessing and model execution. It provides a suite of tools for training and deploying classification, regression, and clustering models on NVIDIA GPUs and GPU clusters. The library is designed for scalability, offering a distributed GPU machine learning environment that can spread computation and data across multiple hardware accelerators and nodes to handle datasets exceeding single-device memory. It mirrors standard estimator interfaces to allow the replacement of CPU-based models

本项目是一套 MLOps 架构指南和框架，旨在设计并将深度学习系统部署到生产环境。它为模型推理部署、机器学习流水线编排以及生产级机器学习架构的构建提供了结构化的方法。 该项目的特色在于专注于分布式深度学习和边缘 AI 优化。它涵盖了在多个 GPU 上并行化模型训练以处理大规模数据集的方法，并应用了量化和蒸馏等技术来减小嵌入式硬件上的模型体积。 其功能范围还扩展到了监控和可观测性，包括跟踪模型性能、数据漂移和实验指标。此外，它还解决了数据工作流编排、通过对象存储进行数据集版本控制，以及使用自适应批处理和容器化编排来管理高并发推理请求的问题。

Efficient-AI-Backbones 是一个轻量级神经网络库和计算机视觉模型库。它提供了一系列经过优化的深度学习骨干网络，旨在最大限度地减少人工智能任务的计算开销和内存占用。 该项目实现了 GhostNet 和 MLP 等专用架构，以降低处理需求。它具有模块化的骨干网络设计，并分发预训练权重，以加速视觉模型的开发和部署。 该库涵盖了高效神经网络设计和边缘设备 AI 优化。其功能包括实现轻量级架构，以及为计算机视觉应用部署预训练骨干网络。

MODNet 是一个深度学习图像分割和人像抠图模型，旨在将人物主体从背景中分离出来。它仅使用标准 RGB 输入即可为图像和视频生成高质量的 Alpha 遮罩，消除了对手动三元图（trimap）引导的需求。 该框架针对实时推理进行了优化，并提供了将预训练模型权重导出为特定格式以部署在目标硬件上的实用工具。 该项目涵盖了人像分离的完整工作流，包括在标记数据集上进行监督式抠图模型训练、实时视频背景移除以及边缘硬件部署。