Awesome GitHub RepositoriesMulti-GPU Workload Distribution

This project is an object detection framework implementing the YOLOv3 architecture using Keras and TensorFlow. It functions as a deep learning vision model and computer vision toolset designed to locate and classify multiple entities within images and video streams using bounding boxes. The system includes a multi-GPU inference engine to distribute computational loads across several graphics processing units. It also provides a pipeline for creating custom object detectors by retraining pre-trained weights on annotated datasets to recognize user-defined object classes. The framework covers m

Tensorpack 是一个高级 TensorFlow 神经网络框架和研究库，专为构建和训练深度学习模型而设计。它提供了一系列可复现的神经网络架构，用于计算机视觉、生成任务、强化学习和自然语言处理。 该项目通过一个专门的深度学习数据流水线脱颖而出，该流水线使用纯 Python 进行并行数据加载和流式传输。它包括一个用于通过数据并行策略分发工作负载的多 GPU 训练编排器，以及一个用于可视化模型显著性和激活图的专用可解释性工具包。 该框架涵盖了广泛的功能，包括用于目标检测和语义分割的计算机视觉流水线、用于语音和文本的序列建模，以及强化学习代理开发。它还提供用于权重量化和低位宽训练的模型优化工具，以及用于复现学术研究论文和转换遗留 Caffe 模型权重的实用程序。

NVIDIA DALI is a GPU-accelerated data loading and preprocessing library designed for deep learning workflows. It constructs high-performance data pipelines that offload decoding, augmentation, and normalization to the GPU, eliminating CPU bottlenecks in training and inference. The library reads data from multiple storage formats and streams it directly into GPU memory, with support for multi-GPU execution to scale throughput across large-scale workloads. DALI distinguishes itself by enabling data pipelines to be built once and executed across multiple deep learning frameworks without code cha

StableSwarmUI 是一个用于 Stable Diffusion 图像生成的 Web 界面和后端编排器。它作为一个分布式 GPU 图像生成器和模块化 AI 图像流水线，提供了一个集中式控制器来管理图像生成请求。 该系统通过将生成任务拆分到多个图形处理器以提高批处理吞吐量的能力而脱颖而出。它利用后端无关的接口连接到本地服务器、远程服务器和云 API，并包含一个用于定义复杂图像处理操作的基于图的可视化工作流设计器。 该平台包括用于添加自定义功能的动态插件扩展系统，以及用于配置系统级依赖的自动化实用程序。它将模块化生成工具和快速编辑界面与跨分布式硬件路由工作负载的能力相结合。

Aibrix 是一个推理编排器，专为跨分布式 vLLM 集群扩展、路由和管理大语言模型部署而设计。它作为一个集中式网关，用于负载均衡并将流量路由到特定的模型副本和版本。 该系统通过 GPU 集群自动缩放器管理资源效率，该缩放器根据实时请求量调整计算实例数量。它通过在单个集群内混合不同加速器类型，并利用模型适配器编排器在共享基础模型上部署轻量级参数适配器，进一步优化了操作。 广泛的功能包括使用分布式键值缓存管理器在推理引擎之间共享 Token 数据，以及实施硬件健康监控以检测处理单元故障。该项目还提供了一个统一的指标流水线，以标准化跨不同运行时环境的性能数据收集。

SwarmUI 是一个用于 Stable Diffusion 的 Web 界面和编排器，旨在生成图像和视频。它作为一个模块化工作流管理器和 API 网关，允许配置和执行生成式 AI 流水线。 该系统的特点是能够将生成工作负载分配到多个显卡上，以提高处理速度和总吞吐量。它采用解耦的客户端-服务端架构和后端无关的接口，使 UI 能够与模型执行环境保持分离。 该平台通过基于插件的架构支持可扩展性，用于添加新的 UI 组件和逻辑处理程序。它通过 HTTP 和 WebSocket 端点提供程序化控制，供外部应用触发生成并实时同步状态。 包含管理员工具，用于授予和管理网络上远程用户对生成环境的访问权限。

Discoart is a diffusion model orchestration framework and distributed GPU generation engine designed to automate and scale image generation workflows across hardware clusters. It functions as a generative AI model API, providing HTTP and gRPC endpoints to trigger and retrieve images from diffusion models as a network service. The system distinguishes itself through a comprehensive task management layer that includes timeline-based prompt and parameter scheduling. It manages the generative art lifecycle by supporting state-based session serialization for recovery, YAML-based configuration mana

QualityScaler is an AI video upscaler and local media processing tool designed to increase the resolution and visual quality of videos and images. It uses deep learning models to enhance detail and remove noise, operating as an offline application that executes all computations on local hardware. The project functions as a GPU-accelerated media processor that distributes workloads across multiple graphics cards to increase rendering speed. To prevent memory overflow during high-resolution tasks, it employs a tiled image processing method that splits large assets into smaller sections. The sy

ComfyUI-SeedVR2_VideoUpscaler is an AI video upscaling tool that uses diffusion models to increase the resolution of videos and images while maintaining visual consistency across frames. The project implements distributed video rendering by splitting datasets into chunks for parallel processing across multiple GPUs. It utilizes model compilation and specialized attention backends to reduce inference latency and increase throughput. Additional capabilities include video color correction using wavelet and LAB matching methods to preserve color fidelity. Hardware memory is managed through block

RLinf is a distributed reinforcement learning orchestrator and embodied AI training framework. It provides the infrastructure to train vision-language-action models and robotic policies using a combination of reinforcement learning and supervised fine-tuning. The system is designed for scaling workloads across GPU clusters, managing the placement of actors, rollout workers, and environment components. It features a specialized robotics data collection pipeline for gathering teleoperated demonstrations and simulation trajectories into standardized replay buffers, alongside a hardware interface