Awesome GitHub RepositoriesGPU Memory Diagnostics

jetson-inference is a set of libraries and tools for executing optimized deep learning models on embedded GPU hardware. Its primary purpose is to enable real-time computer vision and AI inference at the edge with low latency and high throughput. The project distinguishes itself through high-performance streaming analytics and the ability to execute concurrent AI pipelines on auto-grade silicon. It provides specialized support for multi-sensor stream processing, utilizing zero-copy data transport to load camera frames directly into GPU memory. The codebase covers a broad surface of capabiliti

Serf is a decentralized cluster coordination tool that manages node membership, failure detection, and event broadcasting across a distributed system without a central coordinator. Every node runs an identical agent process that independently handles membership, health monitoring, and event propagation through a peer-to-peer gossip protocol, creating a leaderless architecture where no single point of failure exists. The project implements the SWIM failure detection algorithm, where each node monitors a small random subset of peers to detect unreachable or failed nodes in real time. Custom use

Aibrix 是一个推理编排器，专为跨分布式 vLLM 集群扩展、路由和管理大语言模型部署而设计。它作为一个集中式网关，用于负载均衡并将流量路由到特定的模型副本和版本。 该系统通过 GPU 集群自动缩放器管理资源效率，该缩放器根据实时请求量调整计算实例数量。它通过在单个集群内混合不同加速器类型，并利用模型适配器编排器在共享基础模型上部署轻量级参数适配器，进一步优化了操作。 广泛的功能包括使用分布式键值缓存管理器在推理引擎之间共享 Token 数据，以及实施硬件健康监控以检测处理单元故障。该项目还提供了一个统一的指标流水线，以标准化跨不同运行时环境的性能数据收集。

Memberlist 是一个 Go 库，用于通过点对点 Gossip 协议维护分布式集群成员资格和故障检测。它作为一个集群状态同步器，允许节点跟踪活跃成员并传播元数据，而无需中央协调器。 该库实现了一种使用 AES-GCM 加密的安全 Gossip 机制，以保护节点间通信和成员数据。它通过基于怀疑的模型和自适应超时缩放来区分其故障检测，这减少了由瞬态网络延迟引起的误报。 该项目提供了广播自定义用户定义数据、通过推拉机制同步节点状态以及通过解耦接口管理网络传输的功能。它还包括用于操作监控的工具，例如往返时间测量和性能指标发射。

Node Problem Detector is a Kubernetes-native agent that monitors node health and reports hardware failures, kernel issues, and other node-level problems to the cluster control plane. It detects problems by scanning kernel ring buffer messages for error patterns, running user-defined health check scripts, and collecting system metrics from CPU, memory, disk, and network interfaces. The agent distinguishes between permanent and temporary problems by mapping plugin failures to either persistent node conditions visible in kubectl describe node or one-time node events. It supports running multip