Awesome GitHub RepositoriesGPU Memory Diagnostics

jetson-inference is a set of libraries and tools for executing optimized deep learning models on embedded GPU hardware. Its primary purpose is to enable real-time computer vision and AI inference at the edge with low latency and high throughput. The project distinguishes itself through high-performance streaming analytics and the ability to execute concurrent AI pipelines on auto-grade silicon. It provides specialized support for multi-sensor stream processing, utilizing zero-copy data transport to load camera frames directly into GPU memory. The codebase covers a broad surface of capabiliti

Serf is a decentralized cluster coordination tool that manages node membership, failure detection, and event broadcasting across a distributed system without a central coordinator. Every node runs an identical agent process that independently handles membership, health monitoring, and event propagation through a peer-to-peer gossip protocol, creating a leaderless architecture where no single point of failure exists. The project implements the SWIM failure detection algorithm, where each node monitors a small random subset of peers to detect unreachable or failed nodes in real time. Custom use

Aibrix es un orquestador de inferencia diseñado para escalar, enrutar y gestionar el despliegue de modelos de lenguaje grandes a través de clústeres vLLM distribuidos. Sirve como una puerta de enlace centralizada para el balanceo de carga y el enrutamiento de tráfico a réplicas y versiones específicas de modelos. El sistema gestiona la eficiencia de los recursos a través de un autoescalador de clúster de GPU que ajusta el conteo de instancias de cómputo según el volumen de solicitudes en tiempo real. Además, optimiza las operaciones mezclando diferentes tipos de aceleradores dentro de un solo clúster y utilizando un orquestador de adaptadores de modelos para desplegar adaptadores de parámetros ligeros en modelos base compartidos. Las capacidades generales incluyen el uso de un gestor de caché de clave-valor distribuido para compartir datos de tokens a través de motores de inferencia y la implementación de monitoreo de salud del hardware para detectar fallos en las unidades de procesamiento. El proyecto también proporciona un pipeline de métricas unificado para estandarizar la recopilación de datos de rendimiento en diversos entornos de ejecución.

Memberlist is a Go library used for maintaining distributed cluster membership and failure detection via a peer-to-peer gossip protocol. It functions as a cluster state synchronizer, allowing nodes to track active members and propagate metadata without a central coordinator. The library implements a secure gossip mechanism using AES-GCM encryption to protect inter-node communication and membership data. It distinguishes its failure detection through a suspicion-based model and adaptive timeout scaling, which reduces false positives caused by transient network latency. The project provides ca

Node Problem Detector is a Kubernetes-native agent that monitors node health and reports hardware failures, kernel issues, and other node-level problems to the cluster control plane. It detects problems by scanning kernel ring buffer messages for error patterns, running user-defined health check scripts, and collecting system metrics from CPU, memory, disk, and network interfaces. The agent distinguishes between permanent and temporary problems by mapping plugin failures to either persistent node conditions visible in kubectl describe node or one-time node events. It supports running multip