9 dépôts
Monitoring and logging of GPU memory usage during model execution to optimize resource allocation.
Distinct from GPU Memory Optimizations: Focuses on monitoring usage patterns over time rather than low-level hardware layout optimization
Explore 9 awesome GitHub repositories matching operating systems & systems programming · Memory Consumption Tracking. Refine with filters or upvote what's useful.
This project is a comprehensive collection of educational examples and reference implementations for building vision and language models using PyTorch. It serves as a deep learning tutorial covering the end-to-end process of developing neural networks, from initial architecture definition to final production deployment. The repository provides detailed guides on implementing a wide range of domain-specific models, including convolutional neural networks for object detection and segmentation, as well as transformer and recurrent architectures for natural language processing. It emphasizes gene
Tracks GPU memory usage relative to input token counts to optimize hardware resource allocation.
This project is a suite of analytical tools for quantifying web performance, specifically designed for benchmarking the rendering speed and memory usage of various JavaScript frameworks. It provides a standardized set of DOM manipulation tests and a comparison tool that uses weighted geometric means to measure efficiency across different web implementations. The benchmark harness distinguishes itself by providing deep analysis of DOM reconciliation strategies, comparing the performance and correctness of keyed versus non-keyed rendering. It also includes a memory profiler for tracking allocat
Monitors memory consumption and overhead specifically for the runtime engine during DOM update cycles.
JerryScript is a lightweight, ECMAScript-compliant JavaScript engine and bytecode compiler designed for resource-constrained devices. It serves as an embedded interpreter and IoT scripting runtime, enabling the execution of JavaScript code within native C applications on hardware with limited memory. The project differentiates itself through a focus on low-memory runtime management, utilizing bytecode precompilation and pre-compiled state snapshots to reduce startup time and memory overhead. It features a C-binding native bridge for bidirectional communication between native code and scripts,
Measures engine overhead by recording memory usage during runtime or termination.
NCCL est une bibliothèque de communication haute performance et un framework de calcul GPU distribué conçu pour exécuter des échanges de données collectifs et point à point sur plusieurs GPU dans des systèmes à un ou plusieurs nœuds. Il sert de couche de transport GPU RDMA et d'orchestrateur de mémoire, facilitant la synchronisation à large bande passante des données et des gradients de modèle pour l'entraînement et l'inférence GPU distribués. La bibliothèque se distingue par sa capacité à exécuter des primitives de communication directement depuis les noyaux (kernels) GPU, supprimant le CPU hôte du chemin critique. Elle utilise une sélection de chemin consciente de la topologie pour optimiser le mouvement des données et emploie un transport réseau basé sur RDMA, incluant InfiniBand et NVLink, pour permettre un accès mémoire zéro-copie entre les appareils sur différents nœuds physiques. Le projet couvre un large éventail de modèles de communication collective, notamment les réductions, les diffusions (broadcasts), les rassemblements (gathers) et les échanges tous-à-tous, ainsi que l'accès mémoire distant point à point. Il fournit une gestion complète des communicateurs pour initialiser, partitionner et redimensionner les groupes GPU, ainsi qu'une gestion spécialisée de la mémoire pour enregistrer les tampons (buffers) et coordonner la mémoire partagée des appareils. Le système inclut une suite d'outils de surveillance et d'observabilité pour le suivi de la santé, la journalisation diagnostique et la surveillance des événements en temps réel, ainsi que des interfaces d'intégration pour les frameworks de machine learning, les graphes CUDA, MPI et Python.
Monitors and logs GPU memory usage, distinguishing between persistent and suspendable allocations.
This project is a comprehensive instructional resource and course for building neural networks using PyTorch. It covers the fundamental building blocks of deep learning, including tensor manipulation, automatic differentiation, and the construction of modular neural network components. The repository serves as a technical guide for several specialized domains. It provides implementation details for computer vision tasks such as image classification, object detection, and semantic segmentation, as well as natural language processing workflows involving transformers, recurrent networks, and gen
Monitors GPU memory usage relative to input length to determine optimal context truncation limits.
ExecuTorch is a lightweight C++ runtime for deploying PyTorch models on mobile, embedded, and edge hardware. It provides an ahead-of-time compilation pipeline that exports, quantizes, and lowers model graphs into compact serialized programs, then executes them through a minimal runtime with hardware acceleration and on-device large language model inference capabilities. The project distinguishes itself through a hardware accelerator delegate system that partitions model subgraphs and offloads computation to specialized backends including NPUs, GPUs, and DSPs from Apple, Arm, Intel, MediaTek,
ExecuTorch monitors peak and per-operator memory consumption to optimize resource usage on constrained hardware.
imapsync is an IMAP mailbox synchronization tool and data migration utility designed to copy and synchronize email messages and folder structures between two IMAP servers. It functions as a migration manager for transferring bulk email accounts between different hosting providers, preserving folder hierarchies and message metadata. The tool is distinguished by its ability to automate the transfer of multiple mailboxes sequentially from delimited lists using administrative credentials or user-specific authentication. It supports advanced authentication methods including OAuth2 and XOAUTH2, and
Saves memory during large folder synchronizations by using unique identifiers instead of full message headers.
MIRIX is an AI agent state orchestrator and long-term memory system designed to provide persistent context for large language models. It functions as a multi-modal AI memory pipeline that processes text, voice, and screen captures into structured knowledge stores, including a dedicated screen activity knowledge base. The project distinguishes itself by integrating a multi-modal observation pipeline that monitors desktop activity in real-time to build a searchable history of user actions. It utilizes a multi-tiered memory hierarchy—separating episodic, semantic, procedural, and core stores—and
Provides control over whether incoming information is processed immediately or batched for background memory updates.
RLinf is a distributed reinforcement learning orchestrator and embodied AI training framework. It provides the infrastructure to train vision-language-action models and robotic policies using a combination of reinforcement learning and supervised fine-tuning. The system is designed for scaling workloads across GPU clusters, managing the placement of actors, rollout workers, and environment components. It features a specialized robotics data collection pipeline for gathering teleoperated demonstrations and simulation trajectories into standardized replay buffers, alongside a hardware interface
Reduces memory overhead and prevents catastrophic forgetting by freezing backbone parameters and updating only the output head.