9 个仓库
Monitoring and logging of GPU memory usage during model execution to optimize resource allocation.
Distinct from GPU Memory Optimizations: Focuses on monitoring usage patterns over time rather than low-level hardware layout optimization
Explore 9 awesome GitHub repositories matching operating systems & systems programming · Memory Consumption Tracking. Refine with filters or upvote what's useful.
This project is a comprehensive collection of educational examples and reference implementations for building vision and language models using PyTorch. It serves as a deep learning tutorial covering the end-to-end process of developing neural networks, from initial architecture definition to final production deployment. The repository provides detailed guides on implementing a wide range of domain-specific models, including convolutional neural networks for object detection and segmentation, as well as transformer and recurrent architectures for natural language processing. It emphasizes gene
Tracks GPU memory usage relative to input token counts to optimize hardware resource allocation.
This project is a suite of analytical tools for quantifying web performance, specifically designed for benchmarking the rendering speed and memory usage of various JavaScript frameworks. It provides a standardized set of DOM manipulation tests and a comparison tool that uses weighted geometric means to measure efficiency across different web implementations. The benchmark harness distinguishes itself by providing deep analysis of DOM reconciliation strategies, comparing the performance and correctness of keyed versus non-keyed rendering. It also includes a memory profiler for tracking allocat
Monitors memory consumption and overhead specifically for the runtime engine during DOM update cycles.
JerryScript is a lightweight, ECMAScript-compliant JavaScript engine and bytecode compiler designed for resource-constrained devices. It serves as an embedded interpreter and IoT scripting runtime, enabling the execution of JavaScript code within native C applications on hardware with limited memory. The project differentiates itself through a focus on low-memory runtime management, utilizing bytecode precompilation and pre-compiled state snapshots to reduce startup time and memory overhead. It features a C-binding native bridge for bidirectional communication between native code and scripts,
Measures engine overhead by recording memory usage during runtime or termination.
NCCL 是一个高性能通信库和分布式 GPU 计算框架,专为在单节点或多节点系统中的多个 GPU 之间执行集合和点对点数据交换而设计。它充当 RDMA GPU 传输层和内存编排器,为分布式 GPU 训练和推理提供高带宽的数据和模型梯度同步。 该库的特色在于能够直接从 GPU 内核执行通信原语,将主机 CPU 从关键路径中移除。它利用拓扑感知路径选择来优化数据移动,并采用包括 InfiniBand 和 NVLink 在内的基于 RDMA 的网络传输,以实现设备跨不同物理节点之间的零拷贝内存访问。 该项目涵盖了广泛的集合通信模式,包括归约(Reductions)、广播(Broadcasts)、收集(Gathers)和全对全交换(All-to-all exchanges),以及点对点远程内存访问。它提供全面的通信器管理,用于初始化、分区和调整 GPU 组大小,以及用于注册缓冲区和协调共享设备内存的专用内存管理。 该系统包括一套用于健康跟踪、诊断日志记录和实时事件监控的监控与可观测性工具,以及用于机器学习框架、CUDA Graphs、MPI 和 Python 的集成接口。
Monitors and logs GPU memory usage, distinguishing between persistent and suspendable allocations.
这是一个关于使用 PyTorch 构建神经网络的综合教学资源和课程。它涵盖了深度学习的基本构建块,包括张量操作、自动微分以及模块化神经网络组件的构建。 该仓库是多个专业领域的参考指南。它提供了计算机视觉任务(如图像分类、目标检测和语义分割)的实现细节,以及涉及 Transformer、循环网络和生成模型的自然语言处理工作流。此外,它还包括生成式 AI 的参考资料,专门关注通过扩散模型和对抗网络进行图像合成。 材料延伸至模型优化和部署流水线。它涵盖了通过量化和将模型导出为 ONNX 和 TensorRT 等格式来减小模型大小并提高推理速度的技术。其他能力领域包括用于并行加载的数据工程、使用自定义指标的模型评估,以及开源大语言模型的部署。 该项目主要以一系列 Jupyter Notebook 的形式提供。
Monitors GPU memory usage relative to input length to determine optimal context truncation limits.
ExecuTorch is a lightweight C++ runtime for deploying PyTorch models on mobile, embedded, and edge hardware. It provides an ahead-of-time compilation pipeline that exports, quantizes, and lowers model graphs into compact serialized programs, then executes them through a minimal runtime with hardware acceleration and on-device large language model inference capabilities. The project distinguishes itself through a hardware accelerator delegate system that partitions model subgraphs and offloads computation to specialized backends including NPUs, GPUs, and DSPs from Apple, Arm, Intel, MediaTek,
ExecuTorch monitors peak and per-operator memory consumption to optimize resource usage on constrained hardware.
imapsync is an IMAP mailbox synchronization tool and data migration utility designed to copy and synchronize email messages and folder structures between two IMAP servers. It functions as a migration manager for transferring bulk email accounts between different hosting providers, preserving folder hierarchies and message metadata. The tool is distinguished by its ability to automate the transfer of multiple mailboxes sequentially from delimited lists using administrative credentials or user-specific authentication. It supports advanced authentication methods including OAuth2 and XOAUTH2, and
Saves memory during large folder synchronizations by using unique identifiers instead of full message headers.
MIRIX is an AI agent state orchestrator and long-term memory system designed to provide persistent context for large language models. It functions as a multi-modal AI memory pipeline that processes text, voice, and screen captures into structured knowledge stores, including a dedicated screen activity knowledge base. The project distinguishes itself by integrating a multi-modal observation pipeline that monitors desktop activity in real-time to build a searchable history of user actions. It utilizes a multi-tiered memory hierarchy—separating episodic, semantic, procedural, and core stores—and
Provides control over whether incoming information is processed immediately or batched for background memory updates.
RLinf is a distributed reinforcement learning orchestrator and embodied AI training framework. It provides the infrastructure to train vision-language-action models and robotic policies using a combination of reinforcement learning and supervised fine-tuning. The system is designed for scaling workloads across GPU clusters, managing the placement of actors, rollout workers, and environment components. It features a specialized robotics data collection pipeline for gathering teleoperated demonstrations and simulation trajectories into standardized replay buffers, alongside a hardware interface
Reduces memory overhead and prevents catastrophic forgetting by freezing backbone parameters and updating only the output head.