1 个仓库
Awesome GitHub RepositoriesCaching Allocator Tuners
Tools for tuning caching allocator parameters like bin growth and cache limits to balance memory usage against allocation speed.
Distinct from Memory Allocators: Distinct from Memory Allocators: focuses on tuning caching allocator tradeoffs, not general memory allocation strategies.
Explore 1 awesome GitHub repository matching part of an awesome list · Caching Allocator Tuners. Refine with filters or upvote what's useful.
Awesome Caching Allocator Tuners GitHub Repositories
用 AI 发现最棒的仓库。我们将通过 AI 为您搜索最匹配的仓库。
- opennmt/ctranslate2
OpenNMT/CTranslate2
4,319在 GitHub 上查看CTranslate2 is a C++ inference engine and runtime for Transformer models, designed to execute models on both CPU and GPU with optimizations for speed and memory efficiency. It functions as a model format converter, quantization tool, and REST API server, enabling deployment of neural machine translation, automatic speech recognition, and text generation models. The engine distinguishes itself through a suite of runtime optimizations including layer fusion, weight-matrix quantization, batch-by-length grouping, and a caching allocator that reuses GPU memory. It supports tensor-parallel model di
Tunes the caching allocator's bin growth, bin size, and cache limits to balance memory usage against allocation speed.
C++avxavx2cpp
