1 Repo
Awesome GitHub RepositoriesCaching Allocator Tuners
Tools for tuning caching allocator parameters like bin growth and cache limits to balance memory usage against allocation speed.
Distinct from Memory Allocators: Distinct from Memory Allocators: focuses on tuning caching allocator tradeoffs, not general memory allocation strategies.
Explore 1 awesome GitHub repository matching part of an awesome list · Caching Allocator Tuners. Refine with filters or upvote what's useful.
Awesome Caching Allocator Tuners GitHub Repositories
Finde die besten Repos mit KI.Wir suchen mit KI nach den am besten passenden Repositories.
- opennmt/ctranslate2
OpenNMT/CTranslate2
4,319Auf GitHub ansehenCTranslate2 is a C++ inference engine and runtime for Transformer models, designed to execute models on both CPU and GPU with optimizations for speed and memory efficiency. It functions as a model format converter, quantization tool, and REST API server, enabling deployment of neural machine translation, automatic speech recognition, and text generation models. The engine distinguishes itself through a suite of runtime optimizations including layer fusion, weight-matrix quantization, batch-by-length grouping, and a caching allocator that reuses GPU memory. It supports tensor-parallel model di
Tunes the caching allocator's bin growth, bin size, and cache limits to balance memory usage against allocation speed.
C++avxavx2cpp
