1 repository
Awesome GitHub RepositoriesCache State Controllers
Tools and interfaces for monitoring and modifying the internal state of a caching system.
Distinct from Application State Caching: None of the candidates focus on the administrative control of a cache's runtime state via CLI or Dashboards.
Explore 1 awesome GitHub repository matching data & databases · Cache State Controllers. Refine with filters or upvote what's useful.
Awesome Cache State Controllers GitHub Repositories
Găsește cele mai bune repo-uri cu AI.Vom căuta cele mai potrivite repository-uri folosind AI.
- lmcache/lmcache
LMCache/LMCache
6,909Vezi pe GitHubLMCache is a distributed key-value cache manager and tiering system designed to accelerate large language model inference. It functions as a tiered storage layer that offloads tensors from GPU memory to CPU RAM, local disks, or remote object stores, enabling the reuse of cached prefixes across different inference sessions and serving engines. The system differentiates itself through a disaggregated prefill-decode model, which separates prompt processing from token generation by transferring caches between distributed compute nodes. It utilizes peer-to-peer orchestration to share and retrieve
Provides command line tools, HTTP endpoints, and a web dashboard to control and modify cache state.
Pythonamdcudafast
