1 repository
Awesome GitHub RepositoriesRemote Cache Block Sharing
Exchanging metadata and access permissions for memory blocks across distributed nodes.
Distinct from Shared Memory Data Exchange: Distinct from Shared Memory Data Exchange: focuses on metadata and permission exchange for remote cache blocks rather than local zero-copy buffers.
Explore 1 awesome GitHub repository matching data & databases · Remote Cache Block Sharing. Refine with filters or upvote what's useful.
Awesome Remote Cache Block Sharing GitHub Repositories
Găsește cele mai bune repo-uri cu AI.Vom căuta cele mai potrivite repository-uri folosind AI.
- ai-dynamo/dynamo
ai-dynamo/dynamo
6,112Vezi pe GitHubDynamo is a distributed inference orchestration platform designed for large language models. It functions as a system to coordinate prefill and decode phases across GPU nodes, utilizing a multi-backend runtime adapter to connect engines like vLLM and TensorRT-LLM through a unified block-oriented memory interface. An OpenAI-compatible API server provides the frontend for integration with existing tools and clients. The project is distinguished by its disaggregated serving architecture, which separates prompt processing and token generation onto independent GPU pools to optimize throughput and
Exchanges block metadata and access permissions across nodes via high-speed interconnects for remote cache reuse.
Rust
