1 Repo
Awesome GitHub RepositoriesRemote Cache Block Sharing
Exchanging metadata and access permissions for memory blocks across distributed nodes.
Distinct from Shared Memory Data Exchange: Distinct from Shared Memory Data Exchange: focuses on metadata and permission exchange for remote cache blocks rather than local zero-copy buffers.
Explore 1 awesome GitHub repository matching data & databases · Remote Cache Block Sharing. Refine with filters or upvote what's useful.
Awesome Remote Cache Block Sharing GitHub Repositories
Finde die besten Repos mit KI.Wir suchen mit KI nach den am besten passenden Repositories.
- ai-dynamo/dynamo
ai-dynamo/dynamo
6,112Auf GitHub ansehenDynamo is a distributed inference orchestration platform designed for large language models. It functions as a system to coordinate prefill and decode phases across GPU nodes, utilizing a multi-backend runtime adapter to connect engines like vLLM and TensorRT-LLM through a unified block-oriented memory interface. An OpenAI-compatible API server provides the frontend for integration with existing tools and clients. The project is distinguished by its disaggregated serving architecture, which separates prompt processing and token generation onto independent GPU pools to optimize throughput and
Exchanges block metadata and access permissions across nodes via high-speed interconnects for remote cache reuse.
Rust
