1 Repo
Awesome GitHub RepositoriesRDMA Cache Streaming
Transferring prefill data directly into host memory across instances using RDMA to bypass GPU bottlenecks.
Distinct from Inference Cache Management: Focuses on RDMA-based streaming, distinct from general inference cache management.
Explore 1 awesome GitHub repository matching operating systems & systems programming · RDMA Cache Streaming. Refine with filters or upvote what's useful.
Awesome RDMA Cache Streaming GitHub Repositories
Finde die besten Repos mit KI.Wir suchen mit KI nach den am besten passenden Repositories.
- sgl-project/sglang
sgl-project/sglang
29,079Auf GitHub ansehenSglang is a high-performance inference engine and serving system designed for large language and multimodal models. It provides a programmable interface for orchestrating complex generation workflows, enabling developers to coordinate multi-turn dialogues, tool invocations, and reasoning chains through a domain-specific language. The platform is built to support production-scale deployments, offering an OpenAI-compatible API that allows for integration with existing application ecosystems. The system distinguishes itself through a disaggregated architecture that separates compute-intensive pr
Bypasses GPU bottlenecks by transferring prefill data directly into host memory across instances.
Pythonattentionblackwellcuda
