1 Repo
Awesome GitHub RepositoriesRuntime State Controllers
Utilities for inspecting metadata, updating weights, and clearing caches during active inference.
Distinct from Model Weight Management: Focuses on active runtime state management rather than static weight file management.
Explore 1 awesome GitHub repository matching artificial intelligence & ml · Runtime State Controllers. Refine with filters or upvote what's useful.
Awesome Runtime State Controllers GitHub Repositories
Finde die besten Repos mit KI.Wir suchen mit KI nach den am besten passenden Repositories.
- sgl-project/sglang
sgl-project/sglang
29,079Auf GitHub ansehenSglang is a high-performance inference engine and serving system designed for large language and multimodal models. It provides a programmable interface for orchestrating complex generation workflows, enabling developers to coordinate multi-turn dialogues, tool invocations, and reasoning chains through a domain-specific language. The platform is built to support production-scale deployments, offering an OpenAI-compatible API that allows for integration with existing application ecosystems. The system distinguishes itself through a disaggregated architecture that separates compute-intensive pr
Inspects model metadata, updates weights dynamically without restarting, and clears internal caches to maintain performance.
Pythonattentionblackwellcuda
