1 repository
Awesome GitHub RepositoriesSLA-Based Scheduling
Request prioritization mechanisms that ensure execution order meets service level agreement guarantees.
Distinct from Request Schedulers: Specifically addresses the enforcement of SLA guarantees in the request order, distinct from general throughput optimization.
Explore 1 awesome GitHub repository matching artificial intelligence & ml · SLA-Based Scheduling. Refine with filters or upvote what's useful.
Awesome SLA-Based Scheduling GitHub Repositories
Găsește cele mai bune repo-uri cu AI.Vom căuta cele mai potrivite repository-uri folosind AI.
- modeltc/lightllm
ModelTC/LightLLM
3,901Vezi pe GitHubLightLLM is a high-performance serving framework for deploying and executing large language models. It functions as a multi-GPU inference engine and server capable of handling dense architectures, mixture-of-experts designs, and multimodal models that process both text and images. The system is distinguished by its specialized support for Mixture-of-Experts models using expert parallelism and fused kernels. It implements structured text generation through deterministic state machines and pushdown automata to enforce precise output formats. To optimize throughput, the framework employs specula
Manages the execution order of incoming requests to maintain service level agreement guarantees.
Pythondeep-learninggptllama
