1 repo

Awesome GitHub RepositoriesOnline Model Servers

Services that provide real-time model inference and chat completions via standard API protocols.

Explore 1 awesome GitHub repository matching artificial intelligence & ml · Online Model Servers. Refine with filters or upvote what's useful.

We'll search the best matching repositories with AI.

vllm-project/vllm
vllm-project/vllm
70,745GitHubView on GitHub
vLLM is a high-throughput inference engine designed for the efficient serving and execution of large language models. It functions as a production-ready distributed model server, providing standard API protocols for online serving while also supporting offline batch processing. The system is built to maximize token gen
Pythonamdblackwellcuda

1 repo

Services that provide real-time model inference and chat completions via standard API protocols.

Explore 1 awesome GitHub repository matching artificial intelligence & ml · Online Model Servers. Refine with filters or upvote what's useful.

We'll search the best matching repositories with AI.

vllm-project/vllm
vllm-project/vllm
70,745GitHubView on GitHub
vLLM is a high-throughput inference engine designed for the efficient serving and execution of large language models. It functions as a production-ready distributed model server, providing standard API protocols for online serving while also supporting offline batch processing. The system is built to maximize token gen
Pythonamdblackwellcuda

Awesome Online Model Servers GitHub Repositories