3 repository-uri
Awesome GitHub RepositoriesModel Serving Platforms
Infrastructure for hosting and exposing machine learning models for inference.
Distinguishing note: Focuses on serving models as database objects for SQL-based inference.
Explore 3 awesome GitHub repositories matching artificial intelligence & ml · Model Serving Platforms. Refine with filters or upvote what's useful.
Awesome Model Serving Platforms GitHub Repositories
Găsește cele mai bune repo-uri cu AI.Vom căuta cele mai potrivite repository-uri folosind AI.
- qwenlm/qwen3
QwenLM/Qwen3
27,324Vezi pe GitHubQwen3 is a transformer-based large language model designed as a generative AI foundation for understanding, reasoning, and generating human language. It functions as a comprehensive ecosystem for model training, fine-tuning, and production-ready inference, providing the underlying architecture and weights necessary to build diverse artificial intelligence applications. The project distinguishes itself through extensive support for model quantization and distributed inference, enabling efficient execution across a wide range of hardware from consumer-grade devices to scalable cloud infrastruct
TGI Deployment — a named example documented in this learning resource.
Python - modular/modular
modular/modular
26,357Vezi pe GitHubModular is a unified machine learning development platform designed for building, compiling, and deploying high-performance neural network models. It provides a comprehensive execution engine that supports both local and production-grade inference, enabling developers to manage the entire model lifecycle from initial architecture definition to scalable, containerized service deployment. The platform distinguishes itself through a hardware-agnostic runtime that abstracts diverse silicon architectures, allowing models to execute efficiently across varied compute environments. It includes a spec
Packages inference engines and model weights into isolated environments to standardize deployment and scaling.
Mojoailanguagemachine-learning - ericlbuehler/mistral.rs
EricLBuehler/mistral.rs
6,597Vezi pe GitHubmistral.rs is an inference engine for large language models that runs locally and exposes models behind OpenAI and Anthropic-compatible APIs. It serves as a multi-model serving platform, capable of loading several models in a single server process with per-request routing and on-demand loading and unloading. The engine supports multimodal inference, processing text alongside images, video, audio, and speech inputs, and includes a quantized model deployment runtime that reduces memory use and speeds up inference on consumer hardware. The project distinguishes itself through an agentic tool exe
Serves multiple models from a single process with per-request routing and on-demand loading and unloading.
Rustllmrustuqff
