3 Repos
Awesome GitHub RepositoriesModel Serving Platforms
Infrastructure for deploying, scaling, and managing machine learning model inference services.
Distinguishing note: Focuses on the operational deployment and scaling of models, distinct from the model development process.
Explore 3 awesome GitHub repositories matching devops & infrastructure · Model Serving Platforms. Refine with filters or upvote what's useful.
Awesome Model Serving Platforms GitHub Repositories
Finde die besten Repos mit KI.Wir suchen mit KI nach den am besten passenden Repositories.
- tabbyml/tabby
TabbyML/tabby
33,605Auf GitHub ansehenTabby is a self-hosted AI coding assistant designed to provide real-time code completion and interactive chat capabilities within development environments. By functioning as a private server application, it allows teams to maintain control over their infrastructure and data while integrating intelligent code generation directly into their existing workflows. The platform distinguishes itself through its repository-aware knowledge retrieval and multi-model orchestration. It indexes local and remote source code repositories and technical documentation into a searchable vector-based knowledge gr
Manages service replicas and hardware-accelerated infrastructure to handle high-demand coding assistance tasks.
Rustaicodegencoding-assistant - openvinotoolkit/openvino
openvinotoolkit/openvino
10,414Auf GitHub ansehenOpenVINO is an AI inference engine and model serving platform designed to execute optimized deep learning models across CPUs, GPUs, and NPUs through a unified API. It includes a model optimization toolkit for converting, quantizing, and compressing models from various frameworks, alongside a specialized generative AI runtime for large language models. The project distinguishes itself through a plugin-based hardware acceleration layer that maps neural network operations to vendor-specific drivers. It features advanced execution mechanisms such as continuous batching, speculative decoding, and
Provides a high-performance server to host and expose deep learning and generative AI models via REST and gRPC.
C++aicomputer-visiondeep-learning - lyft/flyte
lyft/flyte
7,095Auf GitHub ansehenFlyte is a distributed machine learning pipeline manager and MLOps workflow engine. It functions as a Kubernetes-native orchestrator used to coordinate data, models, and compute resources for executing machine learning pipelines and autonomous agents at scale. The platform provides specialized infrastructure for the full machine learning lifecycle, including a dedicated model serving platform to deploy trained models as scalable production-ready inference services. It also enables the coordination and state management of autonomous AI agents. The system manages scalable pipeline execution th
Provides scalable infrastructure for deploying and managing trained machine learning model inference services.
Go
