Awesome GitHub RepositoriesModel Serving Platforms

Tabby is a self-hosted AI coding assistant designed to provide real-time code completion and interactive chat capabilities within development environments. By functioning as a private server application, it allows teams to maintain control over their infrastructure and data while integrating intelligent code generation directly into their existing workflows. The platform distinguishes itself through its repository-aware knowledge retrieval and multi-model orchestration. It indexes local and remote source code repositories and technical documentation into a searchable vector-based knowledge gr

OpenVINO is an AI inference engine and model serving platform designed to execute optimized deep learning models across CPUs, GPUs, and NPUs through a unified API. It includes a model optimization toolkit for converting, quantizing, and compressing models from various frameworks, alongside a specialized generative AI runtime for large language models. The project distinguishes itself through a plugin-based hardware acceleration layer that maps neural network operations to vendor-specific drivers. It features advanced execution mechanisms such as continuous batching, speculative decoding, and

Flyte is a distributed machine learning pipeline manager and MLOps workflow engine. It functions as a Kubernetes-native orchestrator used to coordinate data, models, and compute resources for executing machine learning pipelines and autonomous agents at scale. The platform provides specialized infrastructure for the full machine learning lifecycle, including a dedicated model serving platform to deploy trained models as scalable production-ready inference services. It also enables the coordination and state management of autonomous AI agents. The system manages scalable pipeline execution th