1 Repo
Awesome GitHub RepositoriesModel
Interfaces for implementing custom logic to route requests between different AI models.
Distinct from Routing Extensions: Specifically targets AI model selection logic rather than database query routing.
Explore 1 awesome GitHub repository matching software engineering & architecture · Model. Refine with filters or upvote what's useful.
Awesome Model GitHub Repositories
Finde die besten Repos mit KI.Wir suchen mit KI nach den am besten passenden Repositories.
- lm-sys/routellm
lm-sys/RouteLLM
5,080Auf GitHub ansehenRouteLLM is a routing framework and traffic manager designed to direct prompts between high-capability and low-cost large language models. It functions as an API gateway that mimics the OpenAI specification to route requests across different model providers. The system optimizes operational costs by splitting traffic between model tiers based on predicted win rates and prompt complexity. It includes a calibration tool to analyze sample queries and determine the optimal cost-quality tradeoff for traffic distribution. The framework provides a tool for measuring the accuracy and cost efficiency
Provides an extensible interface for developers to implement custom routing algorithms and win-rate predictors.
Python
