awesome-repositories.com
Blog
MCP
awesome-repositories.com

Discover the best open-source repositories with AI-powered search.

ExploreCurated searchesOpen-source alternativesSelf-hosted softwareBlogSitemap
ProjectMCP serverAboutHow we rankPress
LegalPrivacyTerms
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
cheaper-inference avatar

cheaper-inference/OmniRoute

0
View on GitHub↗
omniroute.online↗

OmniRoute

OmniRoute is a self-hosted AI gateway that puts one OpenAI-compatible endpoint and key in front of many chat, media, search, and local-model providers. It translates each request into the chosen provider's native format and maps the reply back. It is built to serve coding assistants and command-line agents, and it can configure them to send their model traffic through the gateway. It also catalogs free and keyless model tiers and uses them before falling back to paid capacity.

Routing and failover are central. Models can be chained into ordered combos with strategies such as priority, round-robin, cost-optimized, or quota-weighted. Combos can also fan a request out to several models with a judge merging the answers, or pass output from one step into the next. Automatic routing scores candidate models and shifts traffic away from degraded targets. Circuit breakers, cooldowns, and backoff bypass failing providers, and quota-aware scheduling spreads requests across accounts. A built-in prompt compression pipeline stacks engines that shrink prompts and tool output while preserving code, URLs, and JSON. The gateway also exposes its routing, provider, caching, and memory controls to outside agents through Model Context Protocol tools and agent-to-agent JSON-RPC endpoints.

Around that core are cost and quota tracking with per-key spend limits, a pricing catalog, and free-tier aggregation. Optional proxy-level conversation memory is off by default. Security features include scoped tokens, key-to-model restrictions, and masking of personal data and secrets. Observability covers live quota telemetry, routing flow graphs, traffic inspection, and webhook notifications. A large command-line interface handles setup, diagnostics, interactive chat, and control of a remote gateway, and an editor extension surfaces gateway models in the chat model picker.

OmniRoute installs globally from a package manager, serving a web dashboard and the API on one port. It also runs under Docker, Podman, systemd, Fly.io, Android Termux, a desktop app, or an installable web app, and can run headless as proxy-only. If no prebuilt native database binary matches, it falls back to a pure-JavaScript engine, so installation does not require build tools.

Features

  • OpenAI-Compatible APIs - Offers one OpenAI-compatible endpoint and key that translates requests into each provider's native format across many chat, media, search, and local providers.
  • OpenAI-Compatible Endpoints - Routes chat, media, search, and local-model requests through one OpenAI-compatible endpoint and key, converting to each provider's native format.
  • Free AI Tier Aggregators - Uses free and keyless model quotas first, falls back to paid providers, and compresses prompts and tracks spend to cut costs.
  • Free Tier Aggregators - Surfaces free-forever and keyless providers and tracks free budgets per model so requests are served at no cost before paid quota.

AI search

Explore more awesome repositories

Describe what you need in plain English — the AI ranks thousands of curated open-source projects by relevance.

Start searching with AI
85,439 stars·12,914 forks·TypeScript·MIT·0 views
  • Provider Failover - Shifts requests to a healthy provider automatically when a provider errors, rate-limits, or runs out of quota.
  • Model Routing Combos - Chains models into ordered combos whose targets are picked by strategies such as priority, round-robin, cost-optimized, or quota-weighted.
  • Quota Failover Routing - Chains models into combos with priority, round-robin, and cost-optimized strategies and shifts traffic to healthy providers when quota runs out or errors occur.
  • Request-Response Translation Layers - Translates each OpenAI-style request into the provider's native format and maps replies back so one endpoint fronts many providers.
  • Coding Agents and Assistants - Points coding assistants and command-line agents at one local endpoint, with setup helpers and MCP and A2A support for managing the gateway.
  • Agentic Coding Assistants - Points coding assistants and command-line agents such as Claude Code, Codex, Cursor, and Cline at a local or remote gateway, detecting installed agents.
  • Coding Assistant Model Routing - Lets any OpenAI-compatible coding client use one base URL, key, and model name, routing across providers with automatic fallback or to a chosen model.
  • AI Provider Gateways - Routes chat, media, search, and local model requests through one OpenAI-compatible endpoint and key to many providers.
  • Agent Gateways - Configures coding assistants and command-line agents to send model traffic through one gateway endpoint, with setup helpers and an editor model picker.
  • Coding Tool Profiles - Picks a provider and model interactively, previews settings with a dry run, then writes each coding tool's own configuration file to use the gateway.
  • Model Quota Aware Switching - Monitors remaining provider quotas and live telemetry to spread requests across accounts, using free allowances before paid capacity.
  • Compression Pipeline Hooks - Runs prompts through an ordered stack of independently tunable compression engines that shrink tokens while protecting code, URLs, and JSON.
  • Goal-Based Model Selection - Selects a healthy model from one model name using live scoring, with variants tuned for coding quality, speed, cost, or quota headroom.
  • MCP Tool Servers - Serves a large set of gateway tools to editors and coding agents over MCP using stdio, HTTP, or server-sent events with scoped authentication.
  • Auto-Scoring Self-Healing Routes - Scores candidate models on many weighted factors and automatically shifts traffic away from degraded targets using selectable mode packs.
  • Distribution Strategies - Offers selectable strategies for spreading requests across targets, including priority, round-robin, weighted, cost, quota headroom, latency, and context-aware ranking.
  • Provider Account Onboarding - Links personal provider accounts and credentials from a dashboard so chat, media, and search providers are reachable through one endpoint and key.
  • Multi-Stage Compression Pipelines - Runs prompts and tool output through an ordered stack of compression engines such as filler removal, JSON filtering, and cross-turn deduplication without client changes.
  • Prompt Compression Methods - Stacks composable engines that shrink prompts and tool output while preserving code, URLs, and JSON, escalating only to fit the context window.
  • A2A Protocol Servers - Runs an agent-to-agent JSON-RPC server with a published agent card, task tracking, and streamed results so other agents can delegate work.
  • LLM Token Cost Tracking - Returns per-request cost and usage headers, reports cache-hit savings, and enforces per-key spend quotas across AI model usage.
  • Quota-Aware Account Scheduling - Spreads requests across provider accounts according to remaining quota so free allowances are consumed fairly before any paid capacity.
  • Provider Circuit Breakers - Tracks provider health with circuit breakers, cooldowns, and exponential backoff so failing providers are bypassed and traffic shifts to healthy ones.

    • Star history

    Star history chart for cheaper-inference/omnirouteStar history chart for cheaper-inference/omniroute

    How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.

    Frequently asked questions

    What does cheaper-inference/omniroute do?

    OmniRoute is a self-hosted AI gateway that puts one OpenAI-compatible endpoint and key in front of many chat, media, search, and local-model providers. It translates each request into the chosen provider's native format and maps the reply back. It is built to serve coding assistants and command-line agents, and it can configure them to send their model traffic through the gateway. It also catalogs free and keyless model tiers and uses them before falling back to paid…

    What are the main features of cheaper-inference/omniroute?

    The main features of cheaper-inference/omniroute are: OpenAI-Compatible APIs, OpenAI-Compatible Endpoints, Free AI Tier Aggregators, Free Tier Aggregators, Provider Failover, Model Routing Combos, Quota Failover Routing, Request-Response Translation Layers.

    Which projects share features with cheaper-inference/omniroute?

    Projects with overlapping indexed features include: diegosouzapw/omniroute — OmniRoute is a unified LLM API gateway that connects multiple AI providers to a single endpoint. Its primary purpose… ekkolearnai/ekko-studio — Ekko Studio is a self-hosted AI agent workspace that runs several agents together in shared chats and keeps sessions… experientiallabs/experiential — Experiential is an open-source LLM gateway and router. It exposes OpenAI-compatible and Anthropic-compatible APIs in… tashfeenahmed/freellmapi — freellmapi is a self-hosted gateway that combines the free tiers of many language model providers behind one local,… builderz-labs/mission-control — Mission Control is a self-hosted web dashboard that acts as a control plane for autonomous AI coding agents. It lets… ombharatiya/ai-system-design-guide — This is a written reference for designing, building and operating production AI systems, aimed at engineers who work…

    Projects sharing features with OmniRoute

    These projects share indexed features with OmniRoute. Shared tags can include platform or build tooling; verify the primary use case before treating a result as a replacement.
    • diegosouzapw/omniroutediegosouzapw avatar

      diegosouzapw/OmniRoute

      6,391View on GitHub↗

      OmniRoute is a unified LLM API gateway that connects multiple AI providers to a single endpoint. Its primary purpose is to simplify the integration of various AI models into tools and agents by translating different provider formats into a standardized API. The project distinguishes itself through a multi-strategy request routing system that optimizes for cost, speed, and availability, including automatic model fallbacks and a circuit-breaker resilience model to isolate provider failures. It employs a local-first security posture, using AES-256-GCM encryption to store API keys and conversatio

      TypeScript
      View on GitHub↗6,391
    • ekkolearnai/ekko-studioEKKOLearnAI avatar

      EKKOLearnAI/ekko-studio

      11,715View on GitHub↗

      Ekko Studio is a self-hosted AI agent workspace that runs several agents together in shared chats and keeps sessions and data on the user's own machine. It streams replies, tool traces, and generated-file previews across persistent sessions, and it hosts group rooms where multiple agents answer when mentioned. It also detects, installs, configures, and launches third-party command-line coding agents from one interface, with built-in terminals, session history, a workspace picker, and file diffs. Beyond chat, the workspace includes a visual canvas for building runnable agent workflows with con

      TypeScriptagentai-agentchat-ui
      View on GitHub↗11,715
    • experientiallabs/experientialexperientiallabs avatar

      experientiallabs/experiential

      15,193View on GitHub↗

      Experiential is an open-source LLM gateway and router. It exposes OpenAI-compatible and Anthropic-compatible APIs in front of hosted models, models reached with your own provider keys, and local models, all under one base URL and one bearer key. Existing OpenAI and Anthropic SDK clients, as well as coding agents, can be pointed at it by changing only the endpoint. The project states that usage carries zero markup. What sets it apart is that routing is learned from your own traffic. It captures traces from desktop coding agents and imports past activity from supported monitoring tools, databas

      Python
      View on GitHub↗15,193
    • tashfeenahmed/freellmapitashfeenahmed avatar

      tashfeenahmed/freellmapi

      48,827View on GitHub↗

      freellmapi is a self-hosted gateway that combines the free tiers of many language model providers behind one local, OpenAI-compatible endpoint. Applications talk to a single address with a single bearer token, while the router chooses a provider and model for each request and retries on another when one errors or hits its limit. Provider keys are stored encrypted in a local database, and per-key rate limits are tracked against per-minute and per-day ceilings. Beyond the OpenAI-style format for chat, completion, embedding, image, video, and speech requests, the router also accepts Anthropic, G

      TypeScript
      View on GitHub↗48,827
    Compare all 30 related projects→