# nvidia-nemo/switchyard **Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/nvidia-nemo-switchyard).** _How this analysis was created: the description and tags below were written by an AI model that read this project's README and public documentation pages; stars, license and language come straight from the GitHub API. The model does not read the source code._ 3,494 stars · 357 forks · Rust · Apache-2.0 ## Links - GitHub: https://github.com/NVIDIA-NeMo/Switchyard - awesome-repositories: https://awesome-repositories.com/repository/nvidia-nemo-switchyard.md ## Description Switchyard is a Rust library for routing large language model requests. For each request it chooses which model and provider should respond, balancing answer quality, cost, and response time rather than relying on one fixed model. It is aimed at coding assistants and agent harnesses, where it picks the best available model for each step and can show the chosen model in a tool's status line. Routing decisions come from composable strategies. A language model can judge whether a cheaper, more efficient model can handle a task, and that verdict selects the model tier. Execution signals, such as recent tool activity and outcomes, can move an agent to a different model mid-task. When those signals are too uncertain, the classifier sets the default tier. Automatic routing can also be exposed as a named virtual model that agent tools select like any other model. Switchyard can be embedded as a core library in an existing gateway or agent harness, so the host keeps its own model clients and credentials. Thin adapters attach it to third-party LLM gateways without replacing them, and typed Node.js bindings let TypeScript applications request routing decisions in-process. Requests and responses keep their native OpenAI and Anthropic formats, so existing clients work unchanged while traffic is spread across providers. A local reference server lets you try routing with your own models and coding agents and confirm it is active. A benchmarking setup compares a full agent, model pool, and routing configuration against a single-model baseline on task success, cost, and response time across the same tasks. ## Tags ### Artificial Intelligence & ML - [Cost-Quality Model Routing](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/infrastructure/model-evaluation-and-validation/model-capability-assessment/model-capability-queries/capability-based-routing/cost-quality-model-routing.md) — Selects a model per request by weighing answer quality, cost, and response time rather than relying on one fixed model. - [Model Routing for Agents](https://awesome-repositories.com/f/artificial-intelligence-ml/agentic-systems-frameworks/integration-deployment/ai-agent-tooling/coding-agents/model-routing-for-agents.md) — Picks the best available model for each coding assistant step and reroutes to another model mid-task as tool activity changes. - [Model Request Routing](https://awesome-repositories.com/f/artificial-intelligence-ml/ai-model-clients/model-request-routing.md) — Chooses which model and provider handles each incoming request so traffic spreads across several models instead of one fixed choice. ([source](https://nvidia-nemo.github.io/Switchyard/)) - [Tiered Model Cascades](https://awesome-repositories.com/f/artificial-intelligence-ml/ai-model-clients/model-request-routing/multi-model-pipeline-routings/tiered-model-cascades.md) — Asks a model whether a cheaper, efficient model can handle a task, then routes the request to the matching model tier. ([source](https://github.com/nvidia-nemo/switchyard#readme)) - [Composable Routing Strategies](https://awesome-repositories.com/f/artificial-intelligence-ml/composable-routing-strategies.md) — Chains independent strategies like task classification and execution-signal analysis, with a classifier setting the default tier when signals are uncertain. - [Embeddable Model Routing Cores](https://awesome-repositories.com/f/artificial-intelligence-ml/embeddable-model-routing-cores.md) — Ships a standalone routing core that hosts embed to get model decisions while keeping their own clients and credentials. - [Model Routing Gateways](https://awesome-repositories.com/f/artificial-intelligence-ml/model-routing-gateways.md) — Implements a Rust library that routes each request to a model and provider, with decisions plugging into existing gateways and agent harnesses. - [Activity-Driven Model Switching](https://awesome-repositories.com/f/artificial-intelligence-ml/activity-driven-model-switching.md) — Moves an agent to a better-suited model mid-task based on recent tool activity and outcomes. - [Model Request Routing to Hosts](https://awesome-repositories.com/f/artificial-intelligence-ml/agent-communication-protocols/agent-to-agent-communication/model-request-routing-to-hosts.md) — Switches models mid-task by watching recent tool activity and outcome signals so an agent moves to a better-suited model. ([source](https://github.com/nvidia-nemo/switchyard#readme)) - [LLM Gateways](https://awesome-repositories.com/f/artificial-intelligence-ml/agent-gateways/llm-gateways.md) — Provides thin adapters that attach routing decisions inside existing third-party LLM gateways without replacing them or changing client connections. - [Cost-Aware Model Routers](https://awesome-repositories.com/f/artificial-intelligence-ml/ai-model-clients/model-request-routing/cost-aware-model-routers.md) — Sends routine tasks to cheaper model tiers using a language model verdict, reserving costlier models for tasks that need them. - [Model Routing Strategies](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/infrastructure/model-evaluation-and-validation/model-capability-assessment/model-routing-strategies.md) — Merges task classification with execution signals so a classifier sets the default model tier when signals are too uncertain. ([source](https://github.com/nvidia-nemo/switchyard#readme)) - [Model Aliases](https://awesome-repositories.com/f/artificial-intelligence-ml/model-configuration-managers/model-aliases.md) — Exposes automatic routing as a named virtual model that agent tools select like any real model while the backend is resolved per request. - [Multi-Provider LLM Gateways](https://awesome-repositories.com/f/artificial-intelligence-ml/multi-provider-llm-gateways.md) — Attaches to third-party LLM gateways so traffic spreads across providers while clients keep their native OpenAI and Anthropic connections. ### Development Tools & Productivity - [Task-Based Execution Models](https://awesome-repositories.com/f/development-tools-productivity/parallel-execution/task-execution-sequencing/task-based-execution-models.md) — Uses a language model to judge whether a cheaper model can handle a task and routes the request to the matching tier. ### Part of an Awesome List - [Routing Baseline Comparisons](https://awesome-repositories.com/f/awesome-lists/ai/model-evaluation-and-benchmarking/routing-baseline-comparisons.md) — Benchmarks a full agent, model pool, and routing configuration against a single-model baseline on success, cost, and latency across identical tasks. ([source](https://github.com/nvidia-nemo/switchyard#readme)) - [High-Performance Serving Systems](https://awesome-repositories.com/f/awesome-lists/devtools/high-performance-serving-systems.md) — Rust proxy and library for protocol translation and routing across LLM backends. ### Software Engineering & Architecture - [API Compatibility Layers](https://awesome-repositories.com/f/software-engineering-architecture/api-compatibility-layers.md) — Keeps native OpenAI and Anthropic request and response formats so existing clients work unchanged while traffic is redirected across providers. - [Embeddable Routing Libraries](https://awesome-repositories.com/f/software-engineering-architecture/embeddable-routing-libraries.md) — Embeds as a core library in an existing gateway or agent harness, leaving model clients and credentials with the host.