NVIDIA-NeMo/Switchyard
Switchyard
Switchyard is a Rust library for routing large language model requests. For each request it chooses which model and provider should respond, balancing answer quality, cost, and response time rather than relying on one fixed model. It is aimed at coding assistants and agent harnesses, where it picks the best available model for each step and can show the chosen model in a tool's status line.
Routing decisions come from composable strategies. A language model can judge whether a cheaper, more efficient model can handle a task, and that verdict selects the model tier. Execution signals, such as recent tool activity and outcomes, can move an agent to a different model mid-task. When those signals are too uncertain, the classifier sets the default tier. Automatic routing can also be exposed as a named virtual model that agent tools select like any other model.
Switchyard can be embedded as a core library in an existing gateway or agent harness, so the host keeps its own model clients and credentials. Thin adapters attach it to third-party LLM gateways without replacing them, and typed Node.js bindings let TypeScript applications request routing decisions in-process. Requests and responses keep their native OpenAI and Anthropic formats, so existing clients work unchanged while traffic is spread across providers.
A local reference server lets you try routing with your own models and coding agents and confirm it is active. A benchmarking setup compares a full agent, model pool, and routing configuration against a single-model baseline on task success, cost, and response time across the same tasks.
