NVIDIA-NeMo/Switchyard
A router that swaps the model behind the scenes while your coding agent keeps talking the same way
Switchyard is a Rust-based proxy that lets coding agents like Claude Code or Codex keep using their native API format while their requests actually get served by other models such as vLLM, NVIDIA NIM, or Ollama. It translates between OpenAI and Anthropic API formats, so you can switch backend models or split traffic across several for testing without touching the agent's code. It's currently pre-alpha software, not meant for production use.
What it does
- Apps and agents keep using whatever format they already speak - OpenAI Chat, Anthropic Messages, or OpenAI Responses - while Switchyard translates the request into the backend model's native format behind the scenes.
- It supports multiple routing strategies: random traffic splitting, LLM-based classification to pick a route, a stage router that reads signals already in the conversation (like tool results or errors), and an escalation router that tries a weaker model first and lets a judge decide whether to escalate to a stronger one.
- It can be used three ways: as a standalone server (switchyard-server), through a command-line launcher (switchyard CLI) for tools like Claude Code, or embedded directly into your own Rust app as a library (switchyard-libsy).
- It exposes Prometheus metrics covering requests, errors, latency, tokens, and routing overhead for operational monitoring.
- The Rust project has 1927 GitHub stars and is released under the Apache 2.0 license, copyrighted by NVIDIA.
Why it matters
Teams wanting to cut costs or benchmark open-source models against proprietary ones can swap or A/B test backend models without changing a single line of their coding agent's code. That said, the maintainers themselves describe it as experimental, pre-alpha software whose API and algorithms are still expected to change significantly.
Terms in this repo
- proxy · an intermediary server that relays requests between a client and the real backend service
- OpenAI Chat / Anthropic Messages / OpenAI Responses · the distinct request-response API formats defined by OpenAI and Anthropic for LLM interactions
- vLLM, NVIDIA NIM, Ollama · serving engines used to run open-source or self-hosted LLMs
- Prometheus metrics · standardized measurements (request counts, latency, etc.) that monitoring tools can collect and display
- pre-alpha · an early development stage before official release, where features and APIs are still expected to change significantly
Repository description (English)
Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.
Open on GitHubTrending repos
- vorssaint/vorssaint-utilsOne free menu bar app replaces a dozen paid Mac utilities
- Alishahryar1/free-claude-codeA local proxy that lets coding AI agents run on 49 free or cheap model providers instead of one paid service
- freestylefly/awesome-gpt-image-2A library of 532 reverse-engineered prompts that turn GPT-Image2 into a predictable image-making tool
- block/buzzAn open-source workspace where humans and AI agents chat, code, and review in the same rooms
- NousResearch/hermes-agentNous Research's Hermes is an AI agent that gets smarter the more you use it
- virgiliojr94/book-to-skillA tool that turns technical book PDFs into on-demand reference skills for AI coding agents
- VoltAgent/awesome-agent-skillsA single hub collecting over 1000 'how-to' manuals that make AI coding assistants act like experts
- anthropics/claude-plugins-communityA shared shelf where anyone's Claude add-ons get listed for install