NVIDIA-NeMo/Switchyard

Switchyard solves the problem of API incompatibility between coding agents (which expect OpenAI or Anthropic APIs) and open-source model servers (which often expose OpenAI-compatible endpoints but not Anthropic's format). It also addresses the need for flexible, multi-model routing with cost/quality trade-offs, enabling A/B testing, tiered routing, and custom routing logic without modifying the agent or model server.

Developers can build production-grade LLM gateways that route traffic across models (e.g., vLLM, NVIDIA NIM, Ollama) with protocol translation, implement intelligent routing strategies like LLM-as-classifier or signal-driven stage routing, and embed routing algorithms directly into their own Rust applications. Real-world use cases include running Claude Code or Codex against open-source models, cost-optimized multi-tier serving, and benchmarking different models. The ceiling is high: with custom algorithms and library embedding, it can serve as the core routing engine for any LLM-based system.