NVIDIA-NeMo/Switchyard

解决的问题

Switchyard solves the problem of API incompatibility between coding agents (which expect OpenAI or Anthropic APIs) and open-source model servers (which often expose OpenAI-compatible endpoints but not Anthropic's format). It also addresses the need for flexible, multi-model routing with cost/quality trade-offs, enabling A/B testing, tiered routing, and custom routing logic without modifying the agent or model server.

能力范围

Developers can build production-grade LLM gateways that route traffic across models (e.g., vLLM, NVIDIA NIM, Ollama) with protocol translation, implement intelligent routing strategies like LLM-as-classifier or signal-driven stage routing, and embed routing algorithms directly into their own Rust applications. Real-world use cases include running Claude Code or Codex against open-source models, cost-optimized multi-tier serving, and benchmarking different models. The ceiling is high: with custom algorithms and library embedding, it can serve as the core routing engine for any LLM-based system.

结论

Switchyard is ideal for developers and organizations that want to use open-source models with existing coding agents or need a flexible, programmable routing layer for LLM traffic. It should be avoided by those seeking production-ready stability, as it is pre-alpha and explicitly not for production use. The key trade-off is its powerful flexibility and protocol translation capabilities versus its immaturity and rapidly changing API.