{"full_name":"NVIDIA-NeMo/Switchyard","owner":"NVIDIA-NeMo","name":"Switchyard","shard":"1","url":"https://github.com/NVIDIA-NeMo/Switchyard","description":"Switchyard lets LLM applications route traffic across models and providers while preserving native OpenAI and Anthropic API compatibility - enabling flexible model selection, benchmarking, and cost/performance optimization.","homepage":"","topics":[],"tag_pages":[],"language":"Rust","license":"Apache-2.0","avatar":"https://avatars.githubusercontent.com/u/213689629?v=4","created_at":"2026-05-19T19:06:33Z","pushed_at":"2026-11-17T21:59:32Z","metrics":{"stars":3631,"forks":376,"watchers":12,"issues":170,"prs":1145,"releases":5,"issues_open":48,"issues_closed":122,"prs_open":77,"prs_closed":268,"prs_merged":800,"discussions":4},"growth":{},"history":{"dates":["2026-11-18"],"stars":[3631],"forks":[376],"watchers":[12],"issues_open":[48],"prs_open":[77]},"boards":{},"same_owner":[{"full_name":"NVIDIA-NeMo/Megatron-Bridge","name":"Megatron-Bridge","shard":"1","boards":{}},{"full_name":"NVIDIA-NeMo/Speech","name":"Speech","shard":"1","boards":{}}],"issues":[{"number":723,"title":"feat(routing): add a TypeSafe-backed classifier router","comments":11,"created_at":"2026-09-16T17:22:53Z"},{"number":354,"title":"[Bug] A target's extra_body is applied to another target that shares its model id","comments":5,"created_at":"2026-08-11T14:22:53Z"},{"number":1168,"title":"91 second cuttoff - kills slower systems","comments":4,"created_at":"2026-10-28T14:55:20Z"},{"number":665,"title":"[feature] Serve intermediate judge calls in the LiteLLM routing plugin (classifier-backed routing)","comments":4,"created_at":"2026-09-10T04:52:22Z"},{"number":583,"title":"[feature] Decide where trusted request context should live","comments":4,"created_at":"2026-08-31T21:31:02Z"},{"number":1215,"title":"[feature] Emit diagnostics when codecs drop request fields, and on the stream path","comments":3,"created_at":"2026-11-04T14:15:31Z"},{"number":522,"title":"[feature] Stackable / Hierarchical routers","comments":3,"created_at":"2026-08-22T09:57:16Z"},{"number":345,"title":"feat(libsy): evict targets that reject a request as unservable","comments":3,"created_at":"2026-08-11T03:57:49Z"}],"discussions":[{"number":1065,"title":"humpyard: a gateway built on switchyard-libsy, with eligibility decided before the algorithm runs","comments":0,"category":"Show and tell","created_at":"2026-10-21T02:58:36Z"},{"number":988,"title":"YardMaster: a team gateway and setup UI for switchyard-server","comments":0,"category":"Show and tell","created_at":"2026-10-14T16:33:59Z"},{"number":427,"title":"Pre-Routing Target Filtering: Eligibility vs. Preference in Route Selection (Design Discussion)","comments":1,"category":"Ideas","created_at":"2026-08-14T15:43:56Z"},{"number":362,"title":"Utility as a gate & anonymizer?","comments":0,"category":"Ideas","created_at":"2026-08-11T18:02:17Z"}],"latest_release":{"name":"v0.3.0","tag":"v0.3.0","published_at":"2026-09-22T18:06:59Z","body":"# Switchyard v0.3.0\n\nSwitchyard 0.3.0 adds reusable runner integrations, Advisor Gate and Composite routing, Auto presets, and updated Rust and Python embedding APIs. It also fixes provider translation and tool-loop behavior and removes the legacy Python server and coding-agent launchers.\n\nThis is a breaking upgrade for library and Python CLI users. Review the migration notes below before upgrading.\n\n[Full changelog](https://github.com/NVIDIA-NeMo/Switchyard/blob/v0.3.0/CHANGELOG.md#030) · [Changes since v0.2.0](https://github.com/NVIDIA-NeMo/Switchyard/compare/v0.2.0...v0.3.0)\n\n## Routing and integrations\n\n- Advisor Gate lets an executor model work with a stronger advisor for review, approval, and redo decisions, with session review budgets and stall checkpoints.\n- Composite routing combines an LLM classifier with Stage Router. Auto presets provide capable and efficient Stage configurations.\n- The new `switchyard-runner` crate loads TOML deployments and runs routing in-process. NeMo Relay and LiteLLM integrations use the native routing path.\n- Sub-agent routing, per-target system prompts and reasoning effort, and per-client request deadlines provide more control over agent traffic.\n- DeepSWE v1.1 and NeMo Gym/LiteLLM examples document reproducible evaluation setups. Their results apply to the documented configurations, not every deployment.\n\n## Library and Python APIs\n\n- Python requires 3.10 or newer; the core package has no runtime dependencies.\n- Python streaming exposes `Step.CallModel` and `Step.Done`, with response dictionaries and async iterators.\n- Routing outcomes report ordered `selected_model_ids`, bounded decision metadata, and evidence through Rust, Python, logs, and OpenTelemetry.\n- The decision endpoint supports routing without executing the selected answer model.\n\n## Compatibility and correctness\n\n- Provider translation preserves tool allowlists, parallel-call limits, native tool history, reasoning signatures, citations, and terminal Responses output. Unsupported cross-format native output fails explicitly.\n- Streaming fixes retain late response IDs, fragmented tool names, empty tool inputs, and tool-use stop reasons.\n- Responses continuations return to the provider that holds their state. Cross-format continuations retain conversation history in process memory; records are lost on restart.\n- Media and tool-result translation preserves supported image, audio, document, and file payloads.\n- Credential redaction and header handling protect request trust boundaries. Failed or abandoned streams count as errors, and usage reporting preserves translated cache-write and thinking tokens.\n\n## Migration and removals\n\n- The Python `switchyard` command, `switchyard serve`, coding-agent wrappers, YAML route bundles, legacy server stack, and compatibility bindings are removed. Connect clients to the standalone `switchyard-server` with native TOML configuration.\n- Package extras `[server]`, `[all]`, `[cli]`, `[tracing]`, and `[affinity-redis]` are removed. The Python package declares no optional dependency extras.\n- `classify_trigger` replaces `session_affinity`; use `classify_trigger = \"new_session\"` for the previous session-affinity behavior.\n- Rust and Python embedding callers must adopt the updated step and routing-outcome APIs. Rust struct literals may also need the new `Response.upstream_headers` and `Metadata.subagent_identity_unsupported` fields.\n\n## Install\n\nAfter the tag-triggered package publishing workflow completes, install the Python bindings with:\n\n```bash\npip install nemo-switchyard==0.3.0\n```\n\nInstall the standalone server separately:\n\n```bash\ncargo install --locked switchyard-server --version 0.3.0\n```\n\nSee the [getting-started guide](https://github.com/NVIDIA-NeMo/Switchyard/blob/v0.3.0/docs/getting_started.md) for configuration and source builds.\n\n## Limitations\n\nThe standalone server remains a demo/evaluation component. Release validation covers Ubuntu 24.04 on Linux x86_64; other platforms are outside th","is_prerelease":false},"readme":true,"readme_zh":false,"news":[],"stale":false,"snapshot_at":"2026-11-17T21:58:29Z","history_start":"2026-09-06"}