feat(server): configure system prompts by target - #464
Draft
afourniernv wants to merge 3 commits into
Draft
Conversation
This was referenced Aug 17, 2026
Signed-off-by: Alex Fournier <afournier@nvidia.com>
Signed-off-by: Alex Fournier <afournier@nvidia.com>
Signed-off-by: Alex Fournier <afournier@nvidia.com>
afourniernv
force-pushed
the
afournier/switch-1253-server-target-prompts
branch
from
August 18, 2026 18:43
6731a11 to
87f9f54
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
system_promptto native-server targets so the prompt follows the model that actually serves an answer.This is 3 of 3 for SWITCH-1253. It builds on the translation operation in #455 and the libsy candidate contract in #463.
Use case
Different models can need different standing instructions or model-specific prompt tuning. The caller addresses a Switchyard route and may not know which target will answer, especially after fallback. Switchyard therefore selects the prompt with the answer candidate rather than requiring the caller to predict the route.
Before
Only Stage Router could configure different prompts, and those prompts were tied to its
capableandefficientroles:Other routers could not use target-specific prompts. Prompt selection also happened before a client fallback knew which target would ultimately answer.
After
Prompts live on the targets they describe:
Any router can select those targets. Switchyard applies the prompt when it prepares each answer candidate, so a fallback receives its own prompt rather than the first target's.
The legacy Stage fields remain supported. If both forms configure the same target, the target-level
system_promptwins.Call boundary
This uses libsy's answer-call distinction rather than a router allowlist. Routers that return a terminal
RoutingOutcomeinherit the behavior; routers that produce an answer while routing useDriver::call_answer_model(...), as escalation and Advisor do here.Configuration checks
Validation
cargo fmt --all --check, workspace Clippy, and the full non-PyO3 Rust workspace passed.Suggested review order
crates/switchyard-server/src/config.rs— target field, effective-prompt resolution, and conflict validationcrates/switchyard-server/src/lib.rs— explicit count-tokens preparationcrates/switchyard-server/tests/server.rs— routing, fallback, judge, and count-tokens behaviorcrates/switchyard-server/CONFIGURATION.mdanddocs/— user-facing schema and Stage compatibilityStack
targets.*.system_prompt, compatibility, docs, and integration testsThis PR's unique change is one signed commit,
87f9f541(6 files, +190/-52). GitHub currently compares the draft withmain, so it also displays PRs 1 and 2 below that commit. After the parent PRs merge, this branch will be rebased onto the updatedmainto leave only the server layer in the displayed diff.