feat(libsy): prepare requests for routed candidates - #463
Draft
afourniernv wants to merge 2 commits into
Draft
Conversation
This was referenced Aug 17, 2026
Signed-off-by: Alex Fournier <afournier@nvidia.com>
Signed-off-by: Alex Fournier <afournier@nvidia.com>
afourniernv
force-pushed
the
afournier/switch-1253-libsy-target-prompts
branch
from
August 18, 2026 18:43
16dfb7c to
fb27c6d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Moves target-prompt policy into libsy, where the routing decision and ordered fallback candidates are known.
This is 2 of 3 for SWITCH-1253. It builds on #455, which provides the provider-safe request operation.
Before
A direct libsy host received a
RoutingOutcomeand prepared fallbacks itself, commonly by copying the published request and replacing the model:That works while every candidate receives the same request. It cannot safely select a different prompt per candidate, and preparing the first candidate too early would make a fallback inherit the first target's prompt.
Routing-time calls had the same split: libsy could ask a host to call a model, but the host had no candidate-aware request operation.
After
The host asks libsy to prepare the candidate it is about to call:
For routing-time calls, the same contract is available on
CallModel:The built-in Rust client uses the corresponding Rust methods. For fallback, libsy starts from the request before a target prompt was applied, stamps the new model, and applies only the new target's prompt. Custom Rust, Python, Relay, and other hosts no longer need to reproduce this policy.
What changes
with_target_prompts(...)policy in libsy.RoutingOutcome::request_for(...)andCallModel::request_for(...), with Python bindings for both.switchyard-llm-clientand the direct Python libsy host example/test path.Ownership and async behavior
Arc; requests do not copy the map per call.request_for(...)is called.await.API compatibility
The Python changes and the
CallModelmethods are additive.RoutingOutcomeitself landed onmainafter the last tagged release. This PR adds private preparation state and marks the struct#[non_exhaustive], so code tracking unreleasedmainthat constructs it with a struct literal or destructures every field will need to use its constructors/public fields instead.RoutingOutcome::route_to(...),RoutingOutcome::answered(...), field reads, and existing tagged APIs remain available.Not in this PR
targets.*.system_promptconfigurationThose are isolated in #464.
Validation
Algorithm.run_streamhosting covers prompted first and fallback candidates.Suggested review order
crates/libsy/src/core/algorithm.rs—RoutingOutcome,Driver, and candidate request contractscrates/libsy/src/core/target_prompts.rs— immutable prompt lookup policycrates/libsy-llm-client/src/run.rs— built-in retry/fallback consumercrates/libsy/src/algorithms/stage.rs— legacy Stage prompt compatibilitycrates/switchyard-py/src/libsy_bindings.rsandtests/test_libsy_minimal_bindings.py— direct Python host APIStack
targets.*.system_prompt, compatibility, docs, and integration testsThis PR's unique change is one signed commit,
fb27c6d5(18 files, +518/-151). GitHub currently compares the draft withmain, so it also displays PR1 below that commit. After #455 merges, this branch will be rebased onto the updatedmainto leave only the libsy layer in the displayed diff.