fix(provider): harden no-op regression test and setup.sh model fallback - #100
Merged
Conversation
…h model fallback Adversarial review of the merged #99 surfaced four issues: - The no-op regression test only asserted the stored config was equal, which holds whether or not save_config ran (the rebuilt entry is deep-equal to the stored one). Track save_config calls and assert zero on the no-op path. - The "***" <-> "no-key-required" normalization branch was untested (both sides used "***"). Add a cross-writer test: setup.sh's "no-key-required" vs runtime's "***" must still skip the save. - setup.sh fell back to ["auto"] when plugin.yaml omitted `models:`, diverging from backend.STATIC_MODELS (4 models) and forcing a one-time rewrite on first load. Fall back to the same built-in catalog. - setup.sh iterated a scalar `models:` value character-by-character (e.g. `models: auto` -> model "a"). Guard with isinstance(list), matching backend._load_model_override(). Full suite: 228 passed, 1 skipped.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #99. An adversarial review of the merged changes surfaced four
quality issues (no production bugs). This PR fixes all four.
1. No-op regression test was hollow (P2)
test_provider_register_noop_when_already_currentasserted onlysaved[0] == cfg, which holds whether or notsave_configran — therebuilt entry is deep-equal to the stored one. So the test did not lock in the
PR's headline behavior ("no rewrite on load"). The mock now records
save_configcalls, and the test asserts the call count is zero.2.
"***"↔"no-key-required"normalization untested (P3)The new test used
"***"on both sides, so the cross-writer normalizationbranch (the whole point of
_entries_equivalent) was never exercised. Addedtest_provider_register_noop_cross_writer_key: a config written bysetup.sh(
api_key: no-key-required) vs the runtime sentinel ("***") must still skipthe save.
3.
setup.shmodel fallback diverged frombackend.STATIC_MODELS(P3)When
plugin.yamlomittedmodels:,setup.shwrote{auto: {}}while theruntime rebuilt 4 models — causing a one-time rewrite (comment strip) on first
load.
setup.shnow falls back to the same built-in catalog asbackend.list_models()(documented duplication, sincesetup.shcannot importbackend.py— it depends onrequests).4. Scalar
models:iterated character-by-character (P3)models: auto(a scalar) made[str(m) for m in models]iterate the string'scharacters, yielding
model: "a". Added anisinstance(models, list)guard,matching
backend._load_model_override().Tests
tests/test_coverage_gaps.py: mock tracks save calls; no-op asserts zerosaves; new cross-writer key test.
tests/test_setup.py: new fallback-to-STATIC_MODELS and scalar-guard tests.Full suite: 228 passed, 1 skipped.
ruffclean (3 pre-existing N999).