Repository navigation
fix(llmxrr): refuse to record failed exchanges; heal stored failures - #31
Merged
Merged
Conversation
xrr record mode saves whatever do() returns, errors included; no hook to skip a save. Transport error or 4xx/5xx landed in cassettes, then replayed forever; replay-first -update never retried it. - record mode: live call judged before session save; transport error or non-2xx -> ErrRefused (method, cassette path, status, masked body snippet), nothing written - refusal voids rest of recording: later misses fail same way, no dialing; SDK retries + stream fallback spend no live calls - replay-first record: stored failure (error envelope / non-2xx) treated as missing, re-recorded, overwritten - plain replay: stored failure -> ErrStoredFailure, never handed back as provider reply; later misses lead with that cause - passthrough unchanged - Exchange gains Refused + Error; e2e + tool-round harnesses report them before misses - CheckRecordable, Redact, CheckNoFailures exported; llm catalog cassette gated the same way - guard test: every committed testdata/cassettes dir scanned for key shapes + stored failures on plain go test - e2e: foo -tags xrr binary against local 200/400/500/closed-port stand-ins; refusal, no files, single provider hit, heal under record - CONTRIBUTING: refusal + heal semantics, catalog cassette row
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Recording no longer saves failed exchanges as cassettes, and a stored failure heals under
-update.Bug
xrr's
FileSession.recordsaves whateverdo()returns: a transport error as anerror:envelope, a non-2xx as a payload. Record mode replays first, so a stored failure was never a miss and-updatenever retried it.Fix (
internal/llmxrr.Transport, covers in-process tests and the-tags xrrbinary seam)ErrRefused, nothing written. Message names method, cassette path (no host,key=stripped), status and body start; keys,Bearertokens andRecordingKeysmasked; URL dropped from a wrapped*url.Error.-update, a stored failure counts as missing, is sent again, and the success overwrites it.ErrStoredFailure("re-record it"); a following miss leads with that cause.Exchange.Refused,Exchange.Error) and harnesses report these before misses.internal/llmcatalog cassette.TestCommittedCassettesscans every committed cassette dir for key shapes and stored failures on a plaingo test.xrr alpha.5 has no record filter or pre-save hook, so the check lives in foo's transport.
Tests
internal/llmxrr(200 saved; 400/500/closed port refused with nothing on disk and one provider hit; no secrets; heal; stored-failure replay error),internal/llm(503 refused), e2exrr_record_test.goon the real-tags xrrbinary. Local httptest servers only.Docs
CONTRIBUTING.md: refusal, heal, catalog cassette row.