Repository navigation
Replay provider streams interrupted after partial output - #78
Merged
Merged
Conversation
A retryable failure (transport, idle timeout, in-stream overload, rate limit) after partial text, reasoning or tool calls now discards the attempt and re-sends the same request within the existing retry budget, instead of settling the Turn as interrupted. Replay ends at protocol completion; tool calls only dispatch after the stream completes. - cyber-llm: open_with_retry returns a replaying stream that yields LlmEvent::Discarded before the new attempt; retries before and after output share one budget. collect drops discarded output, keeps usage. - Drain records reasoning items and tool calls with the text when the stream ends, so nothing of a discarded attempt reaches history. Usage a discarded attempt reported is recorded as a failed turn usage.recorded.1 unless a reservation already billed it. - session.retry carries message_id and discarded; the TUI and web dashboard clear the partial answer, exec stream-json prints a retry line. - OpenSpec change retry-interrupted-streams (Retry policy, eager execution durability, exec stream-json schema), docs and ROADMAP.
This was referenced Oct 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Current merge review at
ee5c453d: This head integrates merged #77/#81. The retry API and usage-limit clock exports are both retained, and the dashboard bundle is regenerated from combined sources. Local integration checks pass: eight real-HTTP transport-fault tests, fourteen usage-limit tests, the runtime replay/history regression, 48 dashboard tests and typecheck, Rust formatting and strict OpenSpec validation. New-head hosted CI is pending; the earlier full-workspace results below apply to the preceding implementation head.A provider stream that fails mid-answer with a retryable error (transport, the stream idle timeout, in-stream overload, rate limit) now discards the partial answer and re-sends the same request within the existing retry budget. Before, any output ended retries and the Turn settled as interrupted.
Motivation: in the same-model pilot against Codex CLI,
build-inventory-cliwas lost totransport: error decoding response bodybefore any edits (pilot). The task passes 3/3 on a fresh run; a mid-stream break should cost one retry, not the task.Rules (OpenSpec change
retry-interrupted-streams)execute_toolsrecords the dispatch first) only after the stream completes. A completed but undispatched call has no effect yet, so discarding it is safe. The spec now states that dispatching a call early ends replay for that answer.tool.calledevents together with the text when the stream ends, instead of while it is open. This updates the eager-execution durability wording intool-registryandsession-runtime: calls are recorded no later than stream completion and before they start.usage.recorded.1(purposeturn, same message id), unless a budget reservation already billed it.Clients
session.retrycarriesmessage_idanddiscarded.cyber execstream-json prints aretryline.Tests
crates/cyber-llm/tests/transport_faults.rs(real HTTP faults):adapters.rs: an Anthropic in-stream overload is replayed, and a discarded attempt's usage is kept.cyber-server/tests/runtime.rs:a_mid_stream_retry_records_only_the_replayed_attempt: no stale data in the event log, onetool.called, the tool runs once, and history survives a restart;a_complete_tool_call_is_recorded_when_its_stream_completesreplaces the old "durable while the stream is open" test.a_discarded_attempt_clears_its_streamed_text, the execretryline, and a webmodel.test.tscase.Earlier implementation checks
-D warnings: clean.nextest --workspace --profile ci: 5249 of 5252 passed (2 flaky), 21 skipped.reserved_budgets::cancellation_during_opening_and_wall_deadline_keep_unknown_allocations,review_command::a_budget_stopped_review_workflow_exits_4_with_partial_findingsanduninstall_cli::manager_known_from_listings_only_is_not_run.docs/cli.md: unchanged.spec_lint: 0 errors, 25 warnings (baseline).openspec validate --all --strict: passes.Trade-off
If the process crashes mid-stream, that step's undispatched tool calls are now lost. Before, they were recorded and settled as "never dispatched". Neither version ran them.
Docs:
docs/local-core-guide.md, a dated note indocs/measurements/provider-stream-reliability-follow-up.md(measured results unchanged), ROADMAP. The "Stalled stream" scenario ofimprove-model-protocol-fidelitynow describes a replay.