Conversation
Generated-By: PostHog Desktop Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589
Merge posthog/functional-a3-entries (1443587) into B1. The two conflicts are import lines: B1 moved AGENT_ERROR_CODE to @agent/error-map, so the fix's imports of it from @shared/errors follow. Generated-By: PostHog Desktop Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589
Merge posthog/functional-b1-pr (1d40129), which carries A3's 2aee250 and 1443587, into B2. B2 and the A3 fix both built host cancellation. The merge keeps one design: - The host signal travels on the agent config, and harnesses return A3's typed AgentResult. B2's extra `config.signal` argument and its `{}` results are gone. - A host cancellation replaces a success or the harness's own abort. A failure the run already decided stays the outcome. Host cancellations use B2's hostAborted() so runAgent, the sequences and runProgram report the same code and message. - runAgent keeps B2's run-installed skill cleanup on every non-success ending, inside A3's hardened snapshot and report flush. - B2's project-skill preflight, inference auth provider, program commandments and switchboard data stay, ported to the typed result. - Every runProgram failure now carries an ErrorCode, as the A3 contract requires: thrown errors go through classifyRunFailure, and declined approvals use AgentAbort. Generated-By: PostHog Desktop Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589
The agent, a program and the full TUI each run end to end on their own, so e2e follows that split: - `pnpm test:e2e:tui` runs the real TUI in a PTY (scripts/tui-snapshots). - `pnpm test:e2e:programs` runs `runProgram` with no TUI, store or session; the script is the host. - `pnpm test:e2e:agent` runs `runAgent` on a `quack` skill served from a loopback skills server, in an empty directory. All three read one env contract through `e2e-harness/surface-e2e.ts`. Retired: the dead jest suite under `e2e-tests/` with its fixture apps, `bin/test-e2e`, the `NODE_ENV=test` MSW hook in `bin.ts`, and the `run-live.ts` runner with `e2e-harness/live-e2e-checks.ts`. Fixtures, app copies and assertions stay in wizard-workbench. Generated-By: PostHog Desktop Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589
Merge posthog/functional-b-integration (42a506a) into B3. The one conflict is a test import: B3 keeps its UI reducer import, and the fix's ErrorCodes import joins it. Generated-By: PostHog Desktop Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589
🧙 Wizard CIRun the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands: Test all apps:
Test all apps in a directory:
Test an individual app:
Show more apps
Test against a Context Mill branch:
Add Results will be posted here when complete. |
The registry only maps integrations to framework configs, so it lives beside them. Importers now use @programs/frameworks/registry. Generated-By: PostHog Desktop Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589
Generated-By: PostHog Desktop Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589
…-store imports DiscoveredFeature moves to src/shared/discovered-feature.ts and the session re-exports it, so run-program.ts no longer reaches the session and its allowlist row goes. program-store.ts imports through @agent/types and @shared/api. Generated-By: PostHog Desktop Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589
Generated-By: PostHog Desktop Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589
Main's new task-stream sync lands beside the moved task stream, so its importers use @programs/task-stream/wizard-run-sync and its edges join the known task-stream rows. Generated-By: PostHog Desktop Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589
Generated-By: PostHog Desktop Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589
Main's OAuth session now owns when to refresh. runProgram configures it with the invocation's login and the refresh-token grant, now in credentials.ts, and every rotation lands in data through onRefreshed. Generated-By: PostHog Desktop Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589
…agnostics - The AI SDK stamp moves to posthog-integration/ai-sdk-stamp.ts, and detect.ts re-exports it. runProgram no longer loads detection or the registry. - A cancel during the pre-run refresh keeps the rotated token in data and in the shared OAuth session. - runSessionProgram logs each runProgram diagnostic. Generated-By: PostHog Desktop Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589
…abilities Generated-By: PostHog Desktop Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589
…Context RunnerContext now has log.warn, like CiRunnerContext's log, so a helper shared by run and ciPreRun takes one type. The posthog-cli pre-install takes the runner's log instead of a callback. Generated-By: PostHog Desktop Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589
…b4-api-reference Generated-By: PostHog Desktop Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589
…data - The runAgent example removes Write, Edit and Bash instead of adding Read and Glob, and the field table says allowedTools adds to the base tools and disallowedTools removes them. - The runProgram field table says data and program snapshots hold tokens. - The registry link points at frameworks/registry.ts, and every line anchor points at its declaration again. Generated-By: PostHog Desktop Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589
…stream-retry Generated-By: PostHog Desktop Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589
The stream retry now matches only the drops Pi's own auto-retry misses, `upstream closed the stream` and `ECONNRESET`. Pi already retries `socket hang up` and `terminated`, so those no longer retry twice. Before each resume the failed turn leaves the transcript, the way Pi's retry does, and each wait is jittered by ±20%. A drop that outlasts the retries now fails with a message that points at the network or the gateway and suggests trying again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Sep 25, 2026
Merged
Removes the wizard's own stream retry loop. Pi's auto-retry already re-sends a failed turn with backoff, so the wizard now only observes it: each auto_retry_start is logged and captured as `agent turn retried`, and an auto_retry_end that gives up goes to error tracking. A turn that ends on a dropped model stream fails with a message that points at the network or the gateway and suggests trying again in a few minutes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gewenyu99
changed the title
The pi harness README gains an SVG state diagram of one turn: streaming, failure, pi's retry wait, the wizard's 401 re-mint, give up and cancel. Drawing it surfaced that a cancel during pi's retry wait ends with `auto_retry_end` and `finalError: "Retry cancelled"`, which was reported as a give-up. That case is no longer error-tracked. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gewenyu99
marked this pull request as ready for review
September 25, 2026 18:43
Two runnable scripts replace the inline examples. tsconfig includes docs/examples, so typecheck keeps them in step with the code. Generated-By: PostHog Desktop Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589
…stream-retry Generated-By: PostHog Desktop Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589
An error occurred while trying to automatically change base from
posthog/functional-b4-api-reference
to
posthog/functional-b-host-capabilities
September 25, 2026 23:58
Generated-By: PostHog Desktop Task-Id: d14e92bb-6ee1-49b5-8502-39cb80079589
gewenyu99
changed the base branch from
posthog/functional-b4-api-reference
to
main
September 27, 2026 15:26
gewenyu99
marked this pull request as draft
September 27, 2026 15:26
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Wizard runs on Pi sometimes lose their connection to the model mid-response, and one drop fails the whole run. It hits about 0.53% of runs, roughly 53 people over the past couple of months, so it's most likely network related. Pi has its own auto-retry, but the wizard can't see it: nothing records a retry, and a run that fails after one just says
upstream closed the stream.Related: #1320. Seen on main's Smoke Test run 36152887782, which failed task 7/10 on
upstream closed the stream.Changes
This relies on Pi's own retry rather than adding a second one. The wizard now tracks every Pi retry, sends the ones that give up to error tracking, and explains a dropped stream clearly. With that data, you can see how often drops happen and whether Pi recovers them.
agent turn retriedevent. One capture per Pi retry, with the attempt and error.captureExceptionwithstep: 'pi_auto_retry', skipped on a cancel.sequenceDiagram participant W as Wizard participant P as Pi participant G as AI gateway W->>P: run the task P->>G: stream the reply G-->>P: connection dropped rect rgba(59, 130, 246, 0.15) loop Pi auto-retry, when its pattern matches, up to 3 times P-->>W: auto_retry_start, captured as agent turn retried P->>G: re-send without the cut-off turn G-->>P: reply, or dropped again end end alt recovered P-->>W: turn done else gave up P-->>W: auto_retry_end, sent to error tracking W->>W: fail with a clear network or gateway message endPi owns the retry. The shaded part is what the wizard now sees and records. The full turn lifecycle, including cancel and the 401 re-mint, is drawn in the Pi harness README.
auto-retry.ts,index.ts,task.tsgateway.tsREADME.md,pi-turn-lifecycle.svgDetails
socket hang upandterminated. It doesn't match the gateway'supstream closed the streamorECONNRESET, so those still fail on the first drop. They now fail with the clear message, and the tracking shows how often.upstream closed the stream,upstream connection lost,ECONNRESET,socket hang upandterminated: "The connection to the PostHog AI gateway dropped mid-response, and retrying didn't fix it (…). This is usually a network problem or a brief gateway issue. Try again in a few minutes." The existing wrapper then adds "Please report this to: wizard@posthog.com".finalError: "Retry cancelled", and it isn't error-tracked.withGatewayRemintand the 401 re-mint are unchanged.Test plan
auto-retry.test.ts: a Pi retry is captured with its attempt and error. A retry that gives up goes tocaptureException, and neither a recovered retry nor a cancelled one does.gateway.test.ts: each dropped-stream error fails with the clear message.pnpm typecheckis clean, andvitest run src/agent/runner/harness/pipasses 161 tests.Created with PostHog Desktop
🤖 Generated with Claude Code