Skip to content

fix(ask): arm the wizard_ask timeout per question, not per request - #1368

Merged
Gilbert09 merged 1 commit into
mainfrom
posthog/ask-timeout-per-question
Oct 2, 2026
Merged

Gilbert09 merged 1 commit into
mainfrom
posthog/ask-timeout-per-question

Conversation

@Gilbert09

Copy link
Copy Markdown
Member

Problem

The seeded data-source step is the one part of a run that stops to collect live credentials, and the connection forms behind it are long: a database source asks for most of host, port, database, user, password, schema and TLS/tunnel settings, and wizard_ask carries up to twelve questions in one call for exactly that reason.

The ask overlay walks those questions one at a time and resolves the request once, at the end of the walk. The ask bridge, meanwhile, documents a per-question timeout but armed a single timer when the request opened and raced it against the whole request. So a one-line question and a nine-field connection form got the same allowance, and when it expired on a user who was still working through the form:

  • every field they had already typed was discarded (all answers come back as the cancelled sentinel), and
  • the agent was told, correctly by its own contract but wrongly in fact, that nobody was at the terminal — guidance that tells it to stop asking for the rest of the run and hand over links instead.

Telemetry for this flow shows the losses concentrated on exactly the long forms: prompts carrying six or more questions are answered far less often than one-question prompts, and they account for a disproportionate share of timed-out asks, while short prompts are answered readily. Nothing in the data distinguishes a user who walked away from one who was cut off mid-form, because nothing recorded how far they got.

Why: a user who already agreed to connect a source, and is part-way through typing its credentials, should not lose the whole source to a clock that was never meant to measure the form.

Changes

  • The timeout measures silence, not the age of the request. createWizardAskBridge now hands the host an onAnswer callback alongside the question's abort signal and re-arms its timer each time it fires. A request nobody touches still expires after the same wait, so single-question prompts — the large majority — behave exactly as before.
  • The overlay reports progress. WizardAskScreen calls store.noteAskProgress() as it advances to the next question; WizardStore.requestQuestion takes the callback and drops it when the request settles. AgentInteraction.ask and WizardUI.requestQuestion carry it through; hosts that answer in one shot can ignore it.
  • wizard_ask cancelled carries questions_answered. A count, never a value, so a request nobody touched is distinguishable from one abandoned part-filled — which is what makes this change measurable.

Test plan

  • vitest run — 3437 tests pass across 204 files.
  • tsc --noEmit, eslint and prettier --check clean on the touched files.
  • New bridge tests: the timer re-arms on each answer (a request that outlives the raw allowance survives as long as the user keeps answering, and still expires once they stop), and the cancelled event records how many questions were answered.
  • New store test: the progress callback reaches the bridge and is dropped once the request resolves.

Created with PostHog Desktop

🤖 Generated with Claude Code

Copilot AI lite review requested due to automatic review settings September 27, 2026 13:12
@Gilbert09 Gilbert09 self-assigned this Sep 27, 2026
@Gilbert09
Gilbert09 requested a review from a team as a code owner September 27, 2026 13:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@github-actions

Copy link
Copy Markdown

🧙 Wizard CI

Run the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands:

Test all apps:

  • /wizard-ci all

Test all apps in a directory:

  • /wizard-ci ai-observability
  • /wizard-ci basic-integration
  • /wizard-ci feature-flags
  • /wizard-ci mcp-analytics
  • /wizard-ci replay-vision
  • /wizard-ci revenue
  • /wizard-ci self-driving
  • /wizard-ci warehouse
  • /wizard-ci warehouse-seeded

Test an individual app:

  • /wizard-ci ai-observability/anthropic
  • /wizard-ci ai-observability/google-adk
  • /wizard-ci ai-observability/groq
Show more apps
  • /wizard-ci ai-observability/manual-capture
  • /wizard-ci ai-observability/openai
  • /wizard-ci ai-observability/openai-agents
  • /wizard-ci ai-observability/opentelemetry
  • /wizard-ci ai-observability/vercel-ai
  • /wizard-ci basic-integration/android
  • /wizard-ci basic-integration/angular
  • /wizard-ci basic-integration/astro
  • /wizard-ci basic-integration/django
  • /wizard-ci basic-integration/fastapi
  • /wizard-ci basic-integration/flask
  • /wizard-ci basic-integration/flutter
  • /wizard-ci basic-integration/javascript-node
  • /wizard-ci basic-integration/javascript-web
  • /wizard-ci basic-integration/laravel
  • /wizard-ci basic-integration/next-js
  • /wizard-ci basic-integration/nuxt
  • /wizard-ci basic-integration/python
  • /wizard-ci basic-integration/rails
  • /wizard-ci basic-integration/react-native
  • /wizard-ci basic-integration/react-router
  • /wizard-ci basic-integration/sveltekit
  • /wizard-ci basic-integration/swift
  • /wizard-ci basic-integration/tanstack-router
  • /wizard-ci basic-integration/tanstack-start
  • /wizard-ci basic-integration/vue
  • /wizard-ci feature-flags/django
  • /wizard-ci feature-flags/next-js
  • /wizard-ci mcp-analytics/custom-dispatcher
  • /wizard-ci mcp-analytics/typescript-sdk
  • /wizard-ci replay-vision/javascript-node
  • /wizard-ci replay-vision/next-js
  • /wizard-ci replay-vision/react-vite
  • /wizard-ci revenue/stripe
  • /wizard-ci self-driving/astro
  • /wizard-ci self-driving/fastapi
  • /wizard-ci self-driving/nuxt
  • /wizard-ci self-driving/react-router
  • /wizard-ci self-driving/sveltekit
  • /wizard-ci warehouse/monorepo-env
  • /wizard-ci warehouse/multi-source-next
  • /wizard-ci warehouse/stripe-node
  • /wizard-ci warehouse/zero-source
  • /wizard-ci warehouse-seeded/next-stripe
  • /wizard-ci warehouse-seeded/next-stripe-declined

Test against a Context Mill branch:

  • /wizard-ci all context-mill:my-branch

Add context-mill:<branch> to any command above to pin the Context Mill branch. It defaults to main.

Results will be posted here when complete.

@Gilbert09 Gilbert09 added the stamphog Auto-merge stamp for automated PRs label Sep 27, 2026 — with PostHog
Rebase the timeout fix onto the current main branch while preserving the TUI layer move.
@Gilbert09
Gilbert09 force-pushed the posthog/ask-timeout-per-question branch from 5a9041e to 0138950 Compare October 1, 2026 10:29

Copy link
Copy Markdown
Member Author

The branch is up to date with main, all 18 checks are green, and there are no unresolved review threads. The remaining merge gate is human approval from Team Wizard & Docs.

🦉 via talyn.dev

@johncwaters johncwaters left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Automated review. Not written by a human.

Comment thread src/tui/store.ts
private _resolveManualAuthCode: ((code: string) => void) | null = null;

/** Resolves the in-flight wizard_ask request. */
private _noteAskProgress: (() => void) | null = null;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Automated review. Not written by a human.

Nit: the doc comment "Resolves the in-flight wizard_ask request." now sits above _noteAskProgress instead of _resolvePendingQuestion. Move the new field above that comment or give it its own one-line comment.

@Gilbert09
Gilbert09 merged commit 6cb4093 into main Oct 2, 2026
18 checks passed
@Gilbert09
Gilbert09 deleted the posthog/ask-timeout-per-question branch October 2, 2026 12:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stamphog Auto-merge stamp for automated PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants