[DO NOT MERGE] Cancellation replay A/B for #335076 - #335245
[DO NOT MERGE] Cancellation replay A/B for #335076#335245Sandeep Somavarapu (sandy081) wants to merge 9 commits into
Conversation
Honor user-requested worktree overrides while preserving isolation inheritance when omitted. Reject overrides for current-session chats and resolve known worktree folders to their project root when worktree is false. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Resolve only exact linked-worktree roots reported by Git to the primary checkout. Preserve nested and additional workspace folders instead of inferring worktree identity from session project metadata. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ption-to-create-session-tool-and
Use drive-qualified paths for Windows filesystem test inputs and regenerate Copilot prompt baselines for the explicit worktree tool option. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ption-to-create-session-tool-and Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ption-to-create-session-tool-and
…ption-to-create-session-tool-and
…ption-to-create-session-tool-and Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Temporary macOS A/B probe for #335076; do not merge. Alternate fixed revisions on one runner and retain host, runtime and protocol diagnostics before teardown. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The generated environment metadata is invalid JSON, preventing reliable automated analysis of the diagnostic evidence.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review tier: Balanced
Findings: 1
New issues introduced by this change (1)
| Severity | Finding |
|---|---|
 |
.github/cancellation-ab.sh — Appending sw_vers's plain-text output makes environment.json invalid JSON, so consumers cannot… |
What changed in this PR
Temporary diagnostic PR comparing paused-turn cancellation behavior between fixed base and #335076 feature revisions.
Changes:
- Adds explicit
create_session.worktreebehavior, validation, documentation, and tests. - Captures teardown notifications and runtime/host logs.
- Replaces normal PR CI with a macOS base/feature A/B experiment.
| File | Description |
|---|---|
.github/cancellation-ab.sh |
Runs alternating revisions and records evidence. |
.github/workflows/pr.yml |
Defines the temporary diagnostic workflow. |
src/vs/platform/agentHost/AGENTS.md |
Documents worktree override behavior. |
src/vs/platform/agentHost/node/agentService.ts |
Exposes Git worktree-root lookup. |
src/vs/platform/agentHost/node/agentServiceFoundation.ts |
Forwards worktree-root lookup. |
src/vs/platform/agentHost/node/shared/sessionServerTools.ts |
Implements the worktree option. |
src/vs/platform/agentHost/test/node/sessionServerTools.test.ts |
Adds worktree regression coverage. |
src/vs/platform/agentHost/test/node/e2e/harness/agentHostE2ETestHarness.ts |
Captures pre-disposal diagnostics. |
…gpt-5.prompt.md |
Updates the tool-schema snapshot. |
…gpt-5-mini.prompt.md |
Updates the tool-schema snapshot. |
…gpt-5-codex.prompt.md |
Updates the tool-schema snapshot. |
…gpt-5_6-terra.prompt.md |
Updates the tool-schema snapshot. |
…gpt-5_6-sol.prompt.md |
Updates the tool-schema snapshot. |
…gpt-5_6-luna.prompt.md |
Updates the tool-schema snapshot. |
…gpt-5_1.prompt.md |
Updates the tool-schema snapshot. |
…gpt-5_1-codex.prompt.md |
Updates the tool-schema snapshot. |
…gpt-5_1-codex-mini.prompt.md |
Updates the tool-schema snapshot. |
…gemini-2_0-flash.prompt.md |
Updates the tool-schema snapshot. |
…claude-sonnet-5.prompt.md |
Updates the tool-schema snapshot. |
…claude-sonnet-4_6.prompt.md |
Updates the tool-schema snapshot. |
…claude-sonnet-4_5.prompt.md |
Updates the tool-schema snapshot. |
…claude-opus-5.prompt.md |
Updates the tool-schema snapshot. |
…claude-opus-4_8.prompt.md |
Updates the tool-schema snapshot. |
…claude-opus-4_7.prompt.md |
Updates the tool-schema snapshot. |
…claude-opus-4_6.prompt.md |
Updates the tool-schema snapshot. |
…claude-opus-4_5.prompt.md |
Updates the tool-schema snapshot. |
…claude-haiku-4_5.prompt.md |
Updates the tool-schema snapshot. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| mkdir -p "$evidence" | ||
| printf 'phase\titeration\tcommit\texit_code\n' > "$evidence/results.tsv" | ||
| node -p 'JSON.stringify({node:process.version,platform:process.platform,arch:process.arch,copilot:require("./node_modules/@github/copilot/package.json").version})' > "$evidence/environment.json" | ||
| sw_vers >> "$evidence/environment.json" |
Controlled comparison resultRun: https://github.com/microsoft/vscode/actions/runs/34348262142 Same macos-26-xlarge runner, identical locked dependencies, identical teardown-only diagnostic instrumentation, alternating base/feature/feature/base order. Each repetition ran both cancellation cases.
Baseline failure: input cancellation. Feature failures: input cancellation once and file-approval cancellation twice. All failures timed out after the replacement turn started. The baseline reproduces the same observable stall without the worktree changes. Example baseline ordering:
All three feature failures have the same suppressed-idle signature. Successful examples deliver the aborted idle before sending the replacement, then complete normally. Conclusion: the cancellation stall exists on the base without this PR. Six repetitions per variant are not enough to determine whether the PR changes its frequency. The shared failure signature points to an abort/replacement queueing race; the exact defective host/runtime handoff is not yet proven. Full test logs, protocol snapshots, host logs and runtime logs are attached as |
|
Closing this temporary, non-mergeable diagnostic PR now that the A/B evidence has been collected. The feature PR #335076 is unchanged. |
Temporary diagnostic PR only. Do not merge or enable auto-merge.
Compare fixed base e341a3c against feature head 6acb3aa on the same macos-26-xlarge runner. Dependencies are identical and locked. Execute base/feature/feature/base phases, each with three repetitions of both paused-turn cancellation tests (six repetitions per variant).
The same teardown-only instrumentation is applied to each revision. It archives full host/runtime logs and protocol notifications before session disposal. Results and metadata are uploaded as cancellation-ab-evidence. No assertions are relaxed and no test is skipped.
This experiment is separate from #335076 and will be closed after investigation.