Skip to content

[WIP] Add canvases to saved chats in the Agents window - #335951

Draft
Ulugbek Abdullaev (ulugbekna) wants to merge 24 commits into
mainfrom
ulugbekna/canvases-architecture
Draft

Ulugbek Abdullaev (ulugbekna) wants to merge 24 commits into
mainfrom
ulugbekna/canvases-architecture

Conversation

@ulugbekna

@ulugbekna Ulugbek Abdullaev (ulugbekna) commented Sep 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds a default-off, source-built local desktop preview of runtime-owned canvases in the Agents window. Existing Copilot canvas extensions keep their original source directories, runtime, HTTP/SSE interaction and provider-owned data; VS Code supplies exact-chat ownership and isolated native editor presentation.

The first release is intentionally limited to existing, persisted chats. The user sends a normal first request before canvases become available. Canvas-first/zero-turn chat creation is deferred.

This is the original-directory compatibility implementation, separate from #335902. It does not modify or supersede that parallel implementation's PRs.

First-release scope

  • Untitled sessions and untitled peer chats do not expose a canvas facet or command target.
  • The Agent Host rejects initialize, open, action and provider-restart effects for an unused draft.
  • Initial SDK session creation does not request extensions. After the first request makes the chat durable, explicit canvas initialization starts extensions on the resident session. A cold resume may request them during resume.
  • The integration no longer calls session.retain, stores retained execution intent, or promotes a draft from canvas membership.
  • Canvas membership recorded in an already-persisted chat can still restore after restart. Restoring an editor remains metadata-only and obtains a fresh source without replaying the open or an action.

Canvas-first creation, durable zero-turn owners and their automatic recovery can be designed separately using host-owned persistence. They are not silently preserved through another API in this PR.

Included

  • Canonical AHP canvas state, discovery, lifecycle and action routing, with separately negotiated explicit provider initialization.
  • Host-approved extension launch using exact sessionId and the runtime's original defaultLaunch, strict version-1 negotiation, cancellation/disconnect fencing and no fallback after denial.
  • Deterministic extension/readiness handling and resident-session contribution attachment for persisted chats.
  • Workspace Trust as a hard prerequisite for project-extension execution, with separate approvals preserved for user/plugin/session sources, credentials and recovery.
  • Provider-neutral Sessions projection and native canvas editors. Tab close, hiding and owner switching detach presentation; logical close and provider restart remain separate explicit operations.
  • User-only native isolation, separate browser permissions and file trust, cancellation-safe choosers, accessibility/help and theme integration. Canvas pages do not become ordinary browser/model automation targets.
  • A guarded backport over SDK 1.0.15-preview.2 / CLI 1.0.89-1. The current-main package is patched as a complete image; the obsolete public-1.0.13/B3 transition route has been removed.

The presentation setting sessions.experimental.canvases.enabled defaults to false. Actual canvas support additionally requires a non-built development environment and an explicitly selected compatible runtime through VSCODE_AGENT_HOST_CANVAS_RUNTIME_PATH.

Persisted-chat startup flow

For a new chat, the SDK session starts without extensions. The normal first request creates durable user/assistant history. When the user later selects Initialize Canvas Providers or opens a canvas, VS Code:

  1. rechecks the owning session and Workspace Trust;
  2. attaches the existing runtime session to the canvas launch context;
  3. explicitly reloads extensions;
  4. applies source approval and the launch-v1 callback;
  5. waits for complete extension and canvas registry readiness;
  6. exposes/open the requested canvas.

For a cold existing chat, resume can request extensions during resume. Neither path fabricates a chat turn or calls sessions.save during admission.

Workspace Trust execution boundary

Workspace Trust is owned by VS Code, not by the Copilot runtime:

  • New folder-backed sessions and first/follow-up sends already use the normal session trust gates.
  • Canvas initialize, open, action and provider restart re-enter ISessionsService.canExecuteSession, including when trust was revoked from an already-active session.
  • Every execution workspace folder is checked, including VS Code-created worktrees. Missing workspace metadata blocks execution until it hydrates.
  • Catalog refresh and source resolution remain non-starting. Logical close remains available for cleanup after trust is removed.
  • Immediately before launch, the Copilot adapter also rechecks the actual working directory plus the original and canonical extension entrypoint, preventing a changed symlink or late revocation from authorizing execution.

Workspace Trust is not a Node sandbox, content-bound package approval or replacement for extension/credential/browser/file/recovery permissions.

Dependency alignment

  • Runtime stack 1/4: github/copilot-agent-runtime#22702, head 31879439 — generic numeric schema-constant generation. Its repository CI is green.
  • Runtime stack 2/4: github/copilot-agent-runtime#22714, head b6467d5e — launch-v1 contract, exact owner/default recipe, fail-closed resolver, exact upstream native resume lock-order fix 8f00ab12, and deterministic CI fixtures. This is the direct SDK wire dependency. Public runtimes 1.0.89-0 and 1.0.89-1 predate the native fix.
  • Runtime stack 3/4: github/copilot-agent-runtime#22718, head b5526e14 — resident attachment, deterministic readiness, privileged-callback ordering and initial script safety.
  • Runtime stack 4/4: github/copilot-agent-runtime#20368, head e6668e8b — the two-file persisted-chat integration proof plus a test-only background-interrupt routing precondition.
  • Node SDK: [WIP] Node SDK canvas launch admission for persisted chats github/copilot-sdk#2637, head 82916669. All 93 current-head checks pass (78 existing path-selected skips, zero failures/pending). CI fixes are limited to test/fixture ownership and strict replay history; production SDK, generated schemas, pins and carrier bytes are unchanged.
  • AHP: [WIP] Add LC-17 canvas protocol and presentation clarifications agent-host-protocol#447, canonical revision cd05c63c. No persisted-chat scope change was required.
  • VS Code carrier: SDK 1.0.15-preview.2 / CLI 1.0.89-1, patch SHA-256 987c431f47a68ba6c10d20223405223ac745e4bcad00df62d0c450fdab16a6f9. Final integrated qualification requires a newly published runtime that contains 8f00ab12; the later publication timestamp of 1.0.89-1 does not imply that it contains the fix.

These are candidate contracts, not published SDK/runtime releases. Aligned review and releases remain required before replacing the private carrier or enabling the feature by default.

Validation

The current VS Code head adefbd95 is reconciled with current main and passed:

  • all 35 repository-native hosted checks, including compile/hygiene, all platform Electron/browser/remote lanes, Copilot tests, screenshots, Monaco, CodeQL, metadata, dependency and license checks;
  • client and build-tool typechecking plus valid-layers;
  • hygiene, changed-file ESLint/stylelint and normal signed commit hooks;
  • focused unit selections with 494, 1,051, 1,211, 562, 828 and 1,096 passing tests;
  • provider integration selections with 7, 11 and 5 passing tests;
  • 30/30 carrier tests plus root and remote installation verification.
  • the runtime MCP E2E replay 4/4 twice after adding deterministic warm-up turns and re-recording four Copilot fixtures.

Runtime and SDK PRs have their own focused generation/build/test evidence. SDK's previously failing freshness, macOS Node and macOS Rust jobs are repaired; its full current-head matrix is green.

Runtime CI is not uniformly green and is not represented as such. Layer 1 and the top are terminal green. Layer 2 has zero native/test failures; its only two reds are a GitHub HTTP-500 review-dispatcher reconciliation and its aggregate. Layer 3 has two unresolved root failures plus aggregates: a pending-work handoff result and one extension-environment permission prompt attempt. The one authorized pending-work proof compiled and ran but was inconclusive because its warm assertion misread sessionWasActive and its cold fixture was not persisted. The environment-access failure cannot be assigned because the CI artifact omits the terminal resume/peer-exit/extension-launch witness. No failure is waived.

No Code OSS/Electron/native application qualification was rerun for this scope change.

Historical qualification boundary

Earlier native qualification belongs to older heads that supported canvas-first retention and used older runtime/SDK images. It remains useful evidence for native presentation, isolation, reload, close and restoration mechanics, but it does not qualify this persisted-chat head or its new B4/runtime pair.

Draft gates and limits

  • All four runtime stack PRs, SDK and AHP review/release alignment are still required.
  • The feature remains default-off and development-runtime-only.
  • VS Code, SDK, runtime layer 1 and the runtime top are green. Runtime layer 2 retains only an infrastructure review-dispatcher failure; runtime layer 3 remains red and needs separately authorized corrected diagnostic proofs before further evidence-backed source work. No baseline or failing check is waived.
  • A newly published runtime containing 8f00ab12 is required before final consumer qualification; neither public 1.0.89-0 nor 1.0.89-1 contains it.
  • Application qualification against the aligned persisted-chat runtime/SDK is still required.
  • Other native platforms, all source scopes, physical assistive technology, every model/resume combination, remote/Web execution and native Office built-ins are not qualified by this preview.

Trying the local preview

Use an isolated Code OSS development profile with the compatible runtime selected explicitly and the canvas presentation setting enabled.

  1. Create a normal chat and send its first message.
  2. Trust the project through VS Code's normal Workspace Trust flow.
  3. Open Canvases… and initialize providers or select a live canvas type.
  4. Complete any separate approvals for non-project sources, credentials, browser permissions or file trust.

An ordinary published CLI is not a substitute for the required runtime candidate. In particular, public 1.0.89-1 predates the native resume lock-order fix required by this stack.

See src/vs/sessions/contrib/canvases/README.md for the ownership/lifecycle contract and build/npm/copilot-sdk-canvas.md for the current carrier boundary.

Integrate runtime-owned canvases through AHP with isolated native presentation, exact-owner approval, no-turn retention, and explicit recovery. Keep the source-built local preview disabled by default with separate SDK/runtime release and enablement gates.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings September 12, 2026 10:29

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

External views can survive renderer restoration and canvas icon updates are not projected into canonical state.

Get a fresh assessment by requesting another Copilot review.

Review tier: Balanced
Findings: 2 Medium severity

Open findings (2)
What changed in this PR

Adds an opt-in native canvas preview to the Agents window, integrating provider-owned canvases across Agent Host protocol state, Sessions UI, and isolated Integrated Browser presentation.

Changes:

  • Adds canvas discovery, lifecycle, persistence, action routing, and SDK compatibility support.
  • Adds native canvas editors with accessibility, theming, permissions, and file-trust handling.
  • Adds targeted unit, integration, and smoke coverage.
File Description
build/​npm/​copilotSdkCanvasPatch.ts Validates and applies the SDK backport.
src/​vs/​platform/​agentHost/​common/​agentHostCanvasValidation.ts Validates canvas protocol data.
src/​vs/​platform/​agentHost/​common/​agentHostCanvases.ts Defines canvas service contracts.
src/​vs/​platform/​agentHost/​common/​agentHostExtensionProtocol.ts Adds explicit initialization negotiation.
src/​vs/​platform/​agentHost/​common/​ahpJsonlLogger.ts Redacts transient canvas sources.
src/​vs/​platform/​agentHost/​common/​state/​protocol/​channels-canvas/​* Adds generated canvas protocol state and operations.
src/​vs/​platform/​agentHost/​node/​agentHostCanvasOperationLedger.ts Handles bounded, idempotent operations.
src/​vs/​platform/​agentHost/​node/​agentHostCanvasesService.ts Implements authoritative canvas lifecycle.
src/​vs/​platform/​agentHost/​node/​copilot/​copilotCanvases.ts Adapts Copilot SDK canvases.
src/​vs/​platform/​agentHost/​node/​protocolServerHandler.ts Routes canvas protocol requests.
src/​vs/​sessions/​contrib/​canvases/​README.md Documents ownership and isolation.
src/​vs/​sessions/​contrib/​canvases/​common/​sessionCanvasPresentation.ts Coordinates state and native presentation.
src/​vs/​sessions/​contrib/​canvases/​electron-browser/​sessionCanvasActions.ts Adds canvas commands and pickers.
src/​vs/​sessions/​contrib/​canvases/​electron-browser/​sessionCanvasEditor.ts Implements the canvas editor and accessibility.
src/​vs/​sessions/​contrib/​canvases/​electron-browser/​sessionCanvasService.ts Manages canvas inputs and leases.
src/​vs/​sessions/​contrib/​canvases/​electron-browser/​sessionCanvases.contribution.ts Registers settings, editors, and actions.
src/​vs/​sessions/​contrib/​providers/​agentHost/​browser/​agentHostSessionCanvases.ts Projects Agent Host canvases into Sessions.
src/​vs/​workbench/​contrib/​browserView/​electron-browser/​browserCanvasTheme.ts Maps workbench themes into canvas guests.
src/​vs/​workbench/​contrib/​browserView/​electron-browser/​browserFileTrustWidget.ts Adds trusted-file recovery UI.
src/​vs/​workbench/​contrib/​browserView/​electron-browser/​browserViewWorkbenchService.ts Creates isolated external browser views.
src/​vs/​workbench/​contrib/​browserView/​electron-browser/​overlayManager.ts Recognizes accessible-view overlays.
src/​vs/​workbench/​contrib/​browserView/​electron-browser/​webContentsViewHost.ts Shares native view hosting behavior.
src/​vs/​platform/​browserView/​electron-main/​browserSessionFileAccess.ts Enforces trusted file roots.
src/​vs/​platform/​browserView/​electron-main/​browserViewMainService.ts Enforces native canvas isolation.
src/​vs/​workbench/​services/​agentHost/​browser/​editorRemoteAgentHostServiceClient.ts Forwards canvas operations remotely.
src/​vs/​workbench/​contrib/​browserView/​test/​** Covers isolation, themes, overlays, and trust.
src/​vs/​sessions/​contrib/​canvases/​test/​** Covers canvas ownership and lifecycle.
src/​vs/​platform/​agentHost/​test/​** Covers protocol and provider behavior.
test/​smoke/​src/​areas/​browserView/​browserView.test.ts Exercises accessibility help integration.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/vs/platform/agentHost/node/agentHostCanvasesService.ts
Preserve canvas initialization, retention, transport ownership and early SDK events alongside upstream artifact removal, workspace trust, sandbox and steering updates.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Apply canonical icon changes without replaying provider effects and release orphan native presentations before renderer restoration. Bind CI dependency caches to SDK postinstall inputs, preserve native-only capability coverage, and validate canonical session disposal requests.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Match the existing mock chat-surface override pattern so the canvas creation fixtures pass the define-class-fields check without changing their initialization or cancellation assertions.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@github-actions

github-actions Bot commented Sep 12, 2026 •

Copy link
Copy Markdown
Contributor

Base: 35f0b23b Current: d2375241

No screenshot changes.

Apply the byte-verified package delta with Git line-ending conversion disabled for that subprocess. Cover autocrlf true, input, and false with a CRLF preference without changing package hashes or user Git configuration.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Read already-hydrated exact-owner membership when resolving a logical input. Preserve later observable title updates and cover both hydration orderings without source pulls, provider effects, or native allocation.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Authorize project extension startup using the existing synchronized Workspace Trust for the actual session directory and original/resolved entrypoint. Preserve explicit approvals for other sources, credentials and recovery, and retain sessions before launch. Recheck trust and source identity after asynchronous retention. Share the existing Codex trust matcher without changing its behavior.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Reject project extension launches before source approval or retention unless the owning workspace and original/resolved source paths are trusted. Keep explicit source approval for user, plugin and session extensions. Preserve late trust/path checks, cancellation, credentials, recovery and persistence semantics, and cover the former manual-approval bypass with regressions.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Cover missing launch context, canonical retention error identity, explicit retry, and cancellation while retention is pending. Document that the client uses unchanged public JSON-RPC methods rather than runtime-internal N-API exports; no production API or SDK payload migration is required for the runtime trim.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Reuse the Sessions workspace execution admission for canvas initialize, open, action, and provider restart, including after trust revocation. Keep catalog/source reads and close available without starting a provider.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Defer extension startup and canvas operations until the conversation has a durable first turn. Remove zero-turn retention and draft promotion, adopt the retention-free B4 SDK carrier, and preserve launch admission, readiness, cancellation, and restoration for saved chats.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@ulugbekna Ulugbek Abdullaev (ulugbekna) changed the title [WIP] Add compatible canvases to the Agents window [WIP] Add canvases to saved chats in the Agents window Sep 23, 2026
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants