Skip to content

agent host: refactor: adopt experimental AHP canvas channels - #337780

Merged
Joaquín Ruales (jruales) merged 2 commits into
mainfrom
ulugbekna/minimal-canvas-support
Oct 2, 2026
Merged

Joaquín Ruales (jruales) merged 2 commits into
mainfrom
ulugbekna/minimal-canvas-support

Conversation

@ulugbekna

@ulugbekna Ulugbek Abdullaev (ulugbekna) commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Migrates the local Copilot canvas support already on main from its temporary VS Code-specific transport to the experimental canvas channel merged in microsoft/agent-host-protocol#472.

  • keeps stable chat state limited to minimal canvas resource references
  • publishes full live presentation state, including the optional HTTP(S) source, on independently subscribable ahp-canvas: channels
  • removes the private vscode/canvases/v1 notification, source-resolution request, initialize capability, and parallel snapshot model
  • preserves the existing chat.agentHost.canvases.enabled runtime/presentation gate and durable local-Copilot-only product policy
  • keeps shared Sessions state provider-neutral

The synced AHP source is pinned to the merged PR commit 6eec1b91.

Architecture

flowchart LR
    Runtime["Copilot runtime canvas events"]
    Projection["CopilotAgentSession projection"]
    Chat["ChatState.canvases references"]
    Canvas["Experimental CanvasState channel"]
    Client["AHP resource subscription"]
    Sessions["Provider-neutral ISessionCanvas"]
    Editor["Existing Sessions canvas editor"]

    Runtime --> Projection
    Projection -->|membership only| Chat
    Projection -->|metadata + optional URL| Canvas
    Chat --> Client
    Canvas --> Client
    Client --> Sessions
    Sessions --> Editor
Loading

Host projection and lifecycle

A true canvas-open event creates a new lifetime-scoped ahp-canvas: resource. Metadata or source changes retain that resource and emit canvas/stateChanged; provider unavailability clears the source through full-state replacement. Closing the canvas removes chat membership and prunes the channel. Session/chat teardown also clears owned channels.

Canvas subscriptions are live-only. They return existing host state, pin the owning session through the normal subscription residency path, and never restore a provider or replay an open/action request. The existing eight-canvas bound and restart non-restoration behavior remain unchanged.

Client and Sessions adaptation

The protocol client uses the standard subscription manager and canonical canvas reducer, preserving exact host-advertised resource URIs rather than reconstructing remote identities.

The Agent Host Sessions provider follows each advertised canvas reference with a cached resource-scoped subscription and projects immutable provider-neutral state. Unknown membership is distinct from an authoritative empty collection, and pending/error channel state retains membership without a live source.

Presentation support remains a local provider decision: durable local Copilot sessions expose supportsCanvases; remote, Claude, and Codex sessions do not. The existing setting and AI entitlement still control actual runtime enablement and editor presentation.

Privacy and stale-load protection

Canvas URLs are redacted from AHP action, snapshot, initialize/reconnect, and replay diagnostics without mutating live state. Sources remain HTTP(S)-only. The editor keeps local input, source, owner, and load-sequence checks and disposes superseded BrowserViews. Metadata/source changes do not reopen a user-dismissed canvas.

Removed temporary surfaces

  • AgentCanvasAvailability, IAgentCanvas, and IAgentCanvasSnapshot
  • IAgentHostCanvases and IAgentService.onDidChangeCanvases
  • vscode/canvases/v1/changed and vscode/canvases/v1/resolveSource
  • the VS Code-specific canvas initialize capability and metadata helpers
  • revision-fenced source resolution

A separate AHP agent capability is intentionally absent: clients cannot initiate canvas operations in this scope, and server-published chat references are sufficient discovery. A capability can be introduced later alongside client-originated open/action operations if needed.

Validation

  • npm run typecheck-client — passed
  • npm run transpile-client — passed
  • focused Agent Host, Sessions provider, canvas service, and layout unit tests — 30 passed
  • bundled Copilot runtime/provider integration — 1 passed
  • strict Copilot prompt replay — 24 passed
  • changed-file ESLint — passed for 35 files
  • npm run valid-layers-check — passed
  • pre-commit hygiene — passed
  • git diff --check — passed

Explicitly deferred

  • canvas-first or zero-turn chats
  • manual canvas picker
  • UI-originated canvas open/action requests
  • restoration across VS Code/runtime restart
  • remote Agent Host, Claude, or Codex presentation
  • built-in Office, browser, or terminal canvas providers

Copilot AI balanced review requested due to automatic review settings September 24, 2026 18:53

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

External views are still registered normally, and unresolved lifecycle, redaction, readiness, and accessibility-bounding issues remain.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 2 High severity · 4 Medium severity

Open (6)
What changed in this PR

Adds Copilot canvas support across Agent Host protocol, Copilot runtime integration, Sessions presentation, embedded BrowserView rendering, and accessibility.

Changes:

  • Projects revision-fenced canvas state and source resolution through AHP.
  • Adds a Sessions-owned canvas editor using ephemeral BrowserViews.
  • Adds accessibility snapshots, Help, Accessible View, and focused tests.
File Description
src/​vs/​workbench/​services/​agentHost/​browser/​editorRemoteAgentHostServiceClient.ts Forwards canvas-source requests.
src/​vs/​workbench/​contrib/​browserView/​test/​electron-browser/​browserEditorInput.test.ts Updates BrowserView test stub.
src/​vs/​workbench/​contrib/​browserView/​electron-browser/​webContentsViewHost.ts Extracts native view hosting behavior.
src/​vs/​workbench/​contrib/​browserView/​electron-browser/​features/​webContentsViewRendererFeature.ts Adopts the extracted host.
src/​vs/​workbench/​contrib/​browserView/​electron-browser/​browserViewWorkbenchService.ts Creates external ephemeral views.
src/​vs/​workbench/​contrib/​browserView/​common/​browserView.ts Extends BrowserView contracts.
src/​vs/​workbench/​contrib/​browserView/​browser/​browserView.contribution.ts Adds unsupported web stub.
src/​vs/​workbench/​contrib/​accessibility/​browser/​accessibilityConfiguration.ts Registers canvas verbosity.
src/​vs/​sessions/​SESSIONS.md Documents the canvas contract.
src/​vs/​sessions/​sessions.desktop.main.ts Loads the canvas contribution.
src/​vs/​sessions/​services/​sessions/​common/​session.ts Adds provider-neutral canvas state.
src/​vs/​sessions/​contrib/​providers/​agentHost/​test/​browser/​localAgentHostSessionsProvider.test.ts Tests capability projection.
src/​vs/​sessions/​contrib/​providers/​agentHost/​browser/​baseAgentHostSessionsProvider.ts Projects AHP canvases into Sessions.
src/​vs/​sessions/​contrib/​providers/​agentHost/​AGENT_HOST_SESSIONS_PROVIDER.md Documents provider behavior.
src/​vs/​sessions/​contrib/​canvases/​test/​common/​sessionCanvas.test.ts Tests input identity and restoration.
src/​vs/​sessions/​contrib/​canvases/​electron-browser/​sessionCanvasService.ts Manages canvas presentation lifecycle.
src/​vs/​sessions/​contrib/​canvases/​electron-browser/​sessionCanvases.contribution.ts Registers services, editor, and accessibility.
src/​vs/​sessions/​contrib/​canvases/​electron-browser/​sessionCanvasEditor.ts Implements canvas rendering and accessibility.
src/​vs/​sessions/​contrib/​canvases/​electron-browser/​media/​sessionCanvas.css Styles the canvas editor.
src/​vs/​sessions/​contrib/​canvases/​common/​sessionCanvas.ts Defines canvas editor input and service.
src/​vs/​platform/​browserView/​test/​electron-main/​browserViewAccessibility.test.ts Tests AX-tree formatting.
src/​vs/​platform/​browserView/​electron-main/​browserViewMainService.ts Exposes accessibility snapshots.
src/​vs/​platform/​browserView/​electron-main/​browserViewDebugger.ts Retrieves Chromium AX trees.
src/​vs/​platform/​browserView/​electron-main/​browserViewAccessibility.ts Formats bounded semantic output.
src/​vs/​platform/​browserView/​common/​browserView.ts Adds accessibility snapshot API.
src/​vs/​platform/​agentHost/​test/​node/​providerIntegration/​copilotMockLlm.integrationTest.ts Exercises real SDK canvas projection.
src/​vs/​platform/​agentHost/​test/​node/​protocolServerHandler.test.ts Updates AgentService mock.
src/​vs/​platform/​agentHost/​test/​node/​copilotSessionLauncher.test.ts Tests canvas launch configuration.
src/​vs/​platform/​agentHost/​test/​node/​copilotAgentSession.test.ts Tests live projection and revision fencing.
src/​vs/​platform/​agentHost/​test/​node/​copilotAgent.test.ts Tests capability and launch admission.
src/​vs/​platform/​agentHost/​test/​electron-browser/​agentHostProtocolClient.test.ts Tests source-resolution RPC.
src/​vs/​platform/​agentHost/​test/​common/​ahpJsonlLogger.test.ts Tests URL redaction.
src/​vs/​platform/​agentHost/​node/​protocolServerHandler.ts Routes source-resolution requests.
src/​vs/​platform/​agentHost/​node/​copilot/​copilotSessionWrapper.ts Exposes SDK canvas events.
src/​vs/​platform/​agentHost/​node/​copilot/​copilotSessionLauncher.ts Enables released canvas assets.
src/​vs/​platform/​agentHost/​node/​copilot/​copilotRuntimePaths.ts Resolves runtime and extension assets.
src/​vs/​platform/​agentHost/​node/​copilot/​copilotAgentSession.ts Projects and resolves live canvases.
src/​vs/​platform/​agentHost/​node/​copilot/​copilotAgent.ts Adds launch admission and capability.
src/​vs/​platform/​agentHost/​node/​agentService.ts Routes canvas requests to providers.
src/​vs/​platform/​agentHost/​electron-browser/​localAgentHostService.ts Forwards local canvas requests.
src/​vs/​platform/​agentHost/​common/​state/​sessionState.ts Re-exports canvas protocol types.
src/​vs/​platform/​agentHost/​common/​state/​protocol/​version/​registry.ts Advances protocol version metadata.
src/​vs/​platform/​agentHost/​common/​state/​protocol/​common/​messages.ts Registers the canvas command.
src/​vs/​platform/​agentHost/​common/​state/​protocol/​common/​actions.ts Registers the canvas action.
src/​vs/​platform/​agentHost/​common/​state/​protocol/​channels-root/​state.ts Adds canvas capability metadata.
src/​vs/​platform/​agentHost/​common/​state/​protocol/​channels-chat/​state.ts Defines synchronized canvas state.
src/​vs/​platform/​agentHost/​common/​state/​protocol/​channels-chat/​reducer.ts Reduces canvas state changes.
src/​vs/​platform/​agentHost/​common/​state/​protocol/​channels-chat/​commands.ts Defines source-resolution RPC.
src/​vs/​platform/​agentHost/​common/​state/​protocol/​channels-chat/​actions.ts Defines canvas replacement actions.
src/​vs/​platform/​agentHost/​common/​state/​protocol/​action-origin.generated.ts Updates generated action ownership.
src/​vs/​platform/​agentHost/​common/​state/​protocol/​.ahp-version Records synchronized AHP revision.
src/​vs/​platform/​agentHost/​common/​ahpJsonlLogger.ts Redacts canvas source URLs.
src/​vs/​platform/​agentHost/​common/​agentService.ts Extends service interfaces.
src/​vs/​platform/​agentHost/​common/​agent.ts Extends provider chat APIs.
src/​vs/​platform/​agentHost/​browser/​nullAgentHostService.ts Adds unsupported fallback.
src/​vs/​platform/​agentHost/​browser/​agentHostProtocolClient.ts Implements canvas RPC client.
src/​vs/​platform/​accessibility/​browser/​accessibleView.ts Adds canvas provider identity.
build/​lib/​i18n.resources.json Registers canvas localization resources.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/vs/platform/agentHost/common/ahpJsonlLogger.ts Outdated
Comment thread src/vs/platform/agentHost/node/copilot/copilotAgent.ts Outdated
Comment thread src/vs/platform/agentHost/node/copilot/copilotAgentSession.ts Outdated
Comment thread src/vs/platform/browserView/electron-main/browserViewDebugger.ts Outdated
Comment thread src/vs/sessions/contrib/canvases/electron-browser/sessionCanvasService.ts Outdated
@github-actions

github-actions Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Screenshot Changes

Base: a76ce4f7 Current: 045e0177

Changed (2)

sessions/accountMenu/WeeklyAndFiveHourLimits/Light
Before After
before after
chat/aiCustomizations/aiCustomizationManagementEditor/DiscoverPluginsLoadingMore/Light
Before After
before after

2 insignificant change(s) omitted (≤20 px, Δ≤2). See CI logs for details.

@ulugbekna

Copy link
Copy Markdown
Contributor Author

Code OSS Chess canvas verification recording — extension registration, native canvas tool use, and the rendered Sessions canvas tab.

chess-canvas-code-oss-verification.webm

@ulugbekna

Copy link
Copy Markdown
Contributor Author

Strict no-click automatic canvas reveal verification

After the prompt is submitted, the model calls open_canvas; Chess — e2e becomes the selected visible editor and renders the board without any tab, editor, focus, or browser click.

chess-canvas-strict-auto-reveal.webm

@ulugbekna

Copy link
Copy Markdown
Contributor Author

Native Chess canvas recording

The earlier workbench recording captures only the Electron renderer and omits the OS-composited WebContentsView. This recording targets the native canvas surface directly, so the board UI is visible.

chess-canvas-native-view.webm

@ulugbekna

Ulugbek Abdullaev (ulugbekna) commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor Author

Commit-by-commit reviewer guide

The history remains four signed semantic commits. Follow-up fixes and canvas-originated chat visibility are folded into the feature commits that own their behavior.

Context

The released Copilot runtime already supports extension-provided canvases and the native model tools list_canvas_capabilities, open_canvas, and invoke_canvas_action. This PR adds the missing VS Code path:

  1. load the extension during normal trusted create/resume;
  2. project a model-opened instance as safe chat-owned metadata;
  3. resolve its private HTTP(S) source on demand;
  4. show it automatically beside the owning chat in a private native page.
  5. project model work started by the canvas extension as a normal visible turn in that chat.

1. bf86d9ae016 — reusable native page host

Subject: browser: refactor: extract reusable web contents host

This is a behavior-preserving BrowserView refactor. It extracts the native overlay, layout, focus, screenshot, keybinding, and disposal logic into WebContentsViewHost.

Why needed: the canvas editor needs the proven native WebContentsView host without duplicating Electron lifecycle code or opening the normal browser editor/navigation UI.

Review focus: confirm that regular integrated-browser behavior and disposable ownership remain unchanged.

2. 26e93915e68 — trusted runtime lifecycle, AHP projection, and visible provider turns

Subject: agent host: feat: launch and project Copilot canvases

This commit:

  • enables released SDK extensions/canvas rendering during normal durable create/resume;
  • resolves the paired runtime/SDK/bootstrap assets;
  • admits canonical project extension entrypoints only beneath live session roots;
  • rejects symlink escapes and session-scoped launches;
  • supplies stable VSCODE_CANVAS_DATA_DIR values;
  • waits for extension readiness before the relevant model turn;
  • syncs chat: feat: add canvas projection agent-host-protocol#472;
  • projects revisioned metadata while retaining source URLs privately;
  • resolves only the exact live chat/instance/revision and only HTTP(S);
  • redacts source responses from logs;
  • suppresses duplicates, caps each chat at eight instances, and drops restart replay;
  • projects correlated canvas-extension session.send() requests as provider-started turns in the owning chat;
  • routes their tools, approvals, usage, responses, errors, cancellation, and completion through normal AHP state;
  • excludes extension-authored prompts from human-message telemetry;
  • includes the tokenless bundled-runtime proof and deterministic prompt snapshots.

The former protocol/prompt CI follow-up is folded here. It restores MCP blocking state when the synced AHP optimistic background request is rejected, and it prevents ambient Copilot app feature flags from contaminating prompt baselines.

Why needed: this is the complete host-side correctness and privacy boundary. Without it, the model can open a runtime canvas but VS Code has no safe live state to present; likewise, a canvas can start real model work while the owning chat remains silent.

Expected interaction semantics:

  • A local canvas-only mutation, such as moving a chess piece, does not create a chat turn by itself.
  • If that action causes the extension to call session.send()—manual Agent move or automatic replies—the generated request, tools, approvals, usage, response, errors, cancellation, and completion appear in the owning chat.
  • A visible provider-started turn requires a root SDK message with both public messageId and turnId. Subagent, injected-context, steering-owned, and uncorrelated events do not create turns.

Review focus: extension path admission, readiness races, revision fencing, URL privacy/redaction, provider-started turn correlation, telemetry attribution, restart semantics, and resource limits.

3. 3829af42c26 — private unlisted BrowserViews

Subject: browser: feat: add private unlisted browser views

This commit adds explicit Listed / Unlisted presentation state and an external-view creation API.

Canvas pages are user-owned, ephemeral, unlisted, and have no agent audience or associated browser resource. They are excluded from browser hydration, discovery, quick open, and tools. Popup creation is denied. The renderer subscribes before initial navigation so it cannot miss the first URL update.

It also adds incremental bounded Chromium AX retrieval: 2,000 nodes and 32 KiB formatted text.

Why needed: embedding the page as an ordinary browser tab would leak ownership into browser discovery, persistence, and automation surfaces.

Review focus: listed/unlisted propagation, discovery isolation, popup policy, navigation ordering, and accessibility bounds.

4. 99270b119c9 — Agents-window presentation

Subject: sessions: feat: present Copilot canvases beside chats

This commit adds provider-neutral ISessionCanvas contracts, local-Agent-Host-only capability adaptation, and the default-off sessions.experimental.canvases.enabled rollout setting.

SessionCanvasService automatically reveals a ready new revision beside its active owning chat. A user-closed revision stays dismissed until the model opens a newer revision. Canvas/provider/session removal and setting disablement clean up presentation state.

SessionCanvasEditor resolves the source on demand, mounts an unlisted external BrowserView, manages focus/layout/visibility, and is intentionally non-serializable. Accessibility Help, Accessible View, and verbosity configuration are included.

The former final-rebase fix is folded here: the adapter preserves the latest shared model-selection and dynamic default-chat behavior from main.

Why needed: commits 2 and 3 make the canvas observable and safe to embed; this commit is the minimal user-visible Agents-window vertical slice.

Review focus: active-chat ownership, automatic reveal, dismissal-by-revision, disposal paths, default-off gating, local-only provider scope, and accessibility.

Suggested review order

  1. Commit 1 for the mechanical extraction.
  2. Commit 3 for the isolated BrowserView privacy primitive.
  3. Commit 2 for runtime trust, AHP ownership, and proof.
  4. Commit 4 for the final Sessions UX.

Validation status

  • released pair: @github/copilot-sdk 1.0.15-preview.3, runtime 1.0.89-3
  • full client typecheck
  • full CopilotAgentSession suite: 712 passing
  • deterministic Copilot prompt replay: 23 passing
  • bundled-runtime integration covering extension-before-turn ordering, model-opened canvas, and extension-started visible chat turn: 1 passing
  • focused Sessions and BrowserView privacy/accessibility suites
  • post-rebase focused unit slice: 2,206 passing, 12 pending
  • GitHub's full matrix is rerunning after the rebased head was force-pushed

Highest-value questions

  1. Is canonical project-extension admission strict enough around path and shutdown races?
  2. Is metadata-only AHP state plus revision-fenced source resolution the correct privacy boundary?
  3. Does the provider-started turn bridge admit only correlated live root requests and preserve normal approvals/cancellation?
  4. Can an unlisted BrowserView enter any normal browser discovery or automation path?
  5. Is automatic reveal scoped tightly enough to the active owning chat and current revision?
  6. Are the eight-canvas and accessibility limits appropriate initial production bounds?

The PR description contains the full architecture, validation matrix, Chess screenshot, and OS-composited verification video.

@ulugbekna

Copy link
Copy Markdown
Contributor Author

Definitive OS-composited verification: Agents window + native Chess canvas

This window-level recording captures both the Electron renderer and the native WebContentsView in the same frame. After Enter, there are no tab, editor, focus, or browser actions: the model calls open_canvas, and the Chess canvas automatically appears beside its owning chat.

agents-window-chess-canvas-auto-reveal.mp4

@ulugbekna
Ulugbek Abdullaev (ulugbekna) force-pushed the ulugbekna/minimal-canvas-support branch 6 times, most recently from ba1f61c to fa17bb7 Compare September 25, 2026 11:07
@ulugbekna
Ulugbek Abdullaev (ulugbekna) force-pushed the ulugbekna/minimal-canvas-support branch 5 times, most recently from 99270b1 to 6a795d7 Compare October 2, 2026 22:14
@ulugbekna Ulugbek Abdullaev (ulugbekna) changed the title sessions: feat: add Copilot canvas support agent host: refactor: adopt experimental AHP canvas channels Oct 2, 2026
@ulugbekna
Ulugbek Abdullaev (ulugbekna) marked this pull request as ready for review October 2, 2026 22:14
@vs-code-engineering

Copy link
Copy Markdown
Contributor

📬 CODENOTIFY

The following users are being notified based on files changed in this PR:

Benjamin Christopher Simmonds (@benibenj)

Matched files:

  • src/vs/sessions/contrib/layout/test/browser/desktopStrategies.test.ts

Sandeep Somavarapu (@sandy081)

Matched files:

  • src/vs/sessions/services/sessions/common/session.ts

Ladislau Szomoru (@lszomoru)

Matched files:

  • src/vs/sessions/services/sessions/common/session.ts

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replace the VS Code-specific canvas snapshot and source-resolution transport with the merged Agent Host Protocol canvas channel.

Chat state advertises canvas resource references while live state is synchronized through experimental canvas channels. Preserve the AHP 0.9 archived catalog field alongside 0.10 status flags, and gate chat read actions to 0.10 for negotiated compatibility.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@ulugbekna
Ulugbek Abdullaev (ulugbekna) force-pushed the ulugbekna/minimal-canvas-support branch from 6a795d7 to a728eb8 Compare October 2, 2026 22:36
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@jruales
Joaquín Ruales (jruales) merged commit 35787c9 into main Oct 2, 2026
55 of 56 checks passed
@jruales
Joaquín Ruales (jruales) deleted the ulugbekna/minimal-canvas-support branch October 2, 2026 23:58
@vs-code-engineering vs-code-engineering Bot added this to the 1.141.0 milestone Oct 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants