agentHost: centralize session and chat catalog metadata - #332410
agentHost: centralize session and chat catalog metadata#332410Sandeep Somavarapu (sandy081) wants to merge 53 commits into
Conversation
Add a backward-compatible sessions_v2 catalog, legacy-first synchronization receipts, reconciliation, shadow validation, central fallback reads, and durable chat metadata while retaining open-only content in per-session databases.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adapt the sessions_v2 catalog and reconciliation work to the latest Agent Host composition, adoption, metadata, and test infrastructure changes.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Make sessions_v2 an independent current registry, import directly from current, legacy, and provider sources, mirror runtime identities for downgrade compatibility, and reconcile cross-version changes with durable exclusions and versioned markers.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Move catalog source resolution and downgrade-compatible peer chat persistence out of AgentService into focused helpers without changing migration or runtime behavior.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Introduces a centralized, verified sessions_v2 catalog for efficient Agent Host session listing, migration, downgrade compatibility, and recovery.
Changes:
- Adds catalog schemas, canonical projections, synchronization receipts, migration, reconciliation, and shadow validation.
- Integrates central listing with session mutations, peer chats, Git state, titles, and legacy adoption.
- Adds extensive persistence, compatibility, recovery, and rollout tests.
Show a summary per file
| File | Description |
|---|---|
src/vs/platform/agentHost/common/agent.ts |
Extends adoption results with recovered list metadata. |
src/vs/platform/agentHost/common/sessionDataService.ts |
Defines catalog synchronization receipt APIs. |
src/vs/platform/agentHost/node/agentHostBootstrap.ts |
Wires catalog persistence into bootstrap. |
src/vs/platform/agentHost/node/agentHostCatalogListReader.ts |
Converts verified catalog rows into list metadata. |
src/vs/platform/agentHost/node/agentHostCatalogProjection.ts |
Implements canonical bounded projections and hashing. |
src/vs/platform/agentHost/node/agentHostCatalogReconciliationService.ts |
Repairs interrupted or stale synchronization. |
src/vs/platform/agentHost/node/agentHostCatalogShadowValidator.ts |
Compares central and legacy listing results. |
src/vs/platform/agentHost/node/agentHostCatalogSourceResolver.ts |
Resolves canonical catalog source metadata. |
src/vs/platform/agentHost/node/agentHostCatalogSyncService.ts |
Coordinates local-first catalog synchronization. |
src/vs/platform/agentHost/node/agentHostDatabase.ts |
Adds sessions_v2, exclusions, and compatibility migrations. |
src/vs/platform/agentHost/node/agentHostGitStateService.ts |
Persists Git summaries through the catalog path. |
src/vs/platform/agentHost/node/agentHostPeerChatStore.ts |
Persists downgrade-compatible peer-chat membership. |
src/vs/platform/agentHost/node/agentHostServices.ts |
Registers catalog-aware persistence dependencies. |
src/vs/platform/agentHost/node/agentHostSessionTitleController.ts |
Routes title metadata through catalog synchronization. |
src/vs/platform/agentHost/node/agentHostSessionsV2MigrationService.ts |
Implements direct, resumable v2 migration. |
src/vs/platform/agentHost/node/agentService.ts |
Integrates migration, listing, synchronization, and peer chats. |
src/vs/platform/agentHost/node/agentServiceComposition.ts |
Supplies catalog persistence callbacks. |
src/vs/platform/agentHost/node/agentSessionRegistry.ts |
Adds dual-registry compatibility behavior. |
src/vs/platform/agentHost/node/agentSideEffects.ts |
Persists list-visible state changes centrally. |
src/vs/platform/agentHost/node/copilot/copilotAgent.ts |
Updates legacy adoption and metadata recovery. |
src/vs/platform/agentHost/node/localCommands/localChatCommand.ts |
Extends coordinated metadata persistence. |
src/vs/platform/agentHost/node/localCommands/renameLocalCommand.ts |
Persists title and title-source metadata together. |
src/vs/platform/agentHost/node/sessionCoordination.ts |
Integrates catalog-aware session coordination. |
src/vs/platform/agentHost/node/sessionDatabase.ts |
Adds synchronization snapshot storage and transactions. |
src/vs/platform/agentHost/test/common/sessionTestHelpers.ts |
Extends test database helpers for receipts. |
src/vs/platform/agentHost/test/node/agentHostCatalogListReader.test.ts |
Tests central row conversion and eligibility. |
src/vs/platform/agentHost/test/node/agentHostCatalogProjection.test.ts |
Tests canonicalization, limits, and verification. |
src/vs/platform/agentHost/test/node/agentHostCatalogReconciliationService.test.ts |
Tests repair and interrupted-write recovery. |
src/vs/platform/agentHost/test/node/agentHostCatalogShadowValidator.test.ts |
Tests shadow mismatch classification. |
src/vs/platform/agentHost/test/node/agentHostCatalogSourceResolver.test.ts |
Tests source and legacy metadata resolution. |
src/vs/platform/agentHost/test/node/agentHostCatalogSyncService.test.ts |
Tests synchronization ordering and recovery. |
src/vs/platform/agentHost/test/node/agentHostDatabase.test.ts |
Tests v2 schema and compatibility behavior. |
src/vs/platform/agentHost/test/node/agentHostGitStateService.test.ts |
Tests catalog-backed Git persistence. |
src/vs/platform/agentHost/test/node/agentHostPeerChatStore.test.ts |
Tests peer-chat storage and malformed data handling. |
src/vs/platform/agentHost/test/node/agentHostServices.test.ts |
Tests service dependency registration. |
src/vs/platform/agentHost/test/node/agentHostSessionTitleController.test.ts |
Tests coordinated title persistence. |
src/vs/platform/agentHost/test/node/agentHostTurnTelemetry.test.ts |
Updates telemetry fixtures for persistence wiring. |
src/vs/platform/agentHost/test/node/agentService.test.ts |
Tests rollout modes, migration, compatibility, and adoption. |
src/vs/platform/agentHost/test/node/agentServiceTestUtils.ts |
Extends AgentService database test doubles. |
src/vs/platform/agentHost/test/node/agentSessionRegistry.test.ts |
Tests dual-registry session lifecycle behavior. |
src/vs/platform/agentHost/test/node/agentSideEffects.test.ts |
Tests catalog persistence from state changes. |
src/vs/platform/agentHost/test/node/copilotAgent.test.ts |
Tests legacy adoption metadata behavior. |
src/vs/platform/agentHost/test/node/sessionCoordination.test.ts |
Tests catalog-aware coordination behavior. |
src/vs/platform/agentHost/test/node/sessionDatabase.test.ts |
Tests receipt migration and atomic persistence. |
src/vs/sessions/contrib/providers/agentHost/AGENT_HOST_SESSIONS_PROVIDER.md |
Documents the centralized catalog architecture. |
Review details
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
- Files reviewed: 44/45 changed files
- Comments generated: 7
- Review effort level: Balanced
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adopts the upstream Agent Host provider-service and chat-contribution refactors while keeping the session-catalog (Option D) work: - Rebases the durable `sessions.modified_time` column onto the sessions_v2 chain as migration 9, and adds `sessions_v2.modified_time` as migration 10 so the authoritative registry owns recency in both tables. - Threads `modifiedTime` through every sessions_v2 read, registration, envelope upsert and receipt, and makes `updateSessionModifiedTime` advance both registries in one transaction. - Moves provider registration onto `IAgentHostProviderService` while preserving catalog reconciliation scheduling and serialized discovery registrations, and routes deferred provider catalogs through the sessions_v2 importer. - Renames the catalog payload's artifact flag to `isArtifact` and adopts the upstream artifact parse contract. - Lets the catalog list reader serve the fresher of the payload and registry recency, and withholds the readable-catalog marker when an import pass leaves candidates unimported. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sandeep Somavarapu (sandy081)
changed the title
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Base:
|
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…on-db-data-migration-plan
…on-db-data-migration-plan
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Only rewrite chat-local compatibility metadata for added or changed peers on normal mutations, while retaining full repair for legacy imports and stale mirrors. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Sandeep Somavarapu (@sandy081) Attaching the repro video. For the example missing session, notice the first two sessions named 'Sept 4-*" that shows up at the top initially, which disappear after pressing back button. Most of 800 sessions also disappear. migration-repro.mp4 |
roblourens
left a comment
roblourens
left a comment
There was a problem hiding this comment.
[Experimental performance review bot]
Automated experimental performance review.
(Written by Copilot)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Recreate the released turn-delegation table alongside the catalog snapshot so databases produced by the earlier catalog-only v10 migration converge after updating. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Keep unadopted legacy sessions out of Agent Host title generation so list-only migration cannot create local session storage and prematurely transfer provider ownership. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Update: This earlier title-generation fix was not the complete fix for the 800-session disappearance. Runtime discovery had another storage-creation path, now fixed in Vijay Upadya (@vijayupadya) Thanks for the video — it exposed a concrete ownership bug. I reproduced it with a focused test: an untitled Could you please rerun using a fresh user-data profile (or an uncontaminated copy of the legacy dataset) at the latest head? Existing |
roblourens
left a comment
roblourens
left a comment
There was a problem hiding this comment.
[Experimental performance review bot]
Automated experimental performance review.
(Written by Copilot)
Keep one active synchronization and one merged trailing projection per session so summary bursts do not build an unbounded SQLite write backlog. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use type-safe property access in the payload decoder and the database sequencing test so full-repository ESLint passes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
roblourens
left a comment
roblourens
left a comment
There was a problem hiding this comment.
[Experimental performance review bot]
Automated experimental performance review.
(Written by Copilot)
Publish passive metadata without awaiting full catalog synchronization, batch discovery timestamp and dirty-marker updates, and rotate persisted bounded verification samples instead of rescanning every catalog row on each startup. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Retain central catalog ownership and bounded synchronization. Adapt the new peer workspace-transition test to central chat creation and verify listing does not read transition history. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
roblourens
left a comment
roblourens
left a comment
There was a problem hiding this comment.
[Experimental performance review bot]
Automated experimental performance review.
(Written by Copilot)
Batch parameterized chat inserts within the existing revision-checked transaction. Cover bounded SQLite statement counts and atomic rollback after a later batch fails. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
roblourens
left a comment
roblourens
left a comment
There was a problem hiding this comment.
[Experimental performance review bot]
Automated experimental performance review.
(Written by Copilot)
Avoid creating local session storage for adoptable legacy chats discovered after migration is enabled. Directory creation makes the legacy provider hide sessions before explicit adoption. Preserve existing compatibility synchronization for locally stored sessions and non-legacy discovery. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Vijay Upadya (@vijayupadya) I have now reproduced the disappearance in a real Code OSS window using a macOS port of your supplied 800-session generator and the migration-OFF -> full restart with migration-ON sequence. The confirmed fix is pushed in 892b6e0. Root cause: the initial importer already avoided creating storage for unopened legacy sessions, but runtime discovery still used the storage-creating catalog synchronization path. An empty import completed with migration OFF leaves a persisted marker; after enabling migration, legacy sessions arrive through runtime discovery. That path created Fix: runtime discovery of adoptable legacy sessions now uses the non-creating catalog synchronization path. Absent local storage stays absent; existing local databases retain the coordinated compatibility/receipt path. Other runtime discovery behavior is unchanged. Live validation with fresh, isolated profiles:
After the fix, only the three explicitly opened ordinary legacy sessions acquired local Agent Host databases; unopened legacy sessions did not. The five external fixtures retained their normal separate discovery behavior. All 800 original conversation histories remained intact. Added a regression test for runtime discovery after a completed empty import: it fails before this change and passes afterward. Validation: 529 relevant unit tests passed, 16 skipped; targeted ESLint and commit hygiene passed. Correction to my earlier update: the title-generation fix in 3392bf9 addressed a real additional path, but its focused test did not reproduce or fully fix this 800-session runtime-discovery collapse. The above results are from the actual supplied dataset flow. These changes are not in production. The lingering bad state discussed earlier applies only to test profiles run against earlier buggy builds of this PR, not production users. Please retest with a fresh isolated user-data profile and regenerated dataset; retaining the old affected profile for diagnosis is fine, and no production recovery migration is required for this PR-only bug. The reported |
Compare summary status deltas against the notifier's previous snapshot and only queue catalog persistence when persisted fields change. Activity-only and transient status updates avoid database work while read/archive changes and explicit field clears remain synchronized. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Merge main through 58ba154, retaining central registry and peer membership authority, central-only legacy discovery, and projected summary synchronization. Adopt upstream automation service construction and cover batched artifact persistence and database-free catalog listing after restart. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Stop reconciliation scheduling and drain accepted dispatches and catalog synchronization before shutting down providers and closing the central database. This preserves read/archive updates already published to clients across an immediate host restart without blocking normal passive notifications on catalog work. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Wait for asynchronous tool completions before normal SDK idle clears the active turn. Preserve immediate abort handling and original tool-result attribution when a replacement turn starts while file-edit persistence is pending. Add deterministic delayed-completion, abort, and replacement-turn regressions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Fixed the failing Electron checks with two ordering fixes:
No tests were disabled, fixtures rewritten, or timeouts increased. |
roblourens
left a comment
roblourens
left a comment
There was a problem hiding this comment.
[Experimental performance review bot]
Automated experimental performance review.
(Written by Copilot)
| if (!e.data.aborted && pendingToolCompletions.size > 0) { | ||
| const pendingTurn = this._currentTurn.value; | ||
| // File-edit persistence must finish before idle clears the turn and its tool results. | ||
| await Promise.allSettled(pendingToolCompletions); |
There was a problem hiding this comment.
[Experimental performance review bot]
Severity: medium
Tool completion callbacks are now tracked in one set for the lifetime of the session wrapper, and every normal idle waits for all entries in that set, regardless of which turn created them.
After aborting an edit and immediately sending a replacement prompt, the replacement turn can appear stuck at completion until unrelated cleanup from the cancelled turn finishes, increasing end-of-turn latency in proportion to the slowest outstanding prior edit.
Suggested fix: Track completion promises by their captured turn and await only the current turn's set. On abort, detach that turn's entries from the idle barrier while still allowing their cleanup/persistence promises to settle.
(Written by Copilot)
|
Sandeep Somavarapu (@sandy081)
|
Do not subscribe to untitled UI identities when Back returns to the new-chat input. Wait for the existing provisional-session service to publish a backend and follow its mapping changes, releasing obsolete subscriptions. Cover Back, provisioning, replacement, retirement, and return to a started session. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Vijay Upadya (@vijayupadya) Thanks for confirming that the disappearing sessions issue is resolved. I reproduced your new console error on the latest head in a real Code OSS window using your generated legacy dataset: all three legacy-open -> Back attempts produced The cause was the chat input metadata pills: Back creates a new untitled UI draft, and the pills converted that UI identity directly to a backend URI and subscribed to it before a corresponding backend existed. A debugger breakpoint confirmed Fixed and pushed in 3ec5b68. The pills now wait for an existing provisional backend mapping, use that backend URI, and follow mapping replacement/retirement without subscribing to an untitled UI identity. No database/adoption behavior was changed. Validation:
Please retry on this head; the earlier affected test profile can be used to verify this particular console-error fix. |
Merge main through 0af2bfd while retaining central session identity, peer membership, compatibility receipts, non-creating legacy discovery, shutdown drains, and provisional draft subscriptions. Adopt upstream per-turn edit-completion tracking and coverage. Verify restricted pull-request association changes persist to the central catalog and remain database-free on restarted listings; align architecture documentation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1. What this PR changes
Today,
listSessionsoften enumerates providers and opens many individualsession.dbfiles to obtain titles, workspace information, read state, changes, and chats.This PR makes
agent-host.dbthe central index for session-list data:sessions_v2: authoritative session identity, provider, timestamps, registration source, external state, and chat-backing identity.Full conversation data—including turns, drafts, attachments, annotations, and detailed edits—remains isolated in individual session and chat databases.
2. Listing and synchronization
Normal
listSessionscalls now read verified payloads directly fromagent-host.dbinstead of opening every conversation database.If one payload is missing, stale, or invalid:
Changes to titles, read state, workspace information, changes, or chats mark the payload dirty. Background reconciliation uses revisions and compare-and-set operations so delayed work cannot overwrite newer state.
3. Migration and compatibility
Existing sessions are imported from the legacy registry and provider discovery. Migration is resumable, tracked per provider, and normally runs in the background so the first list does not wait for the full migration.
The PR also protects upgrade and downgrade scenarios:
peerChats.All product schema changes are introduced in one new v5 migration, after upstream’s existing v4 migration. A separate normalizer handles databases produced by earlier versions of this draft PR.
4. Testing
Coverage includes:
The affected unit suites pass, and the real-process compatibility matrix passed 20/20 scenarios.
5. Performance
Real AHP measurements used equivalent pre-migration profile data, three fresh processes per target, and 60 warm calls per target:
origin/mainThe PR returned 32 sessions versus 26 on
origin/main, so it was faster while producing a larger result.6. PR size
The PR adds 11,782 lines, but approximately 6,717 lines—57%—are tests.
The remaining approximately 5K lines implement the database schema, catalog projection, migration, synchronization, reconciliation, chat membership, compatibility handling, and Agent Service integration. The size reflects the reliability work required for a persistence migration rather than the amount of data being moved.