Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
4f732c7
Restore the requested pane before scanning on workspace open
callumalpass Oct 1, 2026
1211eb5
Merge remote-tracking branch 'origin/main' into polish/perf
callumalpass Oct 1, 2026
19194ad
Name deletions in conflict choices and never copy an absent local ver…
callumalpass Oct 1, 2026
ef44f34
Merge remote-tracking branch 'origin/main' into polish/ux
callumalpass Oct 1, 2026
50d8c88
Exercise restored destination state in the background badge focus test
callumalpass Oct 1, 2026
a3eaead
Reopen IndexedDB connections closed while mobile Obsidian was suspended
callumalpass Oct 1, 2026
5236f83
Avoid quadratic attachment tail copies during mobile multipart uploads
callumalpass Oct 1, 2026
5131708
Keep excluded scopes and file-issue actions readable in narrow sync p…
callumalpass Oct 1, 2026
2174c3f
Bound portable glob matching instead of recursively exploding on wild…
callumalpass Oct 1, 2026
a50d3f9
Treat a removed conflict copy source as a changed review, not a sync …
callumalpass Oct 1, 2026
4d90d46
Carry SDK binary preflight expectations into conditional materialization
callumalpass Oct 1, 2026
53f2cf7
Avoid reserializing intact large history logs at startup
callumalpass Oct 1, 2026
cc74fd6
Resolve proven physical path aliases without overwriting colliding files
callumalpass Oct 1, 2026
52eac26
Schedule sync and invalidate descendants for folder-only vault events
callumalpass Oct 1, 2026
24c88fd
Let an explicit pause suppress polling despite a previous offline pro…
callumalpass Oct 1, 2026
f6f2d4e
Normalize sync migration markers and honor only boolean auto-sync cho…
callumalpass Oct 1, 2026
5a41c3b
Keep the status bar paused when automatic sync is deliberately stopped
callumalpass Oct 1, 2026
a245632
Enforce elapsed request deadlines when mobile timers resume late
callumalpass Oct 1, 2026
37be2a1
Prove mobile late receipts, long freezes and suspended blob streams r…
callumalpass Oct 1, 2026
91abf1f
Expose sync problems in workspace navigation when the status bar is u…
callumalpass Oct 1, 2026
86bb3e3
Read a tiny atomic checkpoint summary for background change probes
callumalpass Oct 1, 2026
3b08892
Report unavailable diagnostics clipboard access without losing keyboa…
callumalpass Oct 1, 2026
7ba430a
Rebase debounce and safety-net timestamps when the wall clock moves b…
callumalpass Oct 1, 2026
47551a4
Reuse owned chunk tails instead of copying them for each upload part
callumalpass Oct 1, 2026
92d6bb2
Pass inspected bytes through managed-file removal before trashing
callumalpass Oct 1, 2026
ff3e34f
Expose pending sync on the mobile ribbon when no status bar is available
callumalpass Oct 1, 2026
b1d90cd
Bind hosted-deletion conflict resolution to the inspected local bytes
callumalpass Oct 1, 2026
e5d0ea4
Prove tiny probe summaries see writes from other state adapters
callumalpass Oct 1, 2026
7a412fb
Coordinate mirror leases across renderer windows with Web Locks
callumalpass Oct 1, 2026
5568ec7
Retire connections under the same lease that protects SDK mirror oper…
callumalpass Oct 1, 2026
32a31b0
Merge branch 'polish/merge' into integration/polish-3
callumalpass Oct 1, 2026
2f81ff1
Merge branch 'polish/perf' into integration/polish-3
callumalpass Oct 1, 2026
6f1e75a
Merge branch 'polish/ux' into integration/polish-3
callumalpass Oct 1, 2026
37cda9a
Resolve round 3 integration: deduplicate Obsidian mock, rebuild main.js
callumalpass Oct 1, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions docs/perf/probe-chromium-2026-10-02.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
{
"notes": 20000,
"state_bytes": 43275829,
"summary_bytes": 55,
"baseline": {
"calls": 10,
"total_ms": 596.6,
"mean_ms": 59.66,
"max_call_ms": 82,
"max_frame_gap_ms": 79.1,
"ticks": 14,
"full_reads": 10
},
"optimized": {
"calls": 100,
"total_ms": 8.9,
"mean_ms": 0.089,
"max_call_ms": 0.7,
"max_frame_gap_ms": 16.1,
"ticks": 1,
"full_reads": 0,
"summary_reads": 100
}
}
56 changes: 56 additions & 0 deletions docs/perf/ui-after-2026-10-02.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
{
"mode": "current",
"files": 20052,
"coldOpen": {
"ms": 80.5,
"max_frame_gap_ms": 81.8,
"ticks": 2,
"record_loads": 0
},
"records": {
"ms": 485.1,
"max_frame_gap_ms": 27.5,
"ticks": 31,
"value": 20000
},
"progress": {
"ms": 129.1,
"max_frame_gap_ms": 16.3,
"ticks": 9,
"value": {
"renders": 1
}
},
"closedHistory": {
"ms": 3.3,
"max_frame_gap_ms": 16.1,
"ticks": 1,
"value": {
"rows": 0,
"nodes": 20
}
},
"summariesHistory": {
"ms": 11.7,
"max_frame_gap_ms": 16.1,
"ticks": 1,
"value": {
"rows": 0,
"nodes": 134
}
},
"trace": {
"long_tasks": 1,
"max_task_ms": 69.408,
"top_tasks_ms": [
69.408,
33.592,
16.251,
11.275,
8.91,
8.832,
8.62,
8.34
]
}
}
56 changes: 56 additions & 0 deletions docs/perf/ui-before-2026-10-02.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
{
"mode": "baseline",
"files": 20052,
"coldOpen": {
"ms": 453,
"max_frame_gap_ms": 461,
"ticks": 2,
"record_loads": 3
},
"records": {
"ms": 253.5,
"max_frame_gap_ms": 257,
"ticks": 2,
"value": 20000
},
"progress": {
"ms": 593.6,
"max_frame_gap_ms": 484.3,
"ticks": 9,
"value": {
"renders": 1000
}
},
"closedHistory": {
"ms": 238.6,
"max_frame_gap_ms": 241.5,
"ticks": 2,
"value": {
"rows": 2500,
"nodes": 27644
}
},
"summariesHistory": {
"ms": 217.3,
"max_frame_gap_ms": 220.5,
"ticks": 2,
"value": {
"rows": 2500,
"nodes": 27644
}
},
"trace": {
"long_tasks": 6,
"max_task_ms": 469.234,
"top_tasks_ms": [
469.234,
453.834,
253.73,
240.661,
218.629,
69.817,
26.01,
6.686
]
}
}
59 changes: 59 additions & 0 deletions docs/sync-perf-2026-10-02.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# Large-vault sync performance — 2026-10-02

All data was generated. No personal vault, LAB, staging or hosted authority was used. Browser evidence used an isolated Obsidian profile, a disposable 20k-note vault and CDP. Timings are single samples under simultaneous campaign load, not a throughput SLA.

## The polling fix

`ConnectSyncController.remoteChangesWaiting()` used to clone the complete IndexedDB mirror checkpoint before requesting one change-feed event. It now uses `IndexedDbMirrorStateStore.readCheckpoint()`: a cursor, scope epoch and recovery-required boolean stored alongside the full state in **the same transaction**. Clear removes both atomically. String state keys and array summary keys cannot collide, and no database-version upgrade is required.

A full state read primes/repairs the projection once per adapter lifetime, covering state written by older plugins, stale projections after downgrades, and missing projections. Normal startup inspection already performs that read. Subsequent polls retrieve the tiny projection and verify existence with `getKey`, which does not clone documents. Injected/test state stores without this optional port retain their full-read fallback.

Real Chromium/Obsidian, 20k generated notes:

| Metric | Old full-state probe | New controller probe |
| --- | ---: | ---: |
| State / summary size | 43,275,829 bytes | 55 bytes |
| Mean per call | 59.66ms (10 calls) | 0.089ms (100 calls) |
| Slowest call | 82ms | 0.7ms |
| Largest 16ms timer gap | 79.1ms | 16.1ms |
| Full-state reads | 10 | 0 |

Raw evidence: [`perf/probe-chromium-2026-10-02.json`](perf/probe-chromium-2026-10-02.json). The baseline reproduces the old read-plus-feed operation; the optimized measurement invokes the real controller with a generated IndexedDB state and an in-memory transport override.

Regression coverage includes 20 polls against 20k records with zero full-state `get` calls; recovery/uninitialized short-circuit behaviour; stale legacy summary reconciliation; atomic rollback when the summary write or delete aborts; and existing state-store failure/restart tests. Full plugin tests (342) and lint pass.

## Frontend responsiveness

Earlier CDP comparison of the campaign baseline against the first frontend fixes:

| Operation | Before | After |
| --- | ---: | ---: |
| Open Sync from a fresh view | 453ms / 3 record-load calls | 80.5ms / 0 record-load calls |
| Read 20k cached collection records: largest frame gap | 257ms | 27.5ms |
| 1,000 transfer notifications | 1,000 redraws / 484ms gap | 1 redraw / 16.3ms gap |
| Five closed-history redraws | 239ms / 2,500 hidden rows | 3.3ms / 0 rows |
| Five open-history summary redraws | 217ms / 2,500 hidden rows | 11.7ms / 0 hidden rows |

The record scan takes longer overall (254ms → 485ms) because it yields to input/painting by time budget. That is deliberate. Completion, failures, auth and consent changes remain immediate; only transfer-only presentation notifications coalesce. Actual history disclosure toggles and opening one run's ledger were verified in Obsidian.

Raw evidence: [`perf/ui-before-2026-10-02.json`](perf/ui-before-2026-10-02.json), [`perf/ui-after-2026-10-02.json`](perf/ui-after-2026-10-02.json). These frontend captures precede the second integration round and later glob/history/probe fixes.

Initial event-path trace found low direct handler cost: 1,000 modify callbacks ~4.6ms; 1,000 editor-change callbacks ~5.3ms. The expensive work was deferred whole-vault scans and rendering, not per-keystroke event dispatch.

## Other guarded hotspots

- Receipt history lookup: 20k actions previously caused 200,030,000 action-id reads; indexing once bounds it to 40,000.
- Consent policy: 100 progress notifications on a 20k plan previously caused 4,040,000 command reads; immutable-preview reuse bounds it to 40,000.
- Portable path globs: a small adversarial segment/path fixture previously copied 516,133 suffix arrays. Iterative wildcard matching removes recursive explosion and call-stack growth; Unicode/classes retain semantics. 100k seeded differential cases matched the old implementation. Typical 200k matches improved ~189ms → ~92ms in Node.
- Intact large history logs no longer stringify every transfer again at startup; malformed/torn lines and pruning still trigger serialized repair. A 50k-file history regression guards that path.

## Remaining SDK costs and reproduction

```
npm run test:unit
node --expose-gc --loader ./test/obsidian-loader.mjs scripts/profile-large-sync.mjs 10000 20000 50000
```

The harness generates MemoryAuthority records and notes plus 50 unselected attachments. It measures initial/incremental review, paging/read counts, one-edit apply, heap deltas, full state write volume and fake-indexeddb clone/probe cost. Fake IndexedDB is not a disk-latency measurement. To compare a built SDK candidate, set `MDBASE_PROFILE_SDK_ROOT=/absolute/path/to/packages/sync`.

One 20k-note edit still reads the entire vault during review and again during apply, and writes roughly 216MB of full checkpoint copies across five durable writes. At 50k notes that becomes ~541MB. Do not remove durability or trust mtime/size-only caches to hide this. A tested SDK proposal reuses hashes and canonical structural proofs only after comparing freshly read exact bytes with the common authority document; its patch and evidence are in the campaign lead's `reports/perf-sdk/` directory. Journal-store adoption remains a separate, recovery-sensitive improvement.
2 changes: 1 addition & 1 deletion docs/sync-ux-acceptance.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,6 @@ node scripts/test-sync-layout.cjs after-layout

`MDBASE_UX_PLAYWRIGHT` can be omitted if Playwright is already resolvable. It is not a plugin runtime dependency.

The audit captures 14 states at 1100px and 390px, in dark and light themes: healthy, offline, sign-in, copied device, paused, unexpected failure, progress, first-sync review, deletion-burst review, rebuild, unresolved conflict, pinned conflict history, enrollment, and local upload. Desktop sidebars are collapsed for realistic narrow panes. Assertions reject horizontal overflow and primary/dismiss targets below 44px in narrow panes. Long, unbroken paths are intentional. `*-checks.json` records the measured bounds; the command exits nonzero on a regression.
The audit captures 23 states at 1100px and 390px, in dark and light themes: healthy, long excluded-folder scope, local file issue, offline, sign-in, copied device, paused, unexpected failure, progress, first-sync review, deletion-burst review, rebuild, unresolved conflict (both versions present, local absent, or hosted absent), pinned conflict history, enrollment, and local upload, plus five sync problems viewed from Issues (where mobile needs an in-workspace indicator). Desktop sidebars are collapsed for realistic narrow panes. Assertions reject horizontal overflow, hidden sync problems when viewing another destination, and primary/dismiss targets below 44px in narrow panes. Long, unbroken paths are intentional. `*-checks.json` records the measured bounds; the command exits nonzero on a regression.

Review the PNGs as well as the assertions. Diff panels may scroll horizontally within their own surface; the workspace itself must not. Stop the isolated instance after testing and discard its app/profile/vault. Injected profile state is ephemeral and must never be saved as real credentials.
Loading
Loading