Repository navigation
Design history and sync around AdGuard's per-tab storage (6.12.0) - #23
Conversation
AdGuard hands each tab the storage it had when the tab loaded, so a tab left open writes back an older history carrying the history id it was loaded with. After any history loss, old and new tabs kept swapping two ids, and every new page took the mismatch for a lost history: a full download and a "Restored N visited URLs" note on page after page. With matching ids, the configuration's cursor instead claimed rows the older copy did not hold, and they were never downloaded again. The history document now records how far into the worker's history it is complete (the worker named by a hash of its endpoint), saved in the same write as the rows. A sync resumes from there, so an older copy quietly catches up on the difference; only a history with no position at all is treated as lost. The session marker now also requires the history to reach the cursor it recorded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VCuhK4TQqcEbLXdmxLjggy
boot() initialized state from combineSnapshot(), which carries resetAt and tombstones along without applying them. A visit still held in another page's journal was painted as visited after a V-unmark or a clear, and the page's first save wrote it back next to its own tombstone, where it then loaded from the main document for good. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VCuhK4TQqcEbLXdmxLjggy
Six of eight fail against 6.11.1: a long-open tab's save, once journals pass their 10-minute TTL, drops visits, overrides, unmarks and clears made in tabs opened later; retireJournals deletes a journal its owner rewrote after the retiring tab loaded; and an import cannot restore a visit over a newer stored tombstone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VCuhK4TQqcEbLXdmxLjggy
Eight failing tests documenting sync defects under AdGuard-style per-tab storage snapshots: push horizon stranding, stale sync config write-back, full re-upload at the visit cap, endpoint-change re-download, session freshness, lease exclusion, burst cost and worker reset. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VCuhK4TQqcEbLXdmxLjggy
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VCuhK4TQqcEbLXdmxLjggy
Four cases the fix leaves open: stamp-less documents from tabs still running 6.11.0 (repeated Restored toast with a full download; silent loss when the ids match), a second history save and a follow-up sync pass for every push-only sync, and a stale stamp surviving a document-only exchange. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VCuhK4TQqcEbLXdmxLjggy
AdGuard gives each tab the stored values as they were when it loaded and never shows it other tabs' writes. A review of the whole design against that found data loss and sync faults wherever a long-open tab wrote what it had loaded; each is now reproduced by a test. - Without change notifications, a tab open more than eight minutes saves only to its own journal; newer tabs fold journals in. - Journal keys are written for five minutes at most, then final, and are removed only when the main document a newer tab loaded covers them and no tab lacking them may still write the main document. - The single push horizon is replaced by an outbox of unconfirmed changes, merged by union across the document and journals. - The download position is kept per worker, with the worker's new store id (epoch) and a start generation, so a reset worker is re-seeded and a stale copy can never claim rows it lacks. Journals carry it. - Sync never writes the user's sync settings. - A push-only sync updates the journal, not the main document; startup syncs again after the poll interval; requests and the sync lock are bounded; one sync per tab activation; no storage lease where tabs cannot see each other's writes; imports restore entries unmarked after the backup was taken; boot applies removals from journals. Adds the reviewers' acceptance suites (review-*.test.mjs) and adapts existing tests to the position living in the history. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VCuhK4TQqcEbLXdmxLjggy
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VCuhK4TQqcEbLXdmxLjggy
- Name a journal key for its first write, not the page load, so a tab that first saves long after loading cannot have its journal taken for final and removed while a stale write can still drop it. - Never delete the previous journal key on rotation: AdGuard does not report failed writes, so it may be the only copy. Retirement removes it once a newer tab finds it covered. - Keep the outbox's confirmations per worker, and send everything once when returning to a worker after syncing with another. - Journal pulled and imported rows by key instead of reaching the journal back in time, so one old row no longer makes every save write the whole history; journals carry only changes still to send. - Against a store with no position, download first and upload only what it lacks; entries listed just for that are dropped once confirmed. - Prefer the copy that knows the store's id when two positions tie. Adds the final review's tests (review-final.test.mjs). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VCuhK4TQqcEbLXdmxLjggy
JVS_DOTTED=1 makes every store provide only the dotted GM.* API, as the Userscripts app for Safari does. All 132 tests pass that way. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VCuhK4TQqcEbLXdmxLjggy
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f46fe6965e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| (Array.isArray(answer.changes) ? answer.changes : []).forEach(row => { | ||
| if (!row || typeof row !== 'object' || !row.key) return; | ||
| const id = `${row.kind === 'o' ? 'o' : 'v'}|${row.key}`; | ||
| journalExtras.add(id); |
There was a problem hiding this comment.
Avoid copying every downloaded row into each rotated journal
When a tab downloads a large worker history, every returned row is added to journalExtras, which serializeJournal keeps emitting on every later write. A long-lived tab that continues polling therefore creates another journal containing up to the full 5,000-row local history every five minutes; it cannot retire those post-boot journals because loadedSnapshot.journals is fixed at startup. This can grow userscript storage without bound and eventually prevent saves. Stop carrying rows once they are safely covered, or otherwise bound and clean up the accumulated journals.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Confirmed, and fixed in the commit I'm about to push (not pushed yet when this was posted). Pulled rows stayed in journalExtras for the tab's whole life. Since rotation no longer deletes the page's previous key (so a write that silently fails can't take the only copy with it), a long-open tab left a full-size key behind every five minutes.
Now, on rotation, the new key overlaps only the key it replaces. Extras first carried in an older key, and journalSince before the replaced key's start, are dropped from later keys. The rows stay in the final key that first carried them until a newer tab finds them in the main document and retires it.
New test: a long-open tab that pulled a large history writes small journals a few rotations later in tests/review-final.test.mjs. It fails before the fix (206 KB latest journal) and passes after. 133/133 tests pass.
Generated by Claude Code
🚀 Deploying Preview to Cloudflare 🚀Preview Deployments by commit
|
Rows pulled from the worker stayed in every later journal write for the life of the tab, and since a page no longer removes its own keys, a long-open tab left a full-size key behind every five minutes. A new key now overlaps only the key it replaces; older pulled rows and entries stay in the final key that first carried them until a newer tab finds them in the main document and retires it. Reported by the Codex review on the pull request. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VCuhK4TQqcEbLXdmxLjggy
|
Workers Builds: javstore-sync is failing on this PR, but the failure isn't caused by this change.
The repo's own Generated by Claude Code |
Why
On AdGuard, "Restored N visited URLs from the worker" kept appearing on page after page while many JavStore tabs were open, and each time the device downloaded its whole history again. The cause is how AdGuard stores userscript data. AdGuard for Mac, Windows and Android give each tab the stored values as they were when the tab loaded, and a tab never sees what other tabs save afterwards. AdGuard's own issue tracker confirms this (CoreLibs#170, #1798; AdguardForWindows#1440).
The script assumed every tab reads the same, current storage. A review of the whole design against AdGuard's actual behaviour found that the same cause also produces silent data loss. Every finding below comes with a test that reproduces it.
What changes
GM_addValueChangeListener, a tab open more than 8 minutes saves only to its own journal. Newer tabs fold journals into the shared history. Without this, visits recorded in other tabs were lost, and unmarked items, cleared history and per-card overrides came back.Userscripts for Safari reads storage live, which was confirmed from its source, so this design is conservative but correct there.
Testing
npm test: 132/132 pass. The suites now include reviewer acceptance tests, each written to fail on the code it reviewed:tests/review-persistence.test.mjstests/review-sync.test.mjstests/review-lifecycle.test.mjstests/review-fix.test.mjstests/review-final.test.mjsmakeStore({ snapshotPerTab: true }), a test store that models AdGuard's per-tab snapshots.JVS_DOTTED=1 npm test: 132/132 pass with only the dottedGM.*API, which is the Userscripts app's shape.After updating, users should reload JavStore tabs that were open before the update. Until then those tabs run the old code.
Generated by Claude Code