Skip to content

Design history and sync around AdGuard's per-tab storage (6.12.0) - #23

Merged
jitdor merged 11 commits into
mainfrom
claude/gracious-keller-do6d7n
Nov 12, 2026
Merged

jitdor merged 11 commits into
mainfrom
claude/gracious-keller-do6d7n

Conversation

@jitdor

@jitdor jitdor commented Nov 12, 2026

Copy link
Copy Markdown
Owner

Why

On AdGuard, "Restored N visited URLs from the worker" kept appearing on page after page while many JavStore tabs were open, and each time the device downloaded its whole history again. The cause is how AdGuard stores userscript data. AdGuard for Mac, Windows and Android give each tab the stored values as they were when the tab loaded, and a tab never sees what other tabs save afterwards. AdGuard's own issue tracker confirms this (CoreLibs#170, #1798; AdguardForWindows#1440).

The script assumed every tab reads the same, current storage. A review of the whole design against AdGuard's actual behaviour found that the same cause also produces silent data loss. Every finding below comes with a test that reproduces it.

What changes

  • Long-open tabs no longer overwrite the history. On managers without GM_addValueChangeListener, a tab open more than 8 minutes saves only to its own journal. Newer tabs fold journals into the shared history. Without this, visits recorded in other tabs were lost, and unmarked items, cleared history and per-card overrides came back.
  • Journals are written once and removed safely. Each journal key is named for its first write and written for 5 minutes at most. After that it is final. A journal is removed only once a newer tab finds it covered by the main document that tab loaded, and only when no tab that lacks it may still write the main document. Rotation never deletes the previous key.
  • An outbox replaces the push horizon. Changes not yet confirmed are kept per worker, inside the history and in each journal, and copies are merged by union. A long-open tab can no longer strand a visit by marking everything as pushed.
  • The download position is stored per worker inside the history, keyed by a hash of the endpoint, with the worker's store id and a start generation. Journals carry it along with the rows it covers. An outdated copy catches up from where it reaches; only a history that is really lost shows the note.
  • The worker sends a random store id. A store that comes back empty at the same address is reseeded from the device. The worker needs redeploying, which the Cloudflare build does on merge.
  • Sync never writes the user's sync settings. A long-open tab therefore cannot switch sync back on or restore an old token.
  • Smaller fixes:
    • A first sync downloads first and uploads only what the worker lacks.
    • A sync that only uploads updates the journal, not the whole document.
    • Continuous browsing re-checks the worker once per poll interval.
    • Requests and the sync lock time out, and each tab activation runs one sync.
    • The storage lease is not used where tabs cannot see each other's writes.
    • Imports restore entries that were unmarked after the backup was taken.
    • At startup, removals carried in journals are applied.

Userscripts for Safari reads storage live, which was confirmed from its source, so this design is conservative but correct there.

Testing

  • npm test: 132/132 pass. The suites now include reviewer acceptance tests, each written to fail on the code it reviewed:
    • tests/review-persistence.test.mjs
    • tests/review-sync.test.mjs
    • tests/review-lifecycle.test.mjs
    • tests/review-fix.test.mjs
    • tests/review-final.test.mjs
  • These run against makeStore({ snapshotPerTab: true }), a test store that models AdGuard's per-tab snapshots.
  • JVS_DOTTED=1 npm test: 132/132 pass with only the dotted GM.* API, which is the Userscripts app's shape.
  • Not tested on a real AdGuard or Userscripts install.

After updating, users should reload JavStore tabs that were open before the update. Until then those tabs run the old code.


Generated by Claude Code

AdGuard hands each tab the storage it had when the tab loaded, so a tab
left open writes back an older history carrying the history id it was
loaded with. After any history loss, old and new tabs kept swapping two
ids, and every new page took the mismatch for a lost history: a full
download and a "Restored N visited URLs" note on page after page. With
matching ids, the configuration's cursor instead claimed rows the older
copy did not hold, and they were never downloaded again.

The history document now records how far into the worker's history it
is complete (the worker named by a hash of its endpoint), saved in the
same write as the rows. A sync resumes from there, so an older copy
quietly catches up on the difference; only a history with no position
at all is treated as lost. The session marker now also requires the
history to reach the cursor it recorded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCuhK4TQqcEbLXdmxLjggy
boot() initialized state from combineSnapshot(), which carries resetAt and
tombstones along without applying them. A visit still held in another page's
journal was painted as visited after a V-unmark or a clear, and the page's
first save wrote it back next to its own tombstone, where it then loaded from
the main document for good.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCuhK4TQqcEbLXdmxLjggy
Six of eight fail against 6.11.1: a long-open tab's save, once journals pass
their 10-minute TTL, drops visits, overrides, unmarks and clears made in
tabs opened later; retireJournals deletes a journal its owner rewrote after
the retiring tab loaded; and an import cannot restore a visit over a newer
stored tombstone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCuhK4TQqcEbLXdmxLjggy
Eight failing tests documenting sync defects under AdGuard-style per-tab
storage snapshots: push horizon stranding, stale sync config write-back,
full re-upload at the visit cap, endpoint-change re-download, session
freshness, lease exclusion, burst cost and worker reset.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCuhK4TQqcEbLXdmxLjggy
Four cases the fix leaves open: stamp-less documents from tabs still
running 6.11.0 (repeated Restored toast with a full download; silent loss
when the ids match), a second history save and a follow-up sync pass for
every push-only sync, and a stale stamp surviving a document-only
exchange.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCuhK4TQqcEbLXdmxLjggy
AdGuard gives each tab the stored values as they were when it loaded and
never shows it other tabs' writes. A review of the whole design against
that found data loss and sync faults wherever a long-open tab wrote what
it had loaded; each is now reproduced by a test.

- Without change notifications, a tab open more than eight minutes
  saves only to its own journal; newer tabs fold journals in.
- Journal keys are written for five minutes at most, then final, and are
  removed only when the main document a newer tab loaded covers them and
  no tab lacking them may still write the main document.
- The single push horizon is replaced by an outbox of unconfirmed
  changes, merged by union across the document and journals.
- The download position is kept per worker, with the worker's new store
  id (epoch) and a start generation, so a reset worker is re-seeded and
  a stale copy can never claim rows it lacks. Journals carry it.
- Sync never writes the user's sync settings.
- A push-only sync updates the journal, not the main document; startup
  syncs again after the poll interval; requests and the sync lock are
  bounded; one sync per tab activation; no storage lease where tabs
  cannot see each other's writes; imports restore entries unmarked after
  the backup was taken; boot applies removals from journals.

Adds the reviewers' acceptance suites (review-*.test.mjs) and adapts
existing tests to the position living in the history.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCuhK4TQqcEbLXdmxLjggy
- Name a journal key for its first write, not the page load, so a tab
  that first saves long after loading cannot have its journal taken for
  final and removed while a stale write can still drop it.
- Never delete the previous journal key on rotation: AdGuard does not
  report failed writes, so it may be the only copy. Retirement removes
  it once a newer tab finds it covered.
- Keep the outbox's confirmations per worker, and send everything once
  when returning to a worker after syncing with another.
- Journal pulled and imported rows by key instead of reaching the
  journal back in time, so one old row no longer makes every save write
  the whole history; journals carry only changes still to send.
- Against a store with no position, download first and upload only what
  it lacks; entries listed just for that are dropped once confirmed.
- Prefer the copy that knows the store's id when two positions tie.

Adds the final review's tests (review-final.test.mjs).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCuhK4TQqcEbLXdmxLjggy
JVS_DOTTED=1 makes every store provide only the dotted GM.* API, as the
Userscripts app for Safari does. All 132 tests pass that way.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCuhK4TQqcEbLXdmxLjggy
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-12T20:16:51.747186Z f46fe69 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f46fe6965e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread javstore-full-layout-cleanup.user.js Outdated
(Array.isArray(answer.changes) ? answer.changes : []).forEach(row => {
if (!row || typeof row !== 'object' || !row.key) return;
const id = `${row.kind === 'o' ? 'o' : 'v'}|${row.key}`;
journalExtras.add(id);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid copying every downloaded row into each rotated journal

When a tab downloads a large worker history, every returned row is added to journalExtras, which serializeJournal keeps emitting on every later write. A long-lived tab that continues polling therefore creates another journal containing up to the full 5,000-row local history every five minutes; it cannot retire those post-boot journals because loadedSnapshot.journals is fixed at startup. This can grow userscript storage without bound and eventually prevent saves. Stop carrying rows once they are safely covered, or otherwise bound and clean up the accumulated journals.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed, and fixed in the commit I'm about to push (not pushed yet when this was posted). Pulled rows stayed in journalExtras for the tab's whole life. Since rotation no longer deletes the page's previous key (so a write that silently fails can't take the only copy with it), a long-open tab left a full-size key behind every five minutes.

Now, on rotation, the new key overlaps only the key it replaces. Extras first carried in an older key, and journalSince before the replaced key's start, are dropped from later keys. The rows stay in the final key that first carried them until a newer tab finds them in the main document and retires it.

New test: a long-open tab that pulled a large history writes small journals a few rotations later in tests/review-final.test.mjs. It fails before the fix (206 KB latest journal) and passes after. 133/133 tests pass.


Generated by Claude Code

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

🚀 Deploying Preview to Cloudflare 🚀

Preview Deployments by commit

Status Deployment URL Commit Updated (UTC) See this deployment's details
  • Build: Failed ❌

View logs ↗
693f384 2026-11-12T20:22:02.424Z View logs ↗
  • Build: Failed ❌

View logs ↗
f46fe69 2026-11-12T20:08:06.102Z View logs ↗

Rows pulled from the worker stayed in every later journal write for the
life of the tab, and since a page no longer removes its own keys, a
long-open tab left a full-size key behind every five minutes. A new key
now overlaps only the key it replaces; older pulled rows and entries
stay in the final key that first carried them until a newer tab finds
them in the main document and retires it.

Reported by the Codex review on the pull request.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VCuhK4TQqcEbLXdmxLjggy

jitdor commented Nov 12, 2026

Copy link
Copy Markdown
Owner Author

Workers Builds: javstore-sync is failing on this PR, but the failure isn't caused by this change.

The repo's own test check is the gate for merging.


Generated by Claude Code

@jitdor
jitdor merged commit 1472a0f into main Nov 12, 2026
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants