Skip to content

v1.16.5: Fix a new browser database dropping the card details, the sample-data flag and the import pause - #28

Open
gitluis wants to merge 3 commits into
fix-json-import-keysfrom
fix-new-browser-settings-copy
Open

gitluis wants to merge 3 commits into
fix-json-import-keysfrom
fix-new-browser-settings-copy

Conversation

@gitluis

@gitluis gitluis commented Nov 2, 2026 •

Copy link
Copy Markdown
Owner

Stacked on #27 (fix-json-import-keys), in a chain of fixes on top of the Clarity redesign (#16 → #17 → #19 → #20 → #21). Merge in order: #16, #17, #19, #20, #21, then the fixes #24–#38 and #39–#50 in order, then #10, then #23. #18 is a separate patch on master (already merged).
Depends on #27's code: the copy and DB.loadAll() go through its storageKey().

Summary

When FinLens creates its IndexedDB database while localStorage still holds your settings, it copies them across. That copy had its own hand-written list of keys, and it missed three: cardDetails (the APR and due day under Settings → Credit cards), sampleData and csvAutoImportPaused. DB.loadAll() then set them to their defaults on that boot and on every boot after it that read IndexedDB, though localStorage still held them.

Now the copy and DB.loadAll() read one list, DB.APP_STATE_KEYS.

This was found by the audit in #27 (the JSON import storage keys) and was part of that commit before the chain. It is split out here because it is a separate bug with its own trigger.

Cause

  • When it happens: finlens_db is created while localStorage has data. In practice that's finlens_db alone being deleted (for example from the browser's developer tools) with localStorage kept. DB.open() then runs onupgradeneeded with oldVersion 0 and calls DB._migrateSync().
  • Two lists: _migrateSync() copied the keys in its own LS_KEYS list into app_state. DB.loadAll() read a second list (appStateMap, State key → storage key → default). When cardDetails and sampleData (v1.16.0) were added to the second list, they weren't added to the first, and neither was csvAutoImportPaused (v1.9.0).
  • Why the value is lost, not just uncopied: State starts from lsGet(), so it briefly has the right value. DB.loadAll() runs at boot when finlens-state.json wasn't loaded, finds no app_state row for the key, and sets the default. That repeats on every such boot, because the database now exists and is never filled again.
  • What you see:
    • the card APR and due day are gone from Settings → Credit cards, and a new debt started from that card no longer picks them up;
    • the sample-data flag is reset;
    • the import pause set by Clear transactions & accounts is reset, so a boot that didn't load finlens-state.json (no folder access yet) can import the folder's CSVs again.
  • master has it for csvAutoImportPaused. origin/master (e8e18e7, v1.10.5) has the same two lists; its _migrateSync() misses csvAutoImportPaused, which its loadAll() reads. cardDetails and sampleData don't exist there. A probe on master (localStorage csvAutoImportPaused: true, DB.destroy(), reload) gives State.csvAutoImportPaused === false while a budget in the same localStorage is copied.

What changed

  • DB.APP_STATE_KEYS (db.js): every State key kept in app_state, with the default DB.loadAll() uses when it's missing (null: keep what State has). The 25 keys and defaults are loadAll()'s old list, unchanged.
  • DB._migrateSync() copies APP_STATE_KEYS, each under storageKey() (v1.16.4: Fix JSON import losing spotted-charge choices and category rules #27), so recurringConfirmed and categoryRules are still copied from recurring and catRules.
  • DB.loadAll() reads APP_STATE_KEYS through storageKey(). Same keys, same storage names, same defaults, so a normal boot is unchanged.
  • Not changed: a database that already exists. _migrateSync() runs only when the database is created, so values lost before this fix stay lost: enter the card details again.
  • Version v1.16.5: CHANGELOG, footer, README (badge and footer), Settings → General, CLAUDE.md §1 and Last updated.
  • CLAUDE.md: §4.5 (DB.loadAll() and the new-database copy go through storageKey()), §12 (adding a persisted field: it said lsGet was all you needed; it also takes DB.APP_STATE_KEYS, saveStateToFile()'s list and §20), §19 (DB.APP_STATE_KEYS).

Not verified

  • A real deleted database. The test deletes finlens_db with DB.destroy() in the page. Deleting it from the developer tools in Chrome or Edge wasn't tried, nor Safari or Firefox.
  • Upgrading from before v1.4.0 (when finlens_db didn't exist) also runs the copy. Those versions had none of the three keys, so nothing is lost on that path; not run.
  • Phone layout wasn't checked (desktop only).
Automated checks: earlier run, not re-run · dc1eeb4

Moved unchanged from this PR's "Test results" comment on 2026-11-04; not re-run, so not in the new table format. Originally a description section titled "How it was tested".

Headless Chromium (Playwright) at 1280×900 against samples/ (257 transactions), with the pinned CDN libraries served from npm, on this branch's final commit dc1eeb4, chained on #27. Each script ran as NODE_PATH=$(npm root -g) node <script> <repo> <outdir>, from one harness shared by every branch in the stack: the base suites, plus every test added by this PR or by a PR below it (cumulative regression).

Script Result on dc1eeb4
test-clarity.js 55/55
test-insights.js 137/137
test-finances.js 126/126
test-plan.js 127/127
boot.js 257 transactions, 24 views, 0 errors
race.js 8 errors, all renderImportLog (the race #18 fixes on master; not in this stack)
test-clear-edits.js (from #24) 47/47
test-rebuild-folder.js (from #25) 57/57
test-partial-import.js (from #26) 47/47
test-json-import.js (from #27) 28/28
test-new-browser-copy.js (new in this PR) 10/10

test-new-browser-copy.js (new, 10 checks). It is section 4 of #27's test-json-import.js before the split ("a new database copies every saved key from localStorage"), moved here and extended from 5 keys to all 25:

  • every app_state key, written out in the test rather than read from the app, is set in localStorage to a value that isn't its default; budgets, buckets and debts carry ids, as every item has from v1.16.7;
  • DB.destroy() deletes finlens_db and keeps localStorage; the page reloads;
  • one check each for cardDetails, sampleData and csvAutoImportPaused: copied to app_state and kept in State;
  • the other 22 keys: copied, and kept in State;
  • a second reload, with the database now existing: State still holds every value;
  • the samples imported, Settings → Credit cards shows the copied APR (23.9) and due day (17) for the card;
  • no console or page errors.

Fails without the fix: on #27's commit it gives 5/10. The three missed keys fail, so do the second-reload check and the Credit cards check; the other 22 keys pass.

How to reproduce the bug

Run this on the base branch, fix-json-import-keys. master has the bug only for the import pause (csvAutoImportPaused); the card details below don't exist there.

  1. git checkout fix-json-import-keys. Open an InPrivate window (Ctrl+Shift+N) and open index.html. No folder connected.
  2. On the Welcome, click choose the files, select the three CSVs in samples/, keep the guesses and click Import 3 files. Do not use Try it with sample data.
  3. Go to Settings → Credit cards. Type 24.99 in APR (%) and press Tab. Type 15 in Payment due day and press Tab.
  4. Delete only the database: F12 → Application → Storage → IndexedDB → right-click finlens_db → Delete database. Leave Local storage alone.
  5. Press F5.
  6. F12 → Application → IndexedDB → finlens_db → app_state. Click refresh and look for a cardDetails row.
  7. The Welcome shows again. Click choose the files, select only samples/credit_card_sample.csv, and click Import 1 file.
  8. Go to Settings → Credit cards.

What goes wrong:

  • app_state has no cardDetails row.
  • APR (%) and Payment due day are empty, though Local storage → finlens_cardDetails still holds 24.99 and 15. They stay empty after every further F5.
Manual QA steps

Setup

  1. git checkout fix-new-browser-settings-copy.
  2. The bug needs a boot that does not load finlens-state.json and finds a newly created IndexedDB database. Tests 1–3 do this with no folder connected, in an InPrivate window (Ctrl+Shift+N), so your normal browser data is untouched. InPrivate keeps its data across F5 until you close every InPrivate window.
  3. Test 4 needs a folder: make a scratch folder, for example C:\temp\finlens-28, and copy the three samples/ CSVs into it. Never use your real folder.

To delete only the database: F12 → Application → Storage → IndexedDB → right-click finlens_db → Delete database. Leave Local storage alone.

1. Card APR and due day survive a new database (main fix)

  1. In an InPrivate window, open index.html.
  2. On the Welcome, click choose the files (in "In Safari or Firefox you'll choose the files…"). Select the three CSVs in samples/. Do not use Try it with sample data here.
  3. In Choose how to read these files, keep the guesses (Checking, Savings, Credit card) and click Import 3 files.
  4. Go to the gear icon → Settings → Credit cards. On the card, type 24.99 in APR (%) and press Tab. Type 15 in Payment due day and press Tab. Each shows Saved the APR. / Saved the due day.
  5. Go to Transactions → Spending pill → Category rules. Add UBER TRIP → Personal with Add rule.
  6. Delete finlens_db as described above.
  7. Press F5.
  8. F12 → Application → IndexedDB → finlens_db → app_state. Click the refresh icon. Look for the cardDetails, catRules and sampleData rows.
  9. The app shows the Welcome again: the transactions lived only in the deleted database. Click choose the files, select only samples/credit_card_sample.csv, and click Import 1 file.
  10. Go to Settings → Credit cards.

Expected:

  • In step 8, app_state has a cardDetails row with apr: 24.99 and dueDay: 15, a catRules row with the UBER TRIP rule, and a sampleData row (false).
  • In step 10, APR (%) shows 24.99 and Payment due day shows 15.
  • Category rules on the Spending pill lists UBER TRIP → Personal.

Before this fix: app_state had no cardDetails row, and after the re-import the APR and due day fields were empty.

2. They stay after more reloads (persistence)

  1. Press F5 twice.
  2. Go to Settings → Credit cards.
  3. F12 → Console: look for the boot line.

Expected:

  • APR (%) is still 24.99 and Payment due day is still 15.
  • The Console shows [FinLens] Boot: Loaded state from IndexedDB — 165 transactions.

Before this fix: the values stayed empty on every boot from IndexedDB, though finlens_cardDetails in Local storage still held them.

3. The sample-data flag survives a new database

  1. Close every InPrivate window. Open a new one and open index.html.
  2. Click Try it with sample data and choose the three CSVs in samples/. The yellow banner You're trying FinLens with sample data. appears.
  3. Delete finlens_db, then press F5.
  4. F12 → Application → IndexedDB → finlens_db → app_state. Click refresh and find the sampleData row.
  5. F12 → Console. Type State.sampleData and press Enter.

Expected:

  • app_state has sampleData: true.
  • The Console prints true.

Before this fix: no sampleData row, and the Console printed false, though Local storage → finlens_sampleData was still true.

4. The import pause survives a new database (folder connected)

  1. Use a normal Edge window this time. Open index.html, click Pick your CSV folder, choose the scratch folder and allow access. Set Date range to All time: Transactions reads 257.
  2. Go to Settings → Data & reset → Clear transactions and accounts → Clear → Click again to confirm. The callout says they stay cleared after a reload.
  3. Delete finlens_db.
  4. Close the tab (close Edge completely if it's the only tab) and open index.html again. Folder access lapses, so FinLens can't read finlens-state.json and boots from the new database.
  5. Go to Settings → Import.
  6. Click Reconnect folder and allow access.

Expected:

  • In step 5, the folder panel shows the yellow callout You cleared your transactions, so automatic import from this folder is paused.
  • After step 6 the app still has 0 transactions. Nothing was re-imported.
  • Data & reset → Clear transactions and accounts says automatic import is paused.

Before this fix: the callout was missing, and Reconnect folder re-imported all 257 transactions you had cleared.

5. A normal boot is unchanged (regression)

  1. In the normal window from test 4, go to Settings → Import and click Reload CSVs. Transactions (All time) reads 257.
  2. Go to Settings → Credit cards, enter APR (%) 19.5, press Tab.
  3. Press F5 (folder access is still granted, so this boot loads finlens-state.json).
  4. Check Credit cards and the Transactions tile.

Expected:

  • APR (%) shows 19.5, and Transactions reads 257.
  • F12 → Console has no red errors.

Please check before merging

  • The split: this was part of v1.16.4: Fix JSON import losing spotted-charge choices and category rules #27's commit before the chain. You're fine with it as its own PR.
  • No recovery for values already lost. A database created before this fix stays without the three keys; the PR doesn't re-copy them from localStorage.
  • DB.loadAll() is unchanged in behaviour: the same 25 keys and defaults, now read from DB.APP_STATE_KEYS.
  • Version v1.16.5 everywhere: footer, README badge and footer, Settings → General, CHANGELOG, CLAUDE.md §1.
  • Manual QA passed in Edge/Chrome with a real folder

https://claude.ai/code/session_013Z8iBXzUjiQbYziDZ8CZft


Generated by Claude Code

…d import pause (v1.16.5)

When IndexedDB is created while localStorage still holds data,
DB._migrateSync() copied a hand-written list of keys into app_state,
and DB.loadAll() read a second list. cardDetails and sampleData
(v1.16.0) and csvAutoImportPaused (v1.9.0) were only on the second, so
they weren't copied and loadAll() reset them to their defaults.

- db.js: DB.APP_STATE_KEYS, every app_state key with the default
  loadAll() uses. _migrateSync() copies those keys and loadAll() reads
  them, both under storageKey(). loadAll() loads the same keys with
  the same defaults as before.
- CHANGELOG, version in the four places, CLAUDE.md §1, §4.5, §12, §19.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Z8iBXzUjiQbYziDZ8CZft
…er-settings-copy

Carries master's v1.10.3-v1.10.5 up the stack: declined answers (#14),
literal single "~" (#15) and the Import Logs race fix (#18), with
clarity-insights' port of #14 to the rebuilt AI tab.

One conflict: CLAUDE.md's "Last updated" line. Kept this branch's
line and added the v1.10.3-v1.10.5 history. The CHANGELOG merged cleanly.
Versions stay 1.16.5. renderImportLog() keeps #18's guards.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Z8iBXzUjiQbYziDZ8CZft
@gitluis gitluis added bug Something isn't working data loss Fixes a bug that loses or corrupts saved data labels Nov 3, 2026 — with Claude
@gitluis gitluis added this to the Clarity stack (v1.12.0 → v1.18.1) milestone Nov 3, 2026 — with Claude
…-new-browser-settings-copy

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Z8iBUZo6LRYpLthSp2PRi

# Conflicts:
#	js/views-settings.js
@gitluis gitluis changed the title v1.16.5: fix a new browser database dropping the card details, the sample-data flag and the import pause v1.16.5: Fix a new browser database dropping the card details, the sample-data flag and the import pause Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working data loss Fixes a bug that loses or corrupts saved data

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants