Skip to content

v1.16.6: Fix JSON export leaving out account names, order, flag rules, transfer keywords, Read as choices and theme - #29

Open
gitluis wants to merge 3 commits into
fix-new-browser-settings-copyfrom
fix-json-export-fields
Open

gitluis wants to merge 3 commits into
fix-new-browser-settings-copyfrom
fix-json-export-fields

Conversation

@gitluis

@gitluis gitluis commented Nov 2, 2026 •

Copy link
Copy Markdown
Owner

Stacked on #28 (fix-new-browser-settings-copy), in a chain of fixes on top of the Clarity redesign (#16 → #17 → #19 → #20 → #21). Merge in order: #16, #17, #19, #20, #21, then the fixes #24–#38 and #39–#50 in order, then #10, then #23. #18 is a separate patch on master (already merged).
Depends on #27's code: the import saves each key through storageKey(). It doesn't need #28.

Summary

Export JSON left out six settings you make in the app, so Export → Start over → Import lost them: custom account names, the account order, the transaction (flag) rules, the transfer keywords, the "Read as" choices and the theme. The export and the import now share one list of keys that includes them. The export's version is 1.2. Imported settings take effect without a reload.

Cause

exportAllData() and importJsonFile() each had a hand-written list of 14 keys. Neither list had these six, although all six were already in State and in finlens-state.json when the export was first written (commit 02c9891).

What changed

  • One list: JSON_EXPORT_KEYS in ui.js maps each exported State key to the kind of value it holds (list, map, theme). The export writes them all and the import reads the same list. Each key is saved through storageKey(), from v1.16.4 (v1.16.4: Fix JSON import losing spotted-charge choices and category rules #27).
  • Added (format 1.2): accountLabels, accountOrder, fileAssignments, txRules, transferKeywords, theme.
  • The import accepts 1.0, 1.1 and 1.2. It doesn't check version. A key that's missing, null or holds the wrong kind of value (a list where a map belongs, an unknown theme) keeps its current value. A JSON file that isn't an object is refused: "Import failed: this file is not a FinLens export."
  • Imported settings take effect right away (_applyImportedSettings(), for keys whose value changed):
    • Account names: set on the loaded accounts; the accounts filter is rebuilt. (v1.16.12: Fix clearing an account name not bringing the default name back #35, further up the stack, removes the copy into the accounts again: there an account's own name is always its default, and acctName() shows the imported names from State.accountLabels.)
    • Account order: the accounts filter is rebuilt; screens follow on the render.
    • Flag rules: applyTxRulesToAll().
    • Transfer keywords: reapplyTransferKeywords(), the helper the Transfers screen uses, re-runs detection.
    • Theme: applyTheme().
    • Unchanged: category rules apply to the next import (or Re-categorize all), "Read as" choices to the next CSV read, and transaction edits after a reload, as before.
  • Data & reset → Export and import and the README say what the file holds.
  • v1.16.6 in CHANGELOG.md, the index.html footer, README.md (badge and footer) and views-settings.js. CLAUDE.md: §1 version; §9 JSON_EXPORT_KEYS; §12 adding a persisted field; §20 what the export holds and what the import does; Last updated.

Key by key

Key Exported? Why
budgets, buckets, debts, bills, scenarios, investAlloc, debtPlan yes (already) Plans you made
txOverrides, categories, categoryRules, recurringConfirmed yes (already) Edits, lists and choices you made
manualBalances, cardDetails yes (already) Balances, limits, APR and due days you typed
aiHistory yes (already) Your saved answers
accountLabels yes, new Names you typed (Settings → Accounts, Overview)
accountOrder yes, new The order you chose (▲▼)
txRules yes, new Flag rules you added (Transactions → Transaction rules)
transferKeywords yes, new Keywords you added or removed (Transfers)
fileAssignments yes, new "Read as" choices per CSV file name. Without them, a file FinLens guessed wrong is read differently on the next import, its account id changes, and names, balances and edits saved under the old id stop matching. File names already appear in the export as account ids (card_<file>).
theme yes, new A setting you chose; it's in finlens-state.json too. Only light, dark or system is imported.
apiKey no Never written to a file (CLAUDE.md §15). The test checks that its value isn't in the export.
transactions, accounts no Come from your CSVs. This is what the export has always meant ("your CSVs bring the transactions back").
sampleData no Says whether the loaded transactions are the samples. That's about this browser, and v1.16.0 left it out on purpose.
csvAutoImportPaused, lastFolderName no About this browser's connected folder. The pause is browser-only by design (CLAUDE.md §20).
range, selectedAccounts, currentView, pendingFiles, fileMeta, lastRefresh, _folder* no Session state; none of it is saved anywhere.
customSchemas no Nothing in the app creates one (the column-mapping UI was never built, CLAUDE.md §19). An import would also have to merge them into BANK_SCHEMAS, which only happens at boot.

Not verified

  • A real folder, and Firefox or Safari. Chromium only; no real FileSystemDirectoryHandle.
  • theme: 'system' wasn't imported in the test, only dark.
  • An export with old (pre-v1.12.0) account codes in the new keys. IdMigration.migrateState() already rewrites accountLabels, accountOrder and fileAssignments, but no export from before v1.16.6 has these keys, so no import test covers it.
  • Removing a name by import. An account whose custom name isn't in the imported file keeps the name it shows until its default comes back from the CSV. Clearing a name in Settings → Accounts behaves the same way (see below).

Older bugs found while testing, not fixed here. Each now has its own PR in this stack:

Automated checks: earlier run, not re-run · a06b536

Moved unchanged from this PR's "Test results" comment on 2026-11-04; not re-run, so not in the new table format. Originally a description section titled "How it was tested".

Headless Chromium (Playwright) at 1280×900 against samples/ (257 transactions), with the pinned CDN libraries served from npm, on this branch's final commit a06b536, chained on #28. Each script ran as NODE_PATH=$(npm root -g) node <script> <repo> <outdir>, from one harness shared by every branch in the stack: the base suites, plus every test added by this PR or by a PR below it (cumulative regression).

Script Result on a06b536
test-clarity.js 55/55
test-insights.js 137/137
test-finances.js 126/126
test-plan.js 127/127
boot.js 257 transactions, 24 views, 0 errors
race.js 7 errors, all renderImportLog (the race #18 fixes on master; not in this stack)
test-clear-edits.js (from #24) 47/47
test-rebuild-folder.js (from #25) 57/57
test-partial-import.js (from #26) 47/47
test-json-import.js (from #27) 28/28
test-new-browser-copy.js (from #28) 10/10
test-json-export.js (new in this PR) 31/31

Measured before the chain

By the agent that wrote the fix, on f5c1665 (built on d47dd8d, the JSON import commit before it was split into #27 and #28; then numbered v1.16.2).

Headless Chromium (Playwright) against samples/, on commit f5c1665. The CDN libraries are served from npm (same versions; the SRI hashes in index.html verify).

  • test-clarity.js: 55/55.
  • test-finances.js: 126/126.
  • test-plan.js: 127/127.
  • test-insights.js: 137/137. Its existing Export / Import JSON checks pass unchanged. Its "General: version line" check reads the version from the footer (the edit described in v1.16.4: Fix JSON import losing spotted-charge choices and category rules #27).
  • boot.js: 257 transactions, 24 views, 0 errors.
  • race.js: 6 errors, all renderImportLog / importLogsBody, which v1.10.5: fix Import Logs throwing when the view is left before it loads #18 fixes. None come from this change.
  • test-json-import.js (from v1.16.4: Fix JSON import losing spotted-charge choices and category rules #27, before its new-database section moved to v1.16.5: Fix a new browser database dropping the card details, the sample-data flag and the import pause #28): 31/31.
  • test-json-export.js (new): 31/31. On the JSON import fix's commit before the chain (d47dd8d) it fails 20 of 31. It covers:
    • Round trip: every exported field set to a non-default value through the app's own functions (Transactions → Transaction rules through the UI), then export, Start over, re-import samples/, import, reload. The test first checks that Start over really reset every one of the 20 fields, so the round trip proves something.
    • The file: version is 1.2; it holds exactly the 20 expected keys, each equal to State; it has no API key (neither the key name nor its value) and none of the left-out keys.
    • After the import, before any reload: all 20 keys are in State and saved under their storage keys (localStorage and app_state). Names show in acctName() and the accounts filter. The order applies. RENT PAYMENT rows became transfers and ZELLE rows stopped being transfers. The rule flags apply. The theme is dark.
    • After the reload: all 20 keys are back, and names, order, transfers, theme and transaction edits all show.
    • Older files: a 1.1 export as v1.16.5 writes it, a 1.0-shaped file, a file with wrong kinds and nulls, and a JSON array. None changes the six new keys, in State, storage or on screen, also after a reload. The array is refused with the message.
    • JSON before the CSVs (a new browser): import the file, then the samples. Every value holds, and names, order, keywords, rule flags, theme and edits all apply.
How to reproduce the bug

Run this on the base branch, fix-new-browser-settings-copy.

  1. Do the Setup from the testing steps below (InPrivate window, no folder, samples via choose the files, the six settings, then Export JSON), but check out fix-new-browser-settings-copy instead of fix-json-export-fields.
  2. Open the exported file in Notepad. It says "version": "1.1" and has no accountLabels, accountOrder, fileAssignments, txRules, transferKeywords or theme key.
  3. Go to Settings → Data & reset → Start over. Type RESET and click Erase and start over. Click OK.
  4. On the Welcome, click choose the files, select the three samples/ CSVs and click Import 3 files. Set All time.
  5. Go to Settings → Data & reset → Import JSON…, choose the export, and click OK on Imported successfully.
  6. Check the same places as test 2 below: the theme, Settings → Accounts, Settings → Import → Imported files, Transactions → Transaction rules, and Transfers → Transfer keywords.

What goes wrong: all six settings are lost. The theme is light, the checking account has its default name instead of My Checking, the account order is the default, savings_sample.csv is back to its guessed Read as, there is no LIBERTY INTERNET → Bill rule, and walmart is not a transfer keyword.

Manual QA steps

Setup

  1. git checkout fix-json-export-fields.
  2. This PR's main test uses Start over, so work in an InPrivate window (Ctrl+Shift+N) with no folder connected. Your normal browser data and folder stay untouched. InPrivate keeps its data across reloads until you close every InPrivate window.
  3. In the InPrivate window, open index.html. On the Welcome, click choose the files (in "In Safari or Firefox you'll choose the files…"). Select the three CSVs in samples/. In Choose how to read these files, keep the guesses and click Import 3 files.
  4. Click the calendar icon (Date range) and choose All time. Do this again after every reload.

Make the six settings this PR adds to the export:

  1. Gear icon → Settings → Accounts. In the checking account's Display name box, type My Checking and press Tab. Saved the name "My Checking". appears.
  2. On the savings account, click the up arrow (Move … up) until savings is first in the list.
  3. Settings → Import → Imported files. Set Read as for savings_sample.csv to Other CSV. The message says it applies from the next reload; that's fine, nothing is re-read here.
  4. Settings → General → Theme: click Dark.
  5. Go to Transactions → Transaction rules. In When description contains type LIBERTY INTERNET, Flag as Bill, click Add rule.
  6. Go to Transfers. Under Transfer keywords, type walmart in Add keyword and click Add.
  7. Settings → General → Anthropic API key: type sk-ant-test-123 and click Save key.
  8. Settings → Data & reset → Export and import → Export JSON. Save finlens-export-YYYYMMDD.json somewhere like C:\temp.

1. The export file holds the six new settings, and no API key

  1. Open the exported file in Notepad or VS Code.

Expected:

  • "version": "1.2".
  • "accountLabels" has "bank_checking": "My Checking".
  • "accountOrder" starts with "bank_savings".
  • "fileAssignments" has "savings_sample.csv": "other".
  • "txRules" has LIBERTY INTERNET with "flag": "bill".
  • "transferKeywords" includes "walmart".
  • "theme": "dark".
  • Searching for sk-ant-test-123 and for apiKey finds nothing. There is no transactions, accounts, sampleData or lastFolderName key.

Before this fix: "version": "1.1" and none of the six keys.

2. Export → Start over → Import brings them back (main fix)

  1. Settings → Data & reset → Start over. Type RESET in Type RESET to confirm. Click Erase and start over.
  2. Click OK on the alert. The app reloads empty, in the light theme.
  3. On the Welcome, click choose the files, select the three samples/ CSVs and click Import 3 files. Set All time.
  4. Go to Settings → Data & reset → Import JSON… and choose the export.
  5. Click OK on Imported successfully. Do not reload yet.
  6. Check each place below.

Expected (right away, no reload):

  • The app switches to the Dark theme, and Settings → General → Theme shows Dark selected.
  • Settings → Accounts shows savings first and the checking account named My Checking. The Accounts menu in the top bar (bank icon) lists them in that order with that name.
  • Settings → Import → Imported files: savings_sample.csv shows Read as Other CSV.
  • Transactions → Transaction rules lists LIBERTY INTERNET → Bill, with matches. Searching LIBERTY INTERNET shows the Bill flag on each row.
  • Transfers → Transfer keywords includes walmart. On Transactions, searching WALMART shows the rows as Transfer out.
  • Settings → General → Anthropic API key says No key saved.

Before this fix: all six were lost: light theme, default account names and order, no rule, no walmart keyword, Read as back to the guess.

3. Still there after a reload (persistence)

  1. Press F5. Set All time.
  2. Repeat the checks from test 2.

Expected:

  • Dark theme, My Checking, savings first, Other CSV for the savings file, the LIBERTY INTERNET rule, and the walmart keyword are all still there.

4. Older settings still import (regression)

  1. Settings → Accounts → savings Starting balance: type 1000, press Tab.
  2. Transactions → Spending pill → Category rules: Description contains UBER TRIP, Category Personal, click Add rule.
  3. Settings → Data & reset → Export JSON. Save it as a second file.
  4. Delete the UBER TRIP rule with its ×. Change savings Starting balance to 5, press Tab.
  5. Settings → Data & reset → Import JSON…, choose the second file, click OK.
  6. Press F5.

Expected:

  • Category rules lists UBER TRIP → Personal, before and after the reload.
  • Settings → Accounts shows savings Starting balance 1000.

5. A bad value is skipped, a bad file is refused

  1. Copy the first export to export-bad.json. In Notepad, change "theme": "dark" to "theme": "purple", and change "accountOrder": [ … ] to "accountOrder": {}. Save.
  2. Settings → General → Theme: click Light.
  3. Settings → Data & reset → Import JSON…, choose export-bad.json, click OK.
  4. Create not-export.json in Notepad with the text [1, 2, 3]. Save.
  5. Import JSON… and choose not-export.json.

Expected:

  • Step 3 says Imported successfully. The theme stays Light, and the account order on Settings → Accounts doesn't change.
  • Step 5 says Import failed: this file is not a FinLens export. Nothing changes.
  • F12 → Console shows no red errors.

6. The Export and import description is updated

  1. Go to Settings → Data & reset and read the Export and import panel.

Expected:

  • It reads Download your plans, transaction edits, rules, account names, order and balances, card details, categories, transfer keywords, theme and AI history as a JSON file, or load one you exported.

Please check before merging

https://claude.ai/code/session_013Z8iBXzUjiQbYziDZ8CZft


Generated by Claude Code

….16.6)

exportAllData() and importJsonFile() each had a hand-written key list
that stopped at format 1.1, so a round trip lost custom account names,
the account order, the transaction (flag) rules, the transfer keywords,
the "Read as" choices and the theme. All six were already in State and
finlens-state.json.

- ui.js: JSON_EXPORT_KEYS (State key -> kind of value), used by both.
  Adds accountLabels, accountOrder, fileAssignments, txRules,
  transferKeywords and theme. The export's version is 1.2.
- Import accepts 1.0, 1.1 and 1.2. A key that's missing, null or of
  the wrong kind keeps its current value; a file that isn't a JSON
  object is refused.
- _applyImportedSettings(): names and order show, flag rules
  re-apply, transfer detection re-runs, the theme switches, without a
  reload.
- Not exported: transactions, accounts, the API key, sampleData,
  csvAutoImportPaused, lastFolderName, session state, customSchemas.
- Data & reset says what the file holds. CHANGELOG, version in the
  four places, README, CLAUDE.md §1, §9, §12, §20.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Z8iBXzUjiQbYziDZ8CZft
…json-export-fields

Carries master's v1.10.3-v1.10.5 up the stack: declined answers (#14),
literal single "~" (#15) and the Import Logs race fix (#18), with
clarity-insights' port of #14 to the rebuilt AI tab.

One conflict: CLAUDE.md's "Last updated" line. Kept this branch's
line and added the v1.10.3-v1.10.5 history. The CHANGELOG merged cleanly.
Versions stay 1.16.6. renderImportLog() keeps #18's guards.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Z8iBXzUjiQbYziDZ8CZft
@gitluis gitluis added bug Something isn't working data loss Fixes a bug that loses or corrupts saved data labels Nov 3, 2026 — with Claude
@gitluis gitluis added this to the Clarity stack (v1.12.0 → v1.18.1) milestone Nov 3, 2026 — with Claude
… into fix-json-export-fields

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Z8iBUZo6LRYpLthSp2PRi

# Conflicts:
#	js/views-settings.js

gitluis commented Nov 4, 2026

Copy link
Copy Markdown
Owner Author

Dark theme taken out of the stack (Luis, 2026-11-04). #17 is light only for now: no ◐ toggle, no Light / Dark / Match system setting, no dark tokens. It comes back later in its own PR. #17's change was merged forward into this branch (no force push).


🤖 Generated with Claude Code

https://claude.ai/code/session_013Z8iBUZo6LRYpLthSp2PRi


Generated by Claude Code

@gitluis gitluis changed the title v1.16.6: fix JSON export leaving out account names, order, flag rules, transfer keywords, Read as choices and theme v1.16.6: Fix JSON export leaving out account names, order, flag rules, transfer keywords, Read as choices and theme Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working data loss Fixes a bug that loses or corrupts saved data

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants