Skip to content

v1.16.8: Fix flags set by transaction rules disappearing after a reload - #31

Open
gitluis wants to merge 3 commits into
fix-plan-link-idsfrom
fix-rule-flags-lost-on-reload
Open

gitluis wants to merge 3 commits into
fix-plan-link-idsfrom
fix-rule-flags-lost-on-reload

Conversation

@gitluis

@gitluis gitluis commented Nov 2, 2026 •

Copy link
Copy Markdown
Owner

Stacked on #30 (fix-plan-link-ids), in a chain of fixes on top of the Clarity redesign (#16 → #17 → #19 → #20 → #21). Merge in order: #16, #17, #19, #20, #21, then the fixes #24–#38 and #39–#50 in order, then #10, then #23. #18 is a separate patch on master (already merged).
This fix doesn't use any earlier fix's code; it is stacked only so versions and the CHANGELOG merge cleanly.

Summary

A transaction rule (Transactions → Transaction rules, e.g. LIBERTY INTERNET → Bill) flagged its transactions right away, but a reload unflagged them again (6 → 0 on samples/). The rule itself stayed listed. All three flags a rule can set were affected: Bill, Recurring and Subscription. The reverse also happened: a rule that existed at import time, once removed, came back on the next reload.

Now the flags are worked out from your edits and the current rules every time a transaction is built or loaded. Stored flags are no longer read.

Cause

  • Where the flags were set: mkTx() set them from State.txRules when a CSV row was parsed. DB.importTransactions() stored them in the new IndexedDB row.
  • What a rule change did: adding or removing a rule ran applyTxRulesToAll(), which changed only the transactions in memory. A CSV re-import skips rows already stored, so the stored flags never changed either.
  • Reload from IndexedDB: DB.loadAll() → reconstituteTx() read isBill, isRecurring and isSubscription from the stored row. That row held the flags as they were at import.
  • Boot from finlens-state.json: loadStateFromFile() → reconstituteTx() read the flags from the file. The file was only right if it was saved while the flags in memory were. After a reload had lost them, the next save wrote the lost flags into the file too.
  • Knock-on effects after a reload: the Transactions Flags column, the drawer's checkboxes, Bills (a spotted charge is confirmed as a subscription only if its transactions are) and the AI summary's subscription list all read the lost flags.
  • master has the bug too. The same reconstituteTx() code is on origin/master (9ba3e79, v1.10.4). probe-mech.js gives the same results there as on 70c7125. On master the Re-apply rules button brought the flags back until the next reload. v1.14.0 removed that button, so on this stack there was no workaround.

What changed

  • core.js: txFlags(desc, ov, rules?) (new). Returns the three flags: the edit in txOverrides if there is one, otherwise whether a rule matches the original description.
  • core.js: txRuleFlag(desc, flag, rules = State.txRules) takes an optional rule list.
  • core.js: mkTx() and applyTxRulesToAll() use txFlags(). Same results as before.
  • db.js: reconstituteTx(raw, ov, rules?) uses txFlags() instead of the row's stored flags. This covers DB.loadAll(), DB.queryTransactions(), loadStateFromFile(), loadFiles() (adopted overrides) and DataCleanup.clearEdits().
  • core.js: loadStateFromFile() passes the file's own txRules, because State only gets them after the function returns.
  • Clear transaction edits (Settings → Data & reset) rebuilds transactions with reconstituteTx() (in DataCleanup._unedited(), from v1.16.1: Fix "Clear transaction edits" wiping every transaction when the browser database is empty #24), so it now gets the flags from txFlags() too. v1.16.1: Fix "Clear transaction edits" wiping every transaction when the browser database is empty #24 already set them from the rules there, so its result is the same. clearEdits() itself is unchanged.
  • Unchanged: manual flag edits still beat rules. The IndexedDB rows still store the flags at import; nothing reads them now, so no data needs rewriting. Without rules, nothing changes. On samples/ the AI summary and the AI Eval's sample summary are byte-identical to v1.16.0's, so docs/EVAL.md stays valid.
  • Version v1.16.8 in CHANGELOG.md, the index.html footer, README.md (badge and footer), views-settings.js and CLAUDE.md §1.
  • CLAUDE.md: §4.2 how the flags are worked out; §8 txRuleFlag(), txFlags(), applyTxRulesToAll(); §19 the flags in the transactions store; "Last updated".
  • Not touched: exportAllData(), importJsonFile(), the State-key-to-storage-key handling, DB._migrateSync(), DataCleanup.clearEdits().

Found, not fixed

Not verified

  • No real browser or real folder. The state-file boot used a fake folder handle injected before init.js (idbGet('csvDirHandle') overridden), not the File System Access API in Edge or Chrome.
  • Firefox and Safari were not run.
  • JSON import of rules is now below this PR in the stack: v1.16.6: Fix JSON export leaving out account names, order, flag rules, transfer keywords, Read as choices and theme #29 imports txRules and calls applyTxRulesToAll(), which this change keeps working. test-json-export.js passes on this branch's commit (above); it wasn't checked beyond that suite.
  • Data saved before v1.12.0 (old account codes) combined with rules wasn't tested. The migration rebuilds transactions with reconstituteTx() before renaming ids, so the flags should follow the rules there too.
  • Phone layouts weren't checked; desktop only, as agreed.
Automated checks: earlier run, not re-run · 7289d64

Moved unchanged from this PR's "Test results" comment on 2026-11-04; not re-run, so not in the new table format. Originally a description section titled "How it was tested".

Headless Chromium (Playwright) at 1280×900 against samples/ (257 transactions), with the pinned CDN libraries served from npm, on this branch's final commit 7289d64, chained on #30. Each script ran as NODE_PATH=$(npm root -g) node <script> <repo> <outdir>, from one harness shared by every branch in the stack: the base suites, plus every test added by this PR or by a PR below it (cumulative regression).

Script Result on 7289d64
test-clarity.js 55/55
test-insights.js 137/137
test-finances.js 126/126
test-plan.js 127/127
boot.js 257 transactions, 24 views, 0 errors
race.js 6 errors, all renderImportLog (the race #18 fixes on master; not in this stack)
test-clear-edits.js (from #24) 47/47
test-rebuild-folder.js (from #25) 57/57
test-partial-import.js (from #26) 47/47
test-json-import.js (from #27) 28/28
test-new-browser-copy.js (from #28) 10/10
test-json-export.js (from #29) 31/31
test-plan-links.js (from #30) 60/60
test-rule-flags.js (new in this PR) 67/67

Measured before the chain

By the agent that wrote the fix, on 650ec68 (built on 70c7125, v1.16.0; then numbered v1.16.6). This branch's run uses that version of test-rule-flags.js; from #32 up, the stack runs #32's update of it.

Headless Chromium (Playwright) against samples/, desktop viewport 1280×900. Each script is run as NODE_PATH=$(npm root -g) node <script> <repo> <outdir>. CDNs are blocked in the sandbox, so the harness serves the pinned libraries from npm.

Script Before (70c7125) After (650ec68)
test-rule-flags.js (new) 36/67 67/67
test-clarity.js 55/55 55/55
test-finances.js 126/126 126/126
test-plan.js 127/127 127/127
test-insights.js 137/137 137/137
boot.js 0 errors 0 errors
race.js 6 errors, all renderImportLog 7 errors, all renderImportLog
  • race.js: every error is the known renderImportLog race that PR v1.10.5: fix Import Logs throwing when the view is left before it loads #18 fixes. None come from this change. The count varies between 6 and 8 from run to run.
  • No existing suite was changed.
  • test-rule-flags.js covers:
    • Every flag: the rule form offers exactly Bill, Recurring and Subscription. One rule per flag, each matching 6 sample rows.
    • Reload from IndexedDB: 6/6/6 flags after a reload. Before the fix: 0/0/0.
    • Boot from a finlens-state.json payload: a fake connected folder is injected before init.js, in a new browser context with empty storage. Three payloads are booted: the file as written; rules kept with the flags off (what a session wrote after a reload lost them); rules removed with the flags still on. The flags follow the file's rules each time, and again after a reload. Browser storage holding different rules doesn't change that.
    • CSV re-import: the same samples are imported again (rows already stored are skipped), with and without a reload afterwards.
    • A rule edited and a rule removed: Bill → Recurring through the panel (delete, then add). A keyword changed from LIBERTY INTERNET to LUMA ELECTRIC through State, saved, then applyTxRulesToAll(); the panel has no edit control. Two flags on one keyword, then that keyword's rules deleted. A rule removed after a re-import. Rules that existed at import time, then removed. Each is checked before and after a reload.
    • A manual edit beats a rule: Bill unticked in the drawer on a rule-matched row; Subscription ticked in the drawer and Mark as bill on rows no rule matches. Checked after a reload, after the rule is removed and added back, and after a boot from the state file.
    • Views match State: the Transactions table's Flags column (Bill / Recurring / Sub chips on every row a search finds); the drawer's checkboxes; Bills (T-MOBILE is spotted, and confirming it adds a bill marked Subscription, with "1 is a subscription" in the KPI); the AI summary's Subscriptions list.
    • Clear transaction edits: the rules' flags afterwards, and after a reload.
    • No rules: nothing flagged, and the AI summary is byte-identical between base and fix (summary-no-rules.txt).
  • probe-mech.js shows the mechanism on origin/master, 70c7125 and this branch. It counts the flags in memory and in the stored rows after each step.
    • Rule added after import: in memory 6 → 0 after a reload on master and base; 6 → 6 with the fix. The stored rows stay at 0 throughout.
    • Rule present at import, then removed: 0 → 6 after a reload on master and base; 0 → 0 with the fix.
  • probe-eval.js: the AI Eval's sample summary (AIEval.loadSamples()) is byte-identical on base and fix (941 bytes).
How to reproduce the bug

Run this on the base branch, fix-plan-link-ids. master has the same bug (there, Re-apply rules hides it until the next reload).

  1. Do the Setup from the testing steps below, but check out fix-plan-link-ids instead of fix-rule-flags-lost-on-reload. Import the three sample CSVs with choose the files, so no folder is connected.
  2. Go to Transactions and click Transaction rules.
  3. In When description contains, type LIBERTY INTERNET. Leave Flag as on Bill. Click Add rule.
  4. Search LIBERTY. All 6 LIBERTY INTERNET AUTOPAY rows show a Bill chip.
  5. Press F5. Set All time again if needed.
  6. Search LIBERTY again. Don't open or save any row's drawer.

What goes wrong: the rule is still listed (Contains LIBERTY INTERNET → Bill), but 0 of the 6 rows have the Bill chip after the reload.

Manual QA steps

Setup

  1. Copy the three CSVs from samples/ into a new scratch folder (for example FinLens-test-31). Do not use your real folder.
  2. git checkout fix-rule-flags-lost-on-reload, then open index.html in Edge.
  3. If FinLens already has data in this browser, go to Settings → Data & reset → Start over, type RESET and click Erase and start over.
  4. On the welcome screen, click the choose the files link (under the buttons) and select all three CSVs. This connects no folder, so a reload comes from IndexedDB, the path where the bug showed.
  5. Open the date range button (calendar icon, top bar) and choose All time.

Tip: in tests 1–5, do not click Save changes in a transaction's drawer on a rule-matched row. Saving the drawer pins all three flags on that row (a known older behaviour, fixed in #32), which would hide what you're testing.

1. A Bill rule survives a reload (the main fix)

  1. Go to Transactions and click Transaction rules (top right).
  2. In When description contains, type LIBERTY INTERNET. Leave Flag as on Bill. Click Add rule.
  3. Check the rule row and type LIBERTY in Search description or notes at the top of the list.
  4. Press F5. Set All time again if needed. Search LIBERTY again.

Expected:

  • The rule row reads Contains LIBERTY INTERNET → Bill, 6 matches.
  • Before and after F5, all 6 LIBERTY INTERNET AUTOPAY rows show a Bill chip in the Flags column.

Before this fix: after F5 the rule was still listed, but 0 of the 6 rows had the Bill chip.

2. Recurring and Subscription rules too

  1. In Transaction rules, add LUMA ELECTRIC → Recurring, then T-MOBILE → Subscription.
  2. Press F5.
  3. Search LUMA, then T-MOBILE.
  4. Click the description of one T-MOBILE AUTOPAY row and look at Flags in the drawer. Click Cancel.

Expected:

  • All 6 LUMA ELECTRIC BILL PAY rows show Recurring; all 6 T-MOBILE AUTOPAY rows show Sub.
  • The drawer has Subscription ticked.

3. Removing a rule keeps its flags off after a reload

  1. In Transaction rules, click × on the LIBERTY INTERNET row.
  2. Search LIBERTY. The Bill chips are gone.
  3. Press F5 and search LIBERTY again.

Expected:

  • No Bill chips on the 6 rows, before or after F5.

4. A rule that existed at import, then removed

  1. Add the rule LIBERTY INTERNET → Bill again.
  2. Go to Settings → Data & reset. Click Clear next to Clear transactions and accounts, then Click again to confirm.
  3. Re-import the three CSVs: Settings → Import → Choose CSV files, select all three.
  4. Back in Transactions → Transaction rules, click × on LIBERTY INTERNET.
  5. Press F5 and search LIBERTY.

Expected:

  • No Bill chips after F5.

Before this fix: the 6 Bill chips came back after F5, because the rows were stored with the flag at import.

5. Bills screen and confirming a subscription

  1. Keep the T-MOBILE → Subscription rule from test 2. Press F5.
  2. Go to Transactions and click the Bills pill.
  3. In Spotted recurring charges, find T-MOBILE and click Confirm as bill.

Expected:

  • The new bill shows a Subscription tag next to its name.
  • The Bills tracked tile says 1 is a subscription (or counts it among the subscriptions).

6. A manual flag still beats a rule

  1. Add LIBERTY INTERNET → Bill again.
  2. Open one LIBERTY INTERNET AUTOPAY row. Untick Bill under Flags. Click Save changes.
  3. Open a NETFLIX.COM row. Tick Subscription. Click Save changes.
  4. Press F5. Search LIBERTY, then NETFLIX.
  5. Delete the LIBERTY INTERNET rule and add it back. Search LIBERTY again.

Expected:

  • The row you unticked has no Bill chip; the other 5 do. This holds after F5 and after the rule is removed and re-added.
  • The edited NETFLIX row shows Sub; other NETFLIX rows don't.

7. Boot from finlens-state.json (connected folder)

  1. Settings → Data & reset → Start over, type RESET, click Erase and start over.
  2. Click Pick your CSV folder and choose the scratch folder. Allow read and write access. Set All time.
  3. Add LIBERTY INTERNET → Bill. Wait about 5 seconds (the backup saves 5 seconds after a change).
  4. Close the tab. Open index.html again in a new tab.
  5. If FinLens asks for access, go to Settings → Import and click Reconnect folder, then allow.
  6. Search LIBERTY in Transactions.
  7. Remove the rule, wait 5 seconds, close the tab, reopen and reconnect again. Search LIBERTY.

Expected:

  • After the first reopen, all 6 rows show Bill.
  • After the second, none do.

8. No rules: nothing changes

  1. Delete every rule in Transaction rules.
  2. Press F5.
  3. Look at the Flags column over a page or two, and at Transactions → Bills.

Expected:

  • No row has a flag chip (test 7 started over, so the hand edits from test 6 are gone).
  • Settings → General and the footer show v1.16.8.

Please check before merging

  • In Edge or Chrome with your own data: add a transaction rule, reload, and the matching rows still show the flag in Transactions.
  • With a connected folder: close the browser, reopen, reconnect, and the flags are still there (boot from finlens-state.json).
  • Remove a rule that existed when you imported, reload, and its flags stay off.
  • A flag you set or cleared yourself in the drawer still wins over a rule.
  • Bills → confirm a spotted charge whose transactions a Subscription rule flags: the new bill is marked Subscription.
  • You're fine leaving the drawer pinning all three flags to v1.16.9: Fix the edit drawer freezing a transaction's flags when you only change its category #32.
  • Version v1.16.8 shows in the footer and Settings → General.
  • Manual QA passed in Edge/Chrome with a real folder

https://claude.ai/code/session_013Z8iBXzUjiQbYziDZ8CZft


Generated by Claude Code

A transaction's Bill, Recurring and Subscription flags were worked out
from the transaction rules only when its CSV row was first imported, and
stored in the IndexedDB row. Adding or removing a rule changed only the
transactions in memory. On reload, reconstituteTx() read the flags back
from the row (and from finlens-state.json on a file boot), so a rule
added later lost its flags and a removed rule's flags came back. The bug
is also on master.

txFlags() in core.js now works the three flags out from the user's edit
in txOverrides, else the current rules. mkTx(), reconstituteTx() and
applyTxRulesToAll() all use it; stored flags are no longer read.
loadStateFromFile() passes the file's own txRules, which State only
gets after the file is read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Z8iBXzUjiQbYziDZ8CZft
…ost-on-reload

Carries master's v1.10.3-v1.10.5 up the stack: declined answers (#14),
literal single "~" (#15) and the Import Logs race fix (#18), with
clarity-insights' port of #14 to the rebuilt AI tab.

One conflict: CLAUDE.md's "Last updated" line. Kept this branch's
line and added the v1.10.3-v1.10.5 history. The CHANGELOG merged cleanly.
Versions stay 1.16.8. renderImportLog() keeps #18's guards.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Z8iBXzUjiQbYziDZ8CZft
@gitluis gitluis added bug Something isn't working data loss Fixes a bug that loses or corrupts saved data labels Nov 3, 2026 — with Claude
@gitluis gitluis added this to the Clarity stack (v1.12.0 → v1.18.1) milestone Nov 3, 2026 — with Claude
…le-flags-lost-on-reload

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Z8iBUZo6LRYpLthSp2PRi

# Conflicts:
#	js/views-settings.js
@gitluis gitluis changed the title v1.16.8: fix flags set by transaction rules disappearing after a reload v1.16.8: Fix flags set by transaction rules disappearing after a reload Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working data loss Fixes a bug that loses or corrupts saved data

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants