diff --git a/CHANGELOG.md b/CHANGELOG.md index 0dae7fa..44cb39d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,21 @@ Format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). --- +## [1.16.3] — 2026-11-02 + +### Fixed +- **Importing some of your CSV files lost the other loaded transactions at the next boot.** + - **When:** Choose CSV files (on the Welcome or in Settings → Import) with only some of your files, e.g. just the card's, while other accounts were loaded. Also Reload CSVs when the folder no longer holds all of an account's rows: a newer export that starts where the last one ended, or an old CSV moved out of the folder. + - **Cause:** `loadFiles()` replaced every loaded transaction with the imported files' ones. On the samples, importing only the card file left 165 of 257. IndexedDB kept all 257 (`DB.importTransactions()` only adds rows), but the autosave wrote 165 to `finlens-state.json`, and boot loads that file first. The other accounts' transactions, with their edits, were gone from every screen, and card payments lost their pair with the checking autopay. The bug was already on `master`. + - **Fix:** an import adds the files' new transactions, replaces the ones it already has (same id) with the file's, and keeps every other loaded transaction, as IndexedDB does (`mergeImported()`). A leftover duplicate copy goes from memory as it goes from the store. Kept transfers go back to their CSV type and category first (`undoTransferDetection()`), so detection pairs them exactly as in a full import. + - **Unchanged:** importing your own files still replaces the sample data. A folder reload still replaces the folder's rows with the files' new copies; it no longer drops rows the folder doesn't have. Rebuild from your folder is still the way to make FinLens match the folder exactly. + - **Older files:** a kept transfer loaded from a `finlens-state.json` written by v1.16.0 or earlier doesn't record its CSV type and category. Its IndexedDB row gives them. Without a row, the type goes back to debit or credit by its sign and the category is worked out from its description, then detection pairs it again. + +### Docs +- **CLAUDE.md:** §4.5 an import never drops loaded transactions; §8 `loadFiles()` merges, `mergeImported()`, `undoTransferDetection()`; §10 memory follows the same re-upload rules as IndexedDB; §11 kept transfers are detected again. + +--- + ## [1.16.2] — 2026-11-02 ### Fixed diff --git a/CLAUDE.md b/CLAUDE.md index 82461e3..7c9c2ec 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -59,7 +59,7 @@ Until then, this section is the rule. FinLens is a **single-origin, zero-build, vanilla JavaScript personal finance dashboard** that runs entirely in the browser. There is no server, no npm, no bundler, no framework, and no TypeScript. The entry point is `index.html`; everything else is a plain `.js` file in the `js/` folder. -**Current version:** v1.16.2 (updated in `index.html` footer, `README.md`, and `CHANGELOG.md` on every release). +**Current version:** v1.16.3 (updated in `index.html` footer, `README.md`, and `CHANGELOG.md` on every release). **What it does:** Imports CSV exports from a bank (checking + savings) and credit cards, parses and normalizes them into a unified transaction model, and renders a multi-tab financial dashboard with budgeting, debt tracking, savings forecasting, investment planning, and Claude-powered AI analysis. @@ -323,6 +323,7 @@ loadStateFromFile() → found? - Don't call `saveStateToFile()` directly in hot paths — it's meant to be debounced via `_saveStateDebounced()` in `lsSet()`. - **Never use transactions straight from JSON.** `JSON.parse` turns `date` into a string and freezes the `category` getter. `loadStateFromFile()` rebuilds them with `reconstituteTx()`, as `DB.loadAll()` does; any new code that reads transactions from JSON must do the same. - **IndexedDB may not hold the loaded transactions.** After a boot or restore from `finlens-state.json` (a new browser, cleared site data, `_restoreStateIfBrowserEmpty()`), the `transactions` store can be empty or hold only some of them. Code that changes the loaded transactions works from `State.transactions`, never by replacing them with the store's rows: a store row may only fill in a value a transaction lacks. Up to v1.16.0, Clear transaction edits replaced them and wiped every transaction (§20). +- **An import never drops loaded transactions** (v1.16.3). `loadFiles()` adds the files' transactions to the loaded ones (`mergeImported()`): the same id is replaced by the file's, every other loaded transaction stays, as `DB.importTransactions()` keeps every row (§10). Up to v1.16.2 it replaced all of them with the files' ones, so importing only the card file left only the card's in memory, and the autosave wrote that to the file that boot loads first. Removing rows is Rebuild from your folder's job (§20). - **Secrets never reach the file.** `saveStateToFile()` deliberately omits `apiKey`, and `loadStateFromFile()` strips it from files written by ≤v1.7.0 (moving it to browser storage if needed, then scheduling a re-save). Keep both behaviors when changing the serialized field list. --- @@ -566,11 +567,14 @@ describeRange() // Human-readable date range label loadFiles(files, assignments, { sample, quiet, beforeImport, unopened }?) // the import. Reads every file first and changes nothing // (memory, storage, IndexedDB, sample data, the import pause) unless a transaction was read (v1.16.2). // Then: without `sample`, it clears sample data (State.sampleData) so your files replace it. + // The files' transactions join the loaded ones, none dropped (mergeImported(), v1.16.3). // quiet: no alert, the caller reports; beforeImport(result): runs after the read, false stops it; // unopened: [{ name }] files the caller couldn't open. Returns { imported, reason, transactions, files } parseCsvFile(file, kind) // reads one CSV → { accountId, rows, txs, problem, … } and writes nothing; importParsedCsv(p) imports it importProblemText(res) // why a loadFiles() result's files gave no transactions, in words (importFileProblem(f) for one) reloadFromDirHandle(handle, opts?) // (views-data.js) the folder's CSVs → loadFiles(); opts as loadFiles()'s; returns its result +mergeImported(loaded, fresh) // loaded + the files' transactions by id, leftover copies out, kept transfers undetected (v1.16.3) +undoTransferDetection(t, row?) // a detected transfer back to its CSV type and category, with its edit, before detectTransfers() ``` --- @@ -706,13 +710,15 @@ The account id comes last on purpose: `migrate.js` relies on it to rename accoun **Re-upload safety:** IDs are final when they reach `DB.importTransactions()`, which reads the stored keys once and skips any ID it already holds. Re-uploading a file — or a newer export with rows appended — inserts only genuinely new rows. On each import it also deletes leftover copies from the old bug: a suffixed row whose base transaction is in the file but whose exact ID is not. Same-day identical rows always appear together in a bank export, so the file is authoritative for how many copies exist. The import results card reports these as "leftover duplicate copies removed". +**Memory follows the same rules** (v1.16.3). `loadFiles()` puts the files' transactions into `State.transactions` the way `DB.importTransactions()` puts them into the store (`mergeImported()`): an id it already has is replaced by the file's, a new one is added, a leftover copy as above is removed, and every other loaded transaction stays. So importing one file, or a newer export that starts later, keeps the other accounts and the older rows, and `finlens-state.json` gets them all. + **A stored row holds the CSV's values, never an edit.** `DB.importTransactions()` stores the transaction's `_originalDesc`, `_originalCategories` and `_originalType` as the row's `description`, `categories` and `type`, and the flags from the rules (`txRuleFlag()`), not what `mkTx()` shows. An import can run while edits exist (Rebuild from your folder, importing again after Clear transactions). Up to v1.16.0 the row took the edited category, type and flags, so they came back after Clear transaction edits. Rows stored that way by older versions keep them until Rebuild from your folder rewrites them. --- ## 11. Transfer Detection -`detectTransfers()` runs after every `loadFiles()`. It does two passes: +`detectTransfers()` runs after every `loadFiles()`. It only looks at transactions that aren't transfers yet, so `loadFiles()` first puts the transfers it keeps back to their CSV type and category (`undoTransferDetection()`, v1.16.3): a kept leg then pairs with its partner re-read from a file, exactly as in a full import. It does two passes: 1. **Paired match:** Finds transactions across different accounts with equal-and-opposite amounts within 2 days. Sets `type='transfer'`, `categories=['Transfer out']` (negative) or `categories=['Transfer in']` (positive). 2. **Keyword match:** Marks unmatched transactions whose description matches `/transfer|zelle|ach|venmo|paypal|coinbase|savings booster/i` by direction. @@ -978,7 +984,7 @@ Multiple credit card files can be loaded simultaneously. Each gets a unique `acc --- -*Last updated: v1.16.2 — 2026-11-02 (Imports read every file before changing anything: §8 `loadFiles()` options and result, `parseCsvFile()` reads only, `importParsedCsv()`, `importProblemText()`, `reloadFromDirHandle()`; §20 Rebuild from your folder reads before it clears, read before clearing; v1.16.1: Clear transaction edits works from memory: §4.2 `_originalCategories`, `_originalType` and what a transaction keeps of its row; §4.5 IndexedDB may not hold the loaded transactions; §10 a stored row holds the CSV's values; §20 Clear transaction edits; v1.16.0: Trends, AI Analysis, Settings and the Welcome rebuilt: §3 file roles; §4.1 `cardDetails`, `sampleData`, AI history `model`; §6 tile tokens and Insights/Settings/Welcome components; §7 rule 7 covers Trends; §8 `loadFiles()` sample option; §9 `shellHero()`, `NO_RANGE_VIEWS`, ☰ → ×, JSON export/import buttons; §12 Settings tabs renamed, the Welcome, range label hidden on Trends and Settings; §13 quick analyses, tab state, history; §20 new fields, Use my own files, Export/Import JSON; §21 quick analyses send `AI_PROMPTS`; v1.15.0: Plan screens rebuilt: §3 file roles; §4.1 `debtPlan` and the Budget / Bucket / Debt / Scenario shapes; §6 `--teal-text` and Plan components; §7 rule 7 covers the Plan charts; §8 plan helpers (`payoffPlan()`, `growthByYear()` …); §9 plan form helpers, `simulatePayoff()` kept unused; §12 Plan titles, Forecast tabs, range label hidden on Plan screens; §13 debt balances in the summary; §20 Clear forecast scenarios / investment mix, Clear debts resets `debtPlan`; v1.14.0: Finances screens rebuilt: §4.1 Bill shape and `recurringConfirmed`; §6 new tokens and Finances components; §7 rule 7 inline SVG charts; §8 new aggregation helpers; §9 drawer, `TX_VIEWS`, SVG/heatmap/format helpers, bill helpers, removed renderers; §11 keywords re-run detection, `transferPairs()`, card payments; §12 page actions and Transactions pills; §20 Clear bills also clears spotted-charge choices; v1.13.0: Clarity redesign — §6 tokens and components; §9 theme, chart colors and app bar helpers; §12 app bar navigation replaces the sidebar and the emoji rule; §17 footer; §20 Data & Reset; v1.12.0: bank-neutral account codes, names, samples and docs; §22 `migrate.js` added; §3, §4, §5, §10, §18, §21 updated; v1.10.5: no section changes: Import Logs race fix; v1.10.4: §8, §13 single `~` is literal; §21 excerpts escape `~`; v1.10.3: §13 refusals; §21 declined runs; v1.10.2: §13 summary states its dates; §21 per-month derivations; v1.10.1: §13 `AI_MAX_TOKENS`, thinking and cut-off answers; v1.10.0: §21 AI Eval added; §13 `callClaude()` raw mode; v1.9.6: §14 folder access lasts until the last tab closes, not one page load; v1.9.5: §7 chart guard; v1.9.4: views-db.js changelog reads CRLF checkouts; v1.9.3: §14 folder permission: re-request on click, Reconnect banner, browser copy wins; v1.9.2: §0 branch, PR and approval workflow added; v1.9.1: §4.5 rehydrate transactions loaded from JSON; v1.9.0: §20 Data & Reset added; v1.8.1: §10 re-upload dedup rewritten for the occurrence-suffix fix; v1.8.0: AI Analysis restored with API-key and output hardening; `renderMarkdown()` added; `samples/` added; §1–3, §13, §14, §17 corrected to match the code)* +*Last updated: v1.16.3 — 2026-11-02 (An import keeps the loaded transactions it doesn't replace: §4.5 an import never drops loaded transactions; §8 `mergeImported()`, `undoTransferDetection()`; §10 memory follows the same re-upload rules; §11 kept transfers are detected again; v1.16.2: Imports read every file before changing anything: §8 `loadFiles()` options and result, `parseCsvFile()` reads only, `importParsedCsv()`, `importProblemText()`, `reloadFromDirHandle()`; §20 Rebuild from your folder reads before it clears, read before clearing; v1.16.1: Clear transaction edits works from memory: §4.2 `_originalCategories`, `_originalType` and what a transaction keeps of its row; §4.5 IndexedDB may not hold the loaded transactions; §10 a stored row holds the CSV's values; §20 Clear transaction edits; v1.16.0: Trends, AI Analysis, Settings and the Welcome rebuilt: §3 file roles; §4.1 `cardDetails`, `sampleData`, AI history `model`; §6 tile tokens and Insights/Settings/Welcome components; §7 rule 7 covers Trends; §8 `loadFiles()` sample option; §9 `shellHero()`, `NO_RANGE_VIEWS`, ☰ → ×, JSON export/import buttons; §12 Settings tabs renamed, the Welcome, range label hidden on Trends and Settings; §13 quick analyses, tab state, history; §20 new fields, Use my own files, Export/Import JSON; §21 quick analyses send `AI_PROMPTS`; v1.15.0: Plan screens rebuilt: §3 file roles; §4.1 `debtPlan` and the Budget / Bucket / Debt / Scenario shapes; §6 `--teal-text` and Plan components; §7 rule 7 covers the Plan charts; §8 plan helpers (`payoffPlan()`, `growthByYear()` …); §9 plan form helpers, `simulatePayoff()` kept unused; §12 Plan titles, Forecast tabs, range label hidden on Plan screens; §13 debt balances in the summary; §20 Clear forecast scenarios / investment mix, Clear debts resets `debtPlan`; v1.14.0: Finances screens rebuilt: §4.1 Bill shape and `recurringConfirmed`; §6 new tokens and Finances components; §7 rule 7 inline SVG charts; §8 new aggregation helpers; §9 drawer, `TX_VIEWS`, SVG/heatmap/format helpers, bill helpers, removed renderers; §11 keywords re-run detection, `transferPairs()`, card payments; §12 page actions and Transactions pills; §20 Clear bills also clears spotted-charge choices; v1.13.0: Clarity redesign — §6 tokens and components; §9 theme, chart colors and app bar helpers; §12 app bar navigation replaces the sidebar and the emoji rule; §17 footer; §20 Data & Reset; v1.12.0: bank-neutral account codes, names, samples and docs; §22 `migrate.js` added; §3, §4, §5, §10, §18, §21 updated; v1.10.5: no section changes: Import Logs race fix; v1.10.4: §8, §13 single `~` is literal; §21 excerpts escape `~`; v1.10.3: §13 refusals; §21 declined runs; v1.10.2: §13 summary states its dates; §21 per-month derivations; v1.10.1: §13 `AI_MAX_TOKENS`, thinking and cut-off answers; v1.10.0: §21 AI Eval added; §13 `callClaude()` raw mode; v1.9.6: §14 folder access lasts until the last tab closes, not one page load; v1.9.5: §7 chart guard; v1.9.4: views-db.js changelog reads CRLF checkouts; v1.9.3: §14 folder permission: re-request on click, Reconnect banner, browser copy wins; v1.9.2: §0 branch, PR and approval workflow added; v1.9.1: §4.5 rehydrate transactions loaded from JSON; v1.9.0: §20 Data & Reset added; v1.8.1: §10 re-upload dedup rewritten for the occurrence-suffix fix; v1.8.0: AI Analysis restored with API-key and output hardening; `renderMarkdown()` added; `samples/` added; §1–3, §13, §14, §17 corrected to match the code)* --- diff --git a/README.md b/README.md index b431167..41889f5 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ > A zero-install, browser-native finance dashboard. Drop in your bank CSVs, get instant clarity on your money. -**Version:** v1.16.2 · **Stack:** Vanilla JS, HTML, CSS · **Install:** None +**Version:** v1.16.3 · **Stack:** Vanilla JS, HTML, CSS · **Install:** None --- @@ -261,4 +261,4 @@ See [CHANGELOG.md](CHANGELOG.md) for the full release history. --- -*FinLens v1.16.2 — Built for one person's finances, designed to be useful for anyone's.* +*FinLens v1.16.3 — Built for one person's finances, designed to be useful for anyone's.* diff --git a/index.html b/index.html index 1011cb2..c9fefb6 100644 --- a/index.html +++ b/index.html @@ -1164,7 +1164,7 @@
- + diff --git a/js/core.js b/js/core.js index c622cee..5d4790d 100644 --- a/js/core.js +++ b/js/core.js @@ -848,18 +848,20 @@ async function loadFiles(fileList, assignments, opts = {}) { // (the samples use the same account ids as real checking and savings exports). if (State.sampleData && !opts.sample) { State.accounts = {}; + State.transactions = []; // the import keeps loaded transactions (mergeImported()), so drop the samples here State.selectedAccounts = 'all'; if (typeof DB !== 'undefined' && DB._db) await DB.clearStores(['transactions', 'accounts', 'import_log']); State.sampleData = false; lsSet('sampleData', false); if (typeof updateFolderBanner === 'function') updateFolderBanner(); } State.fileMeta = []; - State.transactions = []; + const fresh = []; for (const p of read) { if (!p.schema || p.problem) continue; // a file that gave no transactions creates no account - const txs = await importParsedCsv(p); - State.transactions.push(...txs); + fresh.push(...await importParsedCsv(p)); } + // The files' transactions join the loaded ones; none of the others is dropped (v1.16.3). + State.transactions = await mergeImported(State.transactions, fresh); // Edits still saved under a transaction's id from before v1.12.0 (e.g. kept while the // transactions were cleared) move to its current id (migrate.js). const adopted = IdMigration.adoptOverrides(State.transactions); @@ -883,6 +885,57 @@ async function loadFiles(fileList, assignments, opts = {}) { return result; } +// The loaded transactions after an import (v1.16.3, for loadFiles()): the files' +// transactions plus every loaded transaction they don't have. That is what IndexedDB holds +// after the same import: DB.importTransactions() adds the rows it lacks and keeps every +// other (CLAUDE.md §10). Up to v1.16.2 an import replaced all loaded transactions with +// the files' ones. Importing only the card file left only the card's transactions in +// memory, and a newer export that starts where the last one ended left only the new rows. +// The autosave wrote that to finlens-state.json, which boot loads first. +// - A loaded transaction with the id of one in the files is replaced by the file's. +// - A leftover copy goes, as importTransactions() removes it from the store: a suffixed +// id whose base is in the files but whose exact id isn't. +// - A kept transfer leg goes back to its values before detection (undoTransferDetection()), +// so loadFiles()'s detectTransfers() can pair it again, with a row just re-read too. +async function mergeImported(loaded, fresh) { + const baseOf = (id) => id.replace(/_\d+$/, ''); + const ids = new Set(fresh.map(t => t.id)); + const bases = new Set(fresh.map(t => baseOf(t.id))); + const kept = loaded.filter(t => !ids.has(t.id) && !(t.id !== baseOf(t.id) && bases.has(baseOf(t.id)))); + // A transfer from a ≤v1.16.0 finlens-state.json doesn't record its CSV type and + // category; its IndexedDB row does, when this browser has one. + const known = (t) => t._originalType && Array.isArray(t._originalCategories) && t._originalCategories.length; + let rows = new Map(); + if (kept.some(t => t.type === 'transfer' && !known(t)) && typeof DB !== 'undefined' && DB._db) { + rows = new Map((await DB._getAllTransactions()).map(r => [r.id, r])); + } + kept.forEach(t => undoTransferDetection(t, rows.get(t.id))); + return kept.concat(fresh); +} + +// Puts a transfer found by detectTransfers() back to what an import makes before detection +// (v1.16.3): its CSV type and category (_originalType, _originalCategories), with its edit. +// Detection only pairs transactions that aren't transfers yet, so a leg kept by +// mergeImported() must go back first or its partner, re-read from a file, stays unpaired. +// A type the user set to transfer stays. Without the CSV values (a transaction from a +// ≤v1.16.0 finlens-state.json), its IndexedDB `row` gives them; without a row, the type +// goes back to debit or credit by its sign and the category is worked out from the +// description, as Clear transaction edits does. Changes `t` in place, as detection does. +function undoTransferDetection(t, row) { + const ov = State.txOverrides[t.id] || {}; + if (t.type !== 'transfer' || ov.type === 'transfer') return; + const has = (c, ty) => Array.isArray(c) && c.length > 0 && !!ty; + let cats = t._originalCategories, type = t._originalType; + if (!has(cats, type) && row) { cats = row.categories; type = row.type; } + if (!has(cats, type)) { + const keep = (t.categories || []).filter(c => c !== 'Transfer out' && c !== 'Transfer in'); + cats = keep.length ? keep : [autoCategory(t._originalDesc || t.description)]; + type = t.amount < 0 ? 'debit' : 'credit'; + } + t.type = ov.type || type; + t.categories = ov.categories && ov.categories.length ? [...ov.categories] : (ov.category ? [ov.category] : [...cats]); +} + function detectTransfers() { // Build keyword regex from user-editable list (falls back to defaults if empty) const kws = (State.transferKeywords && State.transferKeywords.length) diff --git a/js/views-settings.js b/js/views-settings.js index ad36643..6431284 100644 --- a/js/views-settings.js +++ b/js/views-settings.js @@ -60,7 +60,7 @@ function _renderGeneralSettings(root) { /* ---- About (the Theme row is gone: light only for now) ---- */ appendHtml(root,`