Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,20 @@ Format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).

---

## [1.10.3] — 2026-10-26

### AI Analysis
- **Declined questions say so.** When the API's safety classifiers decline a request, it comes back as a normal HTTP 200 with `stop_reason` `refusal` and no answer. The AI tab showed "(no response)" as if that were the answer and saved it to history. It now says Claude declined, shows the reason from `stop_details.explanation` when there is one, and keeps nothing in history or for copying (`aiRefusalHtml()`). Any partial text is discarded, as the API docs advise. The bug was already on `master`.
- **`send()` returns `refusal`**: `{ category, explanation }` for a declined request, otherwise `null`.

### AI Eval
- **Declined answers are counted, not scored.** The *Calls* row reports them as declined, like failed calls, and they're left out of the figures and the token averages. Each run stores its `refusal`. The v1.10.0 run had none, so it re-scores unchanged.

### Docs
- **CLAUDE.md §13** — refusals and the raw-mode result; **§21** — declined runs.

---

## [1.10.2] — 2026-10-25

### AI Analysis
Expand Down
8 changes: 5 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ Until then, this section is the rule.

FinLens is a **single-origin, zero-build, vanilla JavaScript personal finance dashboard** that runs entirely in the browser. There is no server, no npm, no bundler, no framework, and no TypeScript. The entry point is `index.html`; everything else is a plain `.js` file in the `js/` folder.

**Current version:** v1.10.2 (updated in `index.html` footer, `README.md`, and `CHANGELOG.md` on every release).
**Current version:** v1.10.3 (updated in `index.html` footer, `README.md`, and `CHANGELOG.md` on every release).

**What it does:** Imports CSV exports from Ally Bank (Checking + Savings) and Chase credit cards, parses and normalizes them into a unified transaction model, and renders a multi-tab financial dashboard with budgeting, debt tracking, savings forecasting, investment planning, and Claude-powered AI analysis.

Expand Down Expand Up @@ -672,7 +672,8 @@ Views build a fresh DOM on every `showView()` call. This is intentional. Do not
- Responses are read from every `text` content block (don't assume `content[0]` is text) and rendered with `renderMarkdown()` — never `marked.parse()`.
- AI history is capped at 50 entries (`State.aiHistory`).
- **Output budget and cut-off answers (v1.10.1).** `max_tokens` is `AI_MAX_TOKENS` (4,096). It caps the model's thinking *and* the answer combined. Sonnet 5.5 thinks by default (effort `high`), and thinking counts toward `max_tokens` even though FinLens shows only the text. At 1,500, thinking took ~750 tokens and 15 of 24 eval answers were cut off. Asking the model in the prompt to think less doesn't reliably work; raise the budget or set `output_config.effort` instead. `send()` returns `truncated` when `stop_reason` is `max_tokens` or `model_context_window_exceeded`. The AI tab then shows a warning, and the answer is saved to history with `truncated: true` and copied or saved with `AI_CUT_OFF_NOTE`. If it was cut off before any text (the text is `AI_NO_TEXT`; `aiNoAnswer()`), the tab says no answer was written and nothing goes to history. Never show a cut-off answer as if it were complete.
- **Raw mode:** `callClaude(question, label, { raw:true, summary })` sends the identical request through the same inner `send()` but touches no DOM, busy flag or history; it resolves to `{ text, stopReason, truncated, usage, model }` or throws (`.status`/`.detail` on HTTP errors). The AI Eval (§21) depends on it — keep one request path for both, so the eval measures what the AI tab actually sends.
- **Refusals (v1.10.3).** The API's safety classifiers can decline a request. That arrives as HTTP 200 with `stop_reason: "refusal"`, not as an error, so check `stop_reason`, not the content. `send()` then returns `refusal` (`{ category, explanation }` from `stop_details`) and drops any partial text. The AI tab shows `aiRefusalHtml()` — Claude declined, plus `explanation` when present (display it, never parse it) — and nothing goes to history. Never show a declined request as an answer.
- **Raw mode:** `callClaude(question, label, { raw:true, summary })` sends the identical request through the same inner `send()` but touches no DOM, busy flag or history; it resolves to `{ text, stopReason, truncated, refusal, usage, model }` or throws (`.status`/`.detail` on HTTP errors). The AI Eval (§21) depends on it — keep one request path for both, so the eval measures what the AI tab actually sends.
- If updating the Claude model string, update `AI_MODEL` in `views-ai.js` and mention it in CHANGELOG.md. Changing the model, the system prompt or `buildFinancialSummary()` changes what the eval measures — re-run it and replace `docs/EVAL.md`.

---
Expand Down Expand Up @@ -797,7 +798,7 @@ Multiple Chase card files can be loaded simultaneously. Each gets a unique `acco

---

*Last updated: v1.10.2 — 2026-10-25 (§13 summary states its dates; §21 per-month derivations; v1.10.1: §13 `AI_MAX_TOKENS`, thinking and cut-off answers; v1.10.0: §21 AI Eval added; §13 `callClaude()` raw mode; v1.9.6: §14 folder access lasts until the last tab closes, not one page load; v1.9.5: §7 chart guard; v1.9.4: views-db.js changelog reads CRLF checkouts; v1.9.3: §14 folder permission: re-request on click, Reconnect banner, browser copy wins; v1.9.2: §0 branch, PR and approval workflow added; v1.9.1: §4.5 rehydrate transactions loaded from JSON; v1.9.0: §20 Data & Reset added; v1.8.1: §10 re-upload dedup rewritten for the occurrence-suffix fix; v1.8.0: AI Analysis restored with API-key and output hardening; `renderMarkdown()` added; `samples/` added; §1–3, §13, §14, §17 corrected to match the code)*
*Last updated: v1.10.3 — 2026-10-26 (§13 refusals; §21 declined runs; v1.10.2: §13 summary states its dates; §21 per-month derivations; v1.10.1: §13 `AI_MAX_TOKENS`, thinking and cut-off answers; v1.10.0: §21 AI Eval added; §13 `callClaude()` raw mode; v1.9.6: §14 folder access lasts until the last tab closes, not one page load; v1.9.5: §7 chart guard; v1.9.4: views-db.js changelog reads CRLF checkouts; v1.9.3: §14 folder permission: re-request on click, Reconnect banner, browser copy wins; v1.9.2: §0 branch, PR and approval workflow added; v1.9.1: §4.5 rehydrate transactions loaded from JSON; v1.9.0: §20 Data & Reset added; v1.8.1: §10 re-upload dedup rewritten for the occurrence-suffix fix; v1.8.0: AI Analysis restored with API-key and output hardening; `renderMarkdown()` added; `samples/` added; §1–3, §13, §14, §17 corrected to match the code)*

---

Expand Down Expand Up @@ -874,6 +875,7 @@ A developer-only view (`showView('ai-eval')`) that measures how well AI Analysis
- **Same parsing as an import.** `buildDataset()` mirrors `parseCsvFile()` (schema detection, account ids, occurrence suffixes) without touching IndexedDB. If `parseCsvFile()` changes, mirror it here; the sandbox summary must equal what a normal import of `samples/` produces.
- **Same request as the AI tab** — `callClaude()` raw mode (§13). Never build a second request path.
- **Each run records** `meta.maxTokens` and every call's `usage` (including `output_tokens_details.thinking_tokens`), and EVAL.md reports cut-off answers and thinking per answer. Runs from before v1.10.1 didn't record `maxTokens`, so it is inferred from the size the cut-off answers stopped at.
- **Declined runs** (`stop_reason` `refusal`) have no answer: they are counted in the *Calls* row, like failed calls, and left out of the scoring and token averages.
- **Nothing persisted.** Results live in memory only and leave via the download buttons, so §20 needs no entry. Don't add `lsSet()` calls here. The API key is never put in results, exports or logs.
- **A summary loaded from a results JSON is never sent.** It can be re-scored locally (`loadResults()` sets `dataset.fromFile`), but Run stays disabled until samples are reloaded and verified.
- **The summary format is parsed** by `parseSummary()`: the period line (`S.days`, `S.periodMonths`) and month lines with an optional `(through …)` / `(from …)` note. If `buildFinancialSummary()` changes, update the parser and its derivations *before* the next run. Example: the per-month figures over the period (total ÷ months as stated, or ÷ days/30.44) were added in v1.10.2 together with the dates.
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

> A zero-install, browser-native finance dashboard. Drop in your bank CSVs, get instant clarity on your money.

**Version:** v1.10.2  ·  **Stack:** Vanilla JS, HTML, CSS  ·  **Install:** None
**Version:** v1.10.3  ·  **Stack:** Vanilla JS, HTML, CSS  ·  **Install:** None

---

Expand Down Expand Up @@ -260,4 +260,4 @@ See [CHANGELOG.md](CHANGELOG.md) for the full release history.

---

*FinLens v1.10.2 — Built for one person's finances, designed to be useful for anyone's.*
*FinLens v1.10.3 — Built for one person's finances, designed to be useful for anyone's.*
2 changes: 1 addition & 1 deletion index.html
Original file line number Diff line number Diff line change
Expand Up @@ -326,7 +326,7 @@
<div style="margin-top:auto">
<a class="nav" data-view="settings">⚙️ Settings</a>
<div class="footer">
<div style="color:var(--text-3);font-size:11px">v1.10.2 · FinLens</div>
<div style="color:var(--text-3);font-size:11px">v1.10.3 · FinLens</div>
</div>
</div>
</aside>
Expand Down
19 changes: 16 additions & 3 deletions js/views-ai.js
Original file line number Diff line number Diff line change
Expand Up @@ -149,11 +149,20 @@ function aiAnswerHtml(text, truncated) {
return renderMarkdown(text)+(truncated?'<div class="insight warn" style="margin-top:12px"><strong>This answer was cut off.</strong> It reached the length limit, so the end is missing. Ask again, or ask a narrower question.</div>':'');
}

// stop_reason "refusal": the API's safety classifiers declined the request. It arrives as a
// normal HTTP 200, not an error, so check stop_reason rather than the content. There is no
// answer to show or keep. stop_details.explanation is display text (it may be null) — show
// it, don't parse it.
function aiRefusalHtml(refusal) {
const why=refusal&&refusal.explanation?`<div style="margin-top:6px">${escapeHtml(refusal.explanation)}</div>`:'';
return `<div class="insight warn"><strong>Claude declined to answer this question.</strong>${why}<div style="margin-top:6px">Try asking it a different way.</div></div>`;
}

// Every failure path ends in a visible message on the page — never a silent no-op.
//
// Raw mode — callClaude(question, label, { raw:true, summary }) — sends the same request
// but touches no DOM, no busy flag and no AI history. It resolves to
// { text, stopReason, truncated, usage, model } or throws (HTTP errors carry .status and .detail).
// { text, stopReason, truncated, refusal, usage, model } or throws (HTTP errors carry .status and .detail).
// The AI Eval view (views-eval.js) uses it so the eval measures exactly what this tab sends.
async function callClaude(question, label, opts={}) {
if (opts.raw) return send(opts.summary!=null?opts.summary:buildFinancialSummary());
Expand All @@ -179,7 +188,8 @@ async function callClaude(question, label, opts={}) {
}

try {
const { text, truncated }=await send(summary);
const { text, truncated, refusal }=await send(summary);
if (refusal) { respEl.innerHTML=aiRefusalHtml(refusal); return; } // no answer: nothing to copy or keep
respEl.innerHTML=aiAnswerHtml(text, truncated);
if (aiNoAnswer(text, truncated)) return; // nothing to copy or keep (finally clears aiBusy)
// A cut-off answer says so on screen, in copies and saved notes, and in history.
Expand Down Expand Up @@ -230,7 +240,10 @@ async function callClaude(question, label, opts={}) {
// Join all text blocks (don't assume the first content block is text)
const text=(data.content||[]).filter(b=>b.type==='text').map(b=>b.text).join('\n\n')||AI_NO_TEXT;
const stopReason=data.stop_reason||null;
return { text, stopReason, truncated:AI_CUT_OFF_REASONS.includes(stopReason), usage:data.usage||null, model:data.model||AI_MODEL };
// A refusal is not an answer: drop any partial text and return why, for display.
const sd=data.stop_details||{};
const refusal=stopReason==='refusal'?{ category:sd.category??null, explanation:sd.explanation??null }:null;
return { text:refusal?AI_NO_TEXT:text, stopReason, truncated:AI_CUT_OFF_REASONS.includes(stopReason), refusal, usage:data.usage||null, model:data.model||AI_MODEL };
} finally {
clearTimeout(timer);
}
Expand Down
14 changes: 9 additions & 5 deletions js/views-eval.js
Original file line number Diff line number Diff line change
Expand Up @@ -225,6 +225,7 @@ const AIEval = {
...job,
text: result ? result.text : null,
stopReason: result ? result.stopReason : null,
refusal: result ? result.refusal || null : null, // stop_details category/explanation when declined
usage: result ? result.usage : null,
model: result ? result.model : null,
ms: Math.round(performance.now() - t0),
Expand Down Expand Up @@ -879,7 +880,9 @@ const AIEval = {
score(runs, summary) {
const S = this.parseSummary(summary);
const forecastIdx = AI_PROMPTS.findIndex(([label]) => this.FORECAST_LABEL.test(label));
const rows = runs.filter(r => r.text != null).map(r => {
// A declined run (stop_reason "refusal") has no answer, so it isn't scored; it is counted
// separately, like a failed call.
const rows = runs.filter(r => r.text != null && r.stopReason !== 'refusal').map(r => {
const { ex, deps, figures } = this.classifyText(r.text, S, r.prompt === forecastIdx);
const strict = this.classifyText(r.text, S, r.prompt === forecastIdx, 'strict').figures;
figures.forEach((f, i) => {
Expand Down Expand Up @@ -911,6 +914,7 @@ const AIEval = {
overall: this._tally(all), byKind: { '$': this._tally(all.filter(f => f.kind === '$')), '%': this._tally(all.filter(f => f.kind === '%')) },
strict: strictAll, byPrompt, byRep, failures, tags,
runsScored: rows.length, runsFailed: runs.filter(r => r.text == null).length,
runsDeclined: runs.filter(r => r.text != null && r.stopReason === 'refusal').length,
truncated: runs.filter(r => AI_CUT_OFF_REASONS.includes(r.stopReason)).length,
tokens: this._tokenStats(runs),
calibration: this._calibrate(rows, S, forecastIdx),
Expand All @@ -922,7 +926,7 @@ const AIEval = {
// isn't shown). maxTokens: the run's own setting, or for runs before v1.10.1 (which didn't
// record it) the size every cut-off answer stopped at.
_tokenStats(runs) {
const ok = runs.filter(r => r.usage && r.usage.output_tokens != null);
const ok = runs.filter(r => r.usage && r.usage.output_tokens != null && r.stopReason !== 'refusal'); // answers only
const think = ok.filter(r => r.usage.output_tokens_details && r.usage.output_tokens_details.thinking_tokens != null);
const avg = xs => (xs.length ? Math.round(xs.reduce((a, b) => a + b, 0) / xs.length) : null);
const cut = ok.filter(r => AI_CUT_OFF_REASONS.includes(r.stopReason)).map(r => r.usage.output_tokens);
Expand Down Expand Up @@ -1057,7 +1061,7 @@ const AIEval = {
L.push(`| Model | \`${meta.model}\` (\`AI_MODEL\` in \`js/views-ai.js\`) |`);
const utc = iso => new Date(iso).toISOString().slice(0, 16).replace('T', ' '); // same text wherever it's re-scored
L.push(`| Run | ${utc(meta.startedAt)} – ${utc(meta.finishedAt).slice(11)} UTC, FinLens v${meta.appVersion} |`);
L.push(`| Calls | ${AI_PROMPTS.length} prompts × ${meta.repeats} repeats = ${AI_PROMPTS.length * meta.repeats} (${rep.runsScored} answered${rep.runsFailed ? `, ${rep.runsFailed} failed` : ''}${rep.truncated ? `, ${rep.truncated} cut off at max_tokens` : ''}); \`max_tokens\` ${rep.tokens.maxTokens || 'not recorded'}, default temperature |`);
L.push(`| Calls | ${AI_PROMPTS.length} prompts × ${meta.repeats} repeats = ${AI_PROMPTS.length * meta.repeats} (${rep.runsScored} answered${rep.runsFailed ? `, ${rep.runsFailed} failed` : ''}${rep.runsDeclined ? `, ${rep.runsDeclined} declined (\`stop_reason\` refusal, not scored)` : ''}${rep.truncated ? `, ${rep.truncated} cut off at max_tokens` : ''}); \`max_tokens\` ${rep.tokens.maxTokens || 'not recorded'}, default temperature |`);
if (rep.tokens.output != null) L.push(`| Tokens | ${rep.tokens.output} output tokens per answer on average${rep.tokens.thinking != null ? `, of which ${rep.tokens.thinking} thinking (not shown, but counted toward \`max_tokens\`)${rep.tokens.thinkingCut != null ? `; ${rep.tokens.thinkingCut} in the answers that were cut off` : ''}` : ''} |`);
L.push(`| Data | \`samples/\` only — synthetic Ally Checking, Ally Savings and Chase CSVs (${ds.txCount} transactions, ${dayjs(ds.minDate).format('MMM D')} – ${dayjs(ds.maxDate).format('MMM D, YYYY')}), date range **All Time** |`);
L.push(`| Files | ${(this.samples || []).map(s => `\`${s.name}\` sha256 \`${s.sha256.slice(0, 12)}…\``).join(', ')} |`);
Expand Down Expand Up @@ -1224,7 +1228,7 @@ registerView('ai-eval', 'AI Eval', (root) => {
</table></div>
<div class="footnote">Repeats: ${rep.byRep.map(t => `#${t.rep} grounded ${t.pct('grounded').toFixed(1)}% · derived ${t.pct('derived').toFixed(1)}% · unsupported ${t.pct('unsupported').toFixed(1)}%`).join(' &nbsp;|&nbsp; ')}<br>
Strict ±$1 / ±0.5 pts (no rounding allowance): grounded ${rep.strict.pct('grounded').toFixed(1)}% · unsupported ${rep.strict.pct('unsupported').toFixed(1)}%.
${rep.runsFailed ? ` · <span style="color:var(--red)">${rep.runsFailed} calls failed</span>` : ''}${rep.truncated ? ` · ${rep.truncated} answers cut off at max_tokens` : ''}${rep.tokens.thinking != null ? ` · thinking ${rep.tokens.thinking} of ${rep.tokens.output} output tokens per answer` : ''}</div>`;
${rep.runsFailed ? ` · <span style="color:var(--red)">${rep.runsFailed} calls failed</span>` : ''}${rep.runsDeclined ? ` · ${rep.runsDeclined} declined (not scored)` : ''}${rep.truncated ? ` · ${rep.truncated} answers cut off at max_tokens` : ''}${rep.tokens.thinking != null ? ` · thinking ${rep.tokens.thinking} of ${rep.tokens.output} output tokens per answer` : ''}</div>`;
root.appendChild(tbl);

const fail = document.createElement('div'); fail.className = 'card';
Expand All @@ -1239,7 +1243,7 @@ registerView('ai-eval', 'AI Eval', (root) => {
const answers = document.createElement('div'); answers.className = 'card';
answers.innerHTML = `<h2 class="section-title">Answers</h2>` + E.runs.map((r, i) => `
<details style="margin-top:8px;border-top:1px solid var(--border);padding-top:8px">
<summary style="cursor:pointer;font-size:12.5px">${escapeHtml(r.label)} — repeat ${r.rep} ${r.error ? `<span class="pill bad">${escapeHtml(r.error)}</span>` : `<span class="muted">${(r.ms / 1000).toFixed(1)}s${AI_CUT_OFF_REASONS.includes(r.stopReason) ? ' · cut off' : ''}</span>`}</summary>
<summary style="cursor:pointer;font-size:12.5px">${escapeHtml(r.label)} — repeat ${r.rep} ${r.error ? `<span class="pill bad">${escapeHtml(r.error)}</span>` : `<span class="muted">${(r.ms / 1000).toFixed(1)}s${AI_CUT_OFF_REASONS.includes(r.stopReason) ? ' · cut off' : ''}${r.stopReason === 'refusal' ? ' · declined' : ''}</span>`}</summary>
<div class="eval-answer" data-run="${i}" style="font-size:13px;margin-top:8px"></div>
</details>`).join('');
root.appendChild(answers);
Expand Down
2 changes: 1 addition & 1 deletion js/views-settings.js
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ function _renderGeneralSettings(root) {
</select>
</div>
</div>
<div class="muted" style="font-size:11.5px;margin-top:14px">FinLens v1.10.2 ·${State.transactions.length} transactions loaded${State.lastRefresh?' · Refreshed '+dayjs(State.lastRefresh).format('MMM D, YYYY h:mm A'):''}</div>`;
<div class="muted" style="font-size:11.5px;margin-top:14px">FinLens v1.10.3 ·${State.transactions.length} transactions loaded${State.lastRefresh?' · Refreshed '+dayjs(State.lastRefresh).format('MMM D, YYYY h:mm A'):''}</div>`;
root.appendChild(displayCard);

/* ---- Anthropic API Key ---- */
Expand Down