Repository navigation
v1.10.2: AI summary states its dates; stop the model guessing the period length - #13
Merged
Merged
Conversation
The summary said only "Period: All time", so in the v1.10.0 eval the model guessed how many months the totals covered (6, 6-7, 6.5, 9) and gave different monthly figures for the same data (groceries ~$470 vs ~$516/month). - summarySpan(): "Period: All time (transactions from May 1, 2026 to Oct 20, 2026: 173 days, about 5.7 months)" - partialMonthNote(): "2026-10 (through Oct 20)" in the monthly trend - AI Eval parses the new format and checks per-month figures over the period (total / stated months or / days/30.44), added before any run with the new summary; v1.10.0 re-scores unchanged Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WuftAn7qz6C2y8AQp2EkWq
This was referenced Oct 26, 2026
…on to v1.10.2 #12 and #13 now merge before #10 (Luis, 2026-10-26), so #12 is v1.10.1 and this fix is v1.10.2; #10 keeps v1.11.0. Conflicts were only the version strings. Every v1.11.2 is now v1.10.2 and every v1.11.1 is v1.10.1 (CHANGELOG, index.html, README, views-settings.js, CLAUDE.md, and comments in views-ai.js and views-eval.js). No code change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QiRz5pTW83oqNbk2NC7yMR
gitluis
pushed a commit
that referenced
this pull request
Oct 26, 2026
Luis decided on 2026-10-26 that #12 and #13 merge before #10. Master is at v1.10.0, so this bug fix is the patch release v1.10.1 (CLAUDE.md §17); #10 keeps v1.11.0. Renumbered in CHANGELOG, index.html, README, views-settings.js, CLAUDE.md §1/§13/§21/last-updated, and the views-eval.js comment on runs that didn't record maxTokens. No code change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QiRz5pTW83oqNbk2NC7yMR
…-summary-date-range
2 tasks
…ary-date-range # Conflicts: # CLAUDE.md
gitluis
pushed a commit
that referenced
this pull request
Oct 26, 2026
…ve v1.10.1 Luis ran the AI Eval on this branch (v1.10.2) on 2026-10-26. Same model, prompts, samples and max_tokens as the v1.10.1 run; the summary now states its dates (exactly the two-line difference this PR makes). - Every monthly groceries figure is $540-545 ($3,098 / 5.7 months); v1.10.1 said ~$470, ~$515, ~$516. No answer guesses the period. - Traceable 66.9% -> 72.6%, unsupported 29.2% -> 23.3%, 0 cut off; chance-match rate 11.9% -> 13.4% (new per-month derivations) - Unsupported "claims about the data" 46 -> 56, read by hand: none divides by a guessed month count; ~32 correct arithmetic the matcher doesn't model, 9 projections (October is now partial), 15 mis-tagged advice or benchmarks - docs/EVAL.md generated from the new results JSON plus a three-run Runs table; v1.10.1 report archived in docs/eval-runs/v1.10.1/ (relative links fixed); README numbers, CLAUDE.md §21, CHANGELOG 1.10.2 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QiRz5pTW83oqNbk2NC7yMR
gitluis
pushed a commit
that referenced
this pull request
Oct 26, 2026
gitluis
marked this pull request as ready for review
October 26, 2026 12:59
2 tasks
Owner
Author
|
Approved by Luis in chat on 2026-10-26. His words:
Recorded here before merging, per CLAUDE.md §0. Merging next as a merge commit. 🤖 Generated with Claude Code Generated by Claude Code |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The summary FinLens sends to Claude now states the dates it covers, and marks a partial month. The model no longer has to guess how long "All time" is.
Cause
The summary said only
Period: All time. In the v1.10.0 eval (#6), every answer that gave a monthly figure had to guess how many months the totals covered, and the guesses varied:So the same data gave different numbers: groceries were ~$470/month in one answer and ~$516 in another.
Related: the monthly trend listed October as if it were complete, but the data stops on Oct 20.
What changed
js/views-ai.jssummarySpan(): the period line now readsPeriod: All time (transactions from May 1, 2026 to Oct 20, 2026: 173 days, about 5.7 months). It works for every date range, and spans under 45 days give days only.partialMonthNote(): a month the data only partly covers is marked, e.g.2026-10 (through Oct 20)or(from May 15).js/views-eval.jsdocs/EVAL.md— the eval re-run on this branch (below), with the Runs table extended to all three runs. The v1.10.1 report moved todocs/eval-runs/v1.10.1/. README's Evaluation numbers are updated.index.html, README andviews-settings.js. CLAUDE.md §13 now says "never send totals without their dates"; §21 says to keep the parser in step with the summary format.Summary diff for the sample data (exactly two lines):
How it was tested
Headless Chromium, mocked API, on this branch (#6 + #12 + this change):
Summary: a normal import of
samples/produces the new lines. The eval's sandbox summary still equals the real import path byte for byte.Span and partial months: checked on synthetic data, including data starting May 15, a 20-day range, a single day and no data.
Parser: reads
days = 173,months = 5.7, and October's "(through Oct 20)". The 30 summary facts are unchanged.New checks:
Regression:
file://picker; tampered/BOM/CRLF samples.All pass.
Re-tested on
0163c12: 30/30 headless checks pass, including the dated summary, every version string saying v1.10.2, and re-scoring all three committed runs. The committed EVAL.md is exactly what the view generates fromdocs/eval-results.json, plus the hand-written sections. Re-scoring the v1.10.1 answers with this PR's matcher gives exactly the v1.10.1 numbers: the new per-month rule applies only when the summary states its period.Eval re-run
Run by Luis on 2026-10-26, 12:44–12:50 UTC, on this branch (v1.10.2). Same model, prompts, samples and
max_tokensas #12's run; the summary differs by exactly the two lines above. No errors or retries.Full results:
docs/EVAL.md.Please check before merging
Opened in https://claude.ai/code/session_01WuftAn7qz6C2y8AQp2EkWq; re-versioned in the session below.
https://claude.ai/code/session_01QiRz5pTW83oqNbk2NC7yMR
Generated by Claude Code