Strengthen data_ends_where_expected check to scan all columns after the first - #88
Merged
Merged
Conversation
…he first Previously the check only inspected the second column of the table range in the row after end_row. This missed truncated end_row configs where the second column happens to be blank in the dropped rows (e.g. non-DUID generators on the Marginal Loss Factors sheet in workbook 7.8). The check now scans every column from the second to the last of the table range; the first column remains excluded because notes legitimately appear below it. A sweep of all 1,397 tables across workbook versions 6.0-7.8 showed the stricter check fires only on genuine truncations, with one known data artifact: a stray =SUM() total cell below energy_efficiency_residential_green_energy_exports in 6.0, now handled with a skip_checks entry. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests.
🚀 New features to boost your workflow:
|
dylanjmcconnell
approved these changes
Aug 19, 2026
dylanjmcconnell
left a comment
dylanjmcconnell
left a comment
Member
There was a problem hiding this comment.
Looks all good to me!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
data_ends_where_expectedcheck previously inspected only the second column of the table range in the row afterend_row. This missed truncatedend_rowconfigs whenever the second column happened to be blank in the dropped rows — exactly what happened withmarginal_loss_factors_existing_generatorsin the 7.8 workbook (PR #86), where the new non-DUID generator rows have a blank second column, so 84 generators were silently dropped with green tests.The check now scans every column from the second to the last of the table range at
end_row + 1. The first column remains excluded because notes legitimately appear below it.Validation
An empirical sweep of all 1,397 tables across workbook versions 6.0, 7.0, 7.3, 7.5 and 7.8 (branch
v7.8) compared the old and new behaviour. The stricter check fires exactly twice:marginal_loss_factors_existing_generators(7.8) — the genuine truncation, previously undetected (fix suggested on Config for v7.8 of IASR worksheet #86)energy_efficiency_residential_green_energy_exports(6.0) — a lone stray=SUM(H30:H34)total cell in H35 below the table, i.e. a known workbook data artifact, now handled with a commentedskip_checksentryZero other fires across all versions, so no false positives on any currently-correct config. Formula cells intentionally count as data: in the MLF case, some of the cells that betray the truncation are themselves XLOOKUP formulas.
Also renames the check's
rangeparameter tocolumn_range(it shadowed the builtin, and matches the config field name).🤖 Generated with Claude Code