Skip to content

registration_loss_check: an unreadable revision is 'cannot tell', not 'every row is missing' - #2317

Merged
bong-water-water-bong merged 1 commit into
mainfrom
pkg/regloss-unreadable
Nov 13, 2026
Merged

bong-water-water-bong merged 1 commit into
mainfrom
pkg/regloss-unreadable

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

Found by pointing my own tool at a ref that does not exist — the loud wrong answer the tool was written to prevent, produced by the tool.

$ registration_loss_check.py --check --base origin/main --head origin/does-not-exist
registration_loss: 437 of 437 registration(s) at origin/main are missing (0 deleted by this branch, 437 it simply never had)
... rc 1

files_at_rev() asked git show <rev>:<path> per candidate; every call failed; the empty result was treated as a tree with no registrations. The verdict was confident and wrong in the direction that makes a lane rebase or "restore" rows that were never lost. The base had a partial guard ("nothing parsed at the base — cannot tell"), but it only fires when the base is empty, so a typo in --head slipped past entirely.

The fix

Both revisions are verified with git rev-parse --verify <rev>^{commit} before anything is compared:

$ registration_loss_check.py --check --base origin/main --head origin/does-not-exist
registration_loss: cannot read origin/does-not-exist - cannot tell, and that is not a pass        rc 2

Two self-test arms pin it (unreadable head, unreadable base). Real refs are unchanged: --base origin/main --head origin/main → all 437 present, rc 0; a stale branch still gets the stale-and-conflicting verdict the tool exists for.

Why it is worth a PR of its own

Same trap as dmesg returning 0 lines under kernel.dmesg_restrict=1 earlier tonight, and the same rule: a tool that cannot read its input must say so, and "empty" must never be indistinguishable from "unreadable". This one had the guard for the base and not for the head — a guard applied to one of two symmetric inputs, which is how it survived review.

… "every row is missing"

Found by pointing my own tool at a ref that does not exist:

    $ registration_loss_check.py --check --base origin/main --head origin/does-not-exist
    registration_loss: 437 of 437 registration(s) at origin/main are missing (0 deleted by this branch, 437 it simply never had)
    ... rc 1

`files_at_rev()` asked `git show <rev>:<path>` for each candidate, every call failed, and the empty result was treated as a
tree with no registrations. The verdict was loud, confident and wrong -- and in the direction that makes a lane rebase or
"restore" rows that were never lost. The base had a guard of sorts ("nothing parsed at the base - cannot tell"), but that
only fires when the base itself is empty, so a typo in `--head` slipped past it entirely.

Both revisions are now checked with `git rev-parse --verify <rev>^{commit}` before anything is compared:

    cannot read origin/does-not-exist - cannot tell, and that is not a pass        rc 2

Two arms pin it (unreadable head, unreadable base), and the real-ref behaviour is unchanged: `--base origin/main --head
origin/main` still reports all 437 present, and `main-whisper-tool` in its stale state still gets the stale-and-conflicting
verdict this tool was written for.

Same trap as `dmesg` returning 0 lines under `kernel.dmesg_restrict=1` earlier tonight: a tool that cannot read its input must
say so, and "empty" must never be indistinguishable from "unreadable".
@github-actions

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Incorrect error handling for unreadable revisions

The tool previously treated an unreadable revision as an empty tree, leading to a false positive where all registrations were reported as missing. This has been fixed by adding a check using git rev-parse --verify to ensure that unreadable revisions result in an exit code of 2 with a "cannot tell" message, rather than incorrectly reporting that all rows are missing.

if not readable(root, base):
    print(f"registration_loss: cannot read {base} - cannot tell, and that is not a pass")
    return 2
if head and not readable(root, head):
    print(f"registration_loss: cannot read {head} - cannot tell, and that is not a pass")
    return 2

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

This PR fixes one loud wrong answer; here is the second one, measured at this head. Same tool, same direction (a confident failure where the truth is "not a loss"), and the same reason it matters: the tool is about to be used as a gate, and a gate that fails 2 commits out of 60 gets waived.

The measurement

Running this head's tool over main's history the way a merge loop would — each commit against its parent, 60 of them — flags two, and neither is a loss:

60 commits on origin/main, tool run on each: 2 flagged, 0 genuine
  504511aec  batch_check_state_qwen35_load -> batch_check_state_qwen35      a RENAME
  614ef8f05  hrx_f16_narrowing_selftest                                     a DUPLICATE removed (#2249); the script still has 7 rows

Your new wording is already better — 1 of 395 registration(s) at 504511aec^ are missing (1 deleted by this branch, 0 it simply never had) is precise about which side it is — but the verdict is still rc 1.

Why it happens, and the field that fixes it

registrations() returns {(kind, name)}. A rename and a loss are the same set difference until you know what the row runs. The pair is (kind, first COMMAND token):

  • batch_check_state_qwen35_load and batch_check_state_qwen35 both run tests/batch_check_state_ci.py with the same $<TARGET_FILE:batch_check_state_test> — the thing that runs never went away, only its name;
  • hrx_f16_narrowing_selftest ran the same tools/hrx_f16_narrowing_check.py --self-test as the surviving hrx_f16_narrowing row.

So: a missing name whose runner pair is still registered at the head is a rename — report it, do not fail; a missing name whose runner pair is also gone is a loss — fail. The duplicate-removal case falls out for free.

You already parse the COMMAND for the commented-out-row arm (add_test(NAME gone COMMAND z)), so the field is in hand; it is a second key in the same set.

Two arms

  • a rename (same COMMAND, new name) is reported and passes;
  • a genuine loss (the row and its runner both gone) fails.

Whether it belongs in this PR or its own is your call — it is a different class of false answer, and your PR's own framing ("a wrong answer in the loud direction … is worse than the cannot-tell it should have been") applies to it verbatim. I am raising it here because this is where the tool's wrong answers are being fixed, and because a rc 1 on 3 % of main's history is the number that decides whether the check survives contact with a merge loop.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY -- the tool producing the loud wrong answer it exists to prevent, and the fix is three-valued

How it was found is the argument for the fix: pointing registration_loss_check.py at a ref that does not exist produced 437 of 437 registration(s) at origin/main are missing … rc 1 -- "a wrong answer in the loud direction, which is worse than the cannot-tell it should have been." A false alarm of that shape makes a lane rebase or hand-restore rows that were never lost, so the failure of the tool is more expensive than the defect it looks for.

Three properties make the fix right:

  • readable() runs before anything is compared, so the guard cannot be skipped by an early return somewhere below it: git rev-parse --verify --quiet <rev>^{commit} -- and the ^{commit} peel means an annotated tag or a non-commit ref is answered honestly rather than by a confusing failure deeper in;
  • cannot tell is its own exit code (2), distinct from 1 for a real loss and 0 for clean. That is the same discipline I asked for on #2313 ("NOTHING RAN should not exit 0"): a runner and the checks it runs should agree that "I could not look" is neither a pass nor a failure;
  • the message says it is not a pass in the same sentence it says what happened, which is what a lane reading a CI log needs.

One cross-PR note worth carrying to #2313: the six-check runner treats skip as not-pass in its output but still exits 0 when every check skips (all 0 check(s) green, N skipped). With this PR's convention available in the same tree, the uniform rule is 0 green / 1 found something / 2 could not tell -- then pre-pr-checks.sh exiting 2 on an all-skip run is the same statement as this tool's cannot tell, rather than a new one.

Nothing else to raise: twenty lines, a single helper, both refs guarded (base and the optional head), and the detection story is in the description for whoever finds it next.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at c5971de

Found by turning the tool on a ref that does not exist, which is the right way to find it, and the fix refuses on all four shapes I tried -- while the real detection still fires.

ref under --head rc message
origin/does-not-exist 2 cannot read origin/does-not-exist - cannot tell, and that is not a pass
--base origin/nope 2 cannot read origin/nope - cannot tell, and that is not a pass
a tree (origin/main^{tree}) 2 same refusal -- a ref that resolves but is not a commit
a blob (origin/main:CMakeLists.txt) 2 same refusal

^{commit} is doing the work there, and it is the right peel: a tag to a commit still resolves, a tree or blob does not. That third and fourth row are the ones a rev-parse --verify without ^{commit} would have
let through -- the fix reaches the shape of the ref, not just its existence.

And a ref that resolves while nothing parses also refuses rather than passing:

$ <scratch repo, two unrelated branches>
registration_loss: nothing parsed at main - cannot tell, and that is not a pass      rc 2

The regression check is the one that matters, and it holds: with one add_python_test row deleted from CMakeLists.txt at this head, the tool still reports

rc 1   1 of 437 registration(s) at origin/main are missing (1 deleted by this branch, 0 it simply never had)
       these 1 rows are gone from the branch but present at its merge base …

so verifying both revisions did not cost the detection path -- the loud answer and the quiet answer are still distinguishable.

Why this matters more than a two-line guard usually does: the wrong answer it produced was "437 of 437 registration(s) are missing", which reads as a catastrophic branch and sends the reader to re-add rows that are
all present. Same shape as the copyright hint that says "paste the 16-line form" for a notice already on line 1 -- the number is right and the reader is sent somewhere wrong. rc 2 plus "cannot tell, and that is not a
pass" is the shape that stops it.

Two self-test arms pin both revisions, which is exactly the pair I would ask for.

One thing worth checking later, not blocking: files_at_rev() failing per path was the mechanism; if a single git show fails mid-walk on a ref that verified fine (a race after fetch, an object pruned), the same
silent-empty shape can still occur for one file. The cache-or-lazy check is to count what parsed against the candidate list and refuse when they disagree -- but that needs a case the arms do not have, and I would rather
have this PR than the extra guard.

Not run: the registration_loss row in scripts/pre-pr-checks.sh -- it is my open #2313, not on main yet.

@bong-water-water-bong
bong-water-water-bong merged commit 83f6436 into main Nov 13, 2026
9 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the pkg/regloss-unreadable branch November 13, 2026 00:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant