tri gates mutate: refute the equivalence claims instead of quoting them (Refs #2161) - #2656
Merged
Conversation
…em (Refs #2161) `# mutant-equivalent: <why>` marks a survivor as unkillable by construction, and this command printed it as "claims equivalent: ...". The word `claims` was carrying real weight: nothing had ever checked one. Six sit in tools/. The run best placed to notice a stale claim is this one -- it already built the mutant and already knows the verdict. A claimed line whose mutant DIES is now reported as contradicted. Measured against the six that already existed: none contradicted. Counts, not membership: a line can hold two mutable sites (`if a < 1 or b < 1:`), and a claimed line with one of two dead has been contradicted once. Keying on "did the line leave the survivor list" would call that claim intact. Claims name no operator, and every one in the tree argues about a comparison. A line equivalent under `boundary` can die under `invert`, so the report names the direction rather than pretending to judge. `contradicted_claims()` is separate from the run loop and has four tests: survived, died, half-died, and claimed-but-not-mutated-in-this-direction. A checker nobody can test is the same failure one level up.
# Conflicts: # .claude/skills/ci-gates/SKILL.md
…em (Refs #2161) `# mutant-equivalent: <why>` marks a survivor as unkillable by construction, and this command printed it as "claims equivalent: ...". The word `claims` was carrying real weight: nothing had ever checked one. Six sit in tools/. The run best placed to notice a stale claim is this one -- it already built the mutant and already knows the verdict. A claimed line whose mutant DIES is now reported as contradicted. Measured against all six that already existed: none contradicted. Counts, not membership: a line can hold two mutable sites (`if a < 1 or b < 1:`), and a claimed line with one of two dead has been contradicted once. Keying on "did the line leave the survivor list" would call that claim intact. The marker must now OPEN the comment rather than appear anywhere in it. Prose describing the mechanism -- "that reasoning sits on the line as a `# mutant-equivalent:` claim" -- registered as a claim of its own, bound to whatever code line followed, which the new check would have reported as contradicted the day that unrelated line's mutant died. Caught because the count printed 2 where one claim had been written. Every real claim in the tree already opens its comment this way. `contradicted_claims()` is separate from the run loop; six tests cover survived, died, half-died, claimed-but-not-mutated-in-this-direction, mention-is-not-a-claim, and marker-opens-the-comment. First claim written under the check: gft_backprop_microcode.py's `if d >= 26`, proven equivalent over 525_918 points.
Contributor
PR DashboardGenerated at: 2026-08-24 03:28:06 UTC
Summary
Seal Status
|
Contributor
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
# mutant-equivalent: <why>marks a survivor as unkillable by construction, andtri gates mutateprinted it back as "claims equivalent: …".The word
claimswas carrying real weight: nothing had ever checked one. Six sit intools/.A claim is a statement about the code. It ages with the code. And the run best placed to notice it has gone stale is the mutation run itself — it already built the mutant and already knows the verdict. The missing step was one comparison.
Measured against all six that already existed: none contradicted. That is worth exactly as much as the run that could have refuted them and did not, and nothing more.
Counts, not membership
A line can hold more than one mutable site —
if a < 1 or b < 1:holds two — so a claimed line with one of two dead has been contradicted once. Keying on "did this line leave the survivor list" would call that claim intact, and a half-true claim is the hardest kind to catch by eye.Writing about the marker created a claim
The extractor matched
mutant-equivalent:anywhere inside any comment. So this sentence, written in this very PR —— registered as a claim of its own, bound to whatever code line happened to follow.
Caught because the count printed 2 where I had written 1. Nothing else would have caught it: a claim nobody made, on a line it says nothing about, silently waiting to be reported as contradicted the day that unrelated line's mutant died. A false positive in the extractor becomes a false refutation in the checker — the new check gave the old bug a way to lie.
The marker must now open the comment. Every real claim in the tree already did.
The general shape: a scanner that matches a marker anywhere cannot tell a use from a mention, and documentation is exactly where mentions live. Any tool that greps the tree for its own vocabulary will eventually read its own documentation as data — and the count is the cheapest place to notice, which is an argument for printing counts nobody asked for.
The positive control, and how it nearly did not run
Planting a false claim over a line whose mutant is known to die must produce a contradiction. My first attempt printed nothing, and I read that as "the check does not fire".
It fired at nothing. Planting made
tools/dirty,mutaterefuses to start on a dirty tree, and I was reading its output throughgrep -i CONTRADICTED— a filter that cannot show a refusal. The verdict and the error went to the same stream and I had closed the half that was speaking.Satisfying the guard rather than bypassing it, it fired exactly once, naming the line and the claim.
Tests
contradicted_claims()is separate from the run loop — a checker nobody can test is the same failure one level up. Six cases: survived, died, half-died, claimed-but-not-mutated-in-this-direction, mention-is-not-a-claim, marker-opens-the-comment.First claim written under the check:
gft_backprop_microcode.py'sif d >= 26, proven equivalent over 525,918 points in #2653.Refs #2161