gates: three scanner defects, and the assert column measures the control (Refs #2161) - #2687
Merged
Conversation
…rol (Refs #2161) A five-way fan-out audit raised 19 findings; 11 survived adversarial refutation, 8 were killed. SCANNER (cli/tri/src/gates.rs), three defects, each with a test that fails when its fix is removed: * The three LINE-oriented site finders had no string state, so a flush-left line inside a control function's docstring cleared `in_control` and handed the operators the control's own `assert` -- a site nobody can kill, because neutering a control makes it pass. Two files differing by four spaces of indentation on ONE docstring line scored 3 assert sites versus 0. * The same blindness scored word-wrapped prose beginning a line with `assert` as an assertion to neuter. * `assert_sites`' message splitter advanced one byte on a backslash instead of two, so an escaped quote either produced a mutant that does not PARSE (python exits 1, the site scores KILLED -- a false green in the column this command exists to make trustworthy) or stranded the quote state and silently dropped the message. `boundary_sites` had `i += 2` all along. CONTROL (tools/gft_backprop_microcode.py): the second plant did not test what it says. `input [31:0] x0i` spelled out occurs exactly once in the file -- at the assertion that checks for it -- because the emitter builds ports with an f-string. The plant rewrote the assertion's own expected string and the emitted Verilog was untouched. It now targets the emitter's format string. STALE NUMBERS of mine, corrected: * boundary denominators of the form k/31 came from the scanner bug; the file has 62 sites. Killed count is unchanged at 14. Errata added. * `tri pointers`' docstring froze "193 mentions and 16 unresolved"; the tool printed 191 and 12 within a day. The counts are removed rather than updated -- a number in a docstring cannot learn it went stale. * `--boundary`'s help advertised "the worst kill rate of the five (26 of 77 across the tree)", whose denominator this same fix doubled. Also: external/README.md and external/OWNERS.md pointed at a canon path that moved. Owner issues filed: #2685 (a live PRL draft claims zero admitted lemmas; proofs/ has 32, and the cited audit trail never existed) and #2686 (Gate Topology audits 15 of 45 workflows and misses two live violators; coq-proofs.yml has never been green in 50 runs).
Contributor
PR DashboardGenerated at: 2026-08-24 08:49:38 UTC
Summary
Seal Status
|
Contributor
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A five-way fan-out audit raised 19 findings; 11 survived adversarial refutation, 8 were killed. This lands the ones that are mine to fix.
The headline is not a fix — it is what a number turned out to mean
--assertscores 2 of 34, the worst of the five operators, deferred three iterations as "the column to fix".It is not a column about assertions. It is arithmetic on the number of planted faults.
self_check()spawns three whole-program runs: one clean, two with a fault planted. A neutered assertion changes nothing on the clean run, so it can only be noticed on a run where that assertion would have fired — and only if it fires first, because Python stops there.Measured, one variable at a time:
One kill per plant. The kill set is exactly
{632, 668}— precisely the two lines missing from the survivor list. All 34 sites do execute on a clean run, so no survivor is unreachable; they are invisible, not dead. Iterating on the operator cannot move the number — only adding aspawned()case per assertion can.My published explanation was wrong for at least 26 of the 32. The earlier NOW entry accounts for the six held-out-threshold assertions and offers a boundary rationale for the whole column. A partial explanation presented as a complete one is the most expensive kind of wrong: it closes the question, and three iterations then deferred the column as understood rather than unexplained.
The control's second plant did not test what it says
The case is "a renamed port is caught by the emitter check".
input [31:0] x0ispelled out occurs exactly once in the file — at the assertion checking for it — because the emitter builds ports asf"input [31:0] x{k}i".So the plant's
str.replacerewrote the assertion's own expected string; the emitted Verilog was untouched, and the assertion failed for a reason unrelated to the emitter.T124 fixed the case where a needle's first occurrence was the control's own source. This is the next one along: the needle's only occurrence is the subject's check rather than the subject. It now targets the emitter's format string, whose first occurrence is above the control. Verified: the plant edits line 336, the assertion at 668 is untouched, and the control still passes.
Three scanner defects, each with a test that fails without its fix
in_controlassertas an assertionassert anything.in a module docstring became a siteassert_sitesadvanced one byte on a backslash instead of twoassert s == "a\",b", "msg"→ mutantassert True,b", "msg", which does not parse → python exits 1 → site scored KILLEDThe third is the dangerous one: a false green in the exact column this command exists to make trustworthy.
boundary_sites, byte-oriented, hadi += 2all along.Neutering a control makes it pass, so a site inside one is recorded as a survivor nobody can ever kill — which is what
leaves_function's own doc comment warns is the worst outcome: breaking the instrument instead of the thing being measured.Three of my own numbers were stale
k/31boundary denominatorsdef self_check(). Killed count unchanged at 14; errata addedtri pointersdocstring--boundaryhelp textThe frozen counts are removed, not updated. A number in help text has no way to learn it went stale, and the pointers docstring had reached the point where running the file refuted its own documentation — the failure the tool exists to find, one level up.
Filed rather than fixed
docs/reports/PRL_DRAFT_OUTLINE.md(a live manuscript outline) claims "Zero admitted lemmas (audited W211 — seedocs/COQ_STATUS.md)".proofs/contains 32Admitted., and that audit trail has no commit in the whole history. Changing a physics paper's factual claims is the author's call.Gate Topologyhas run 433 times, never red, and audits 15 of 45 workflows; the two that today carry the exact filter it forbids are in neither its covered nor its excluded list. Andcoq-proofs.ymlis 0 successes in 50 runs, still wired to push and pull_request.Also:
external/README.mdandexternal/OWNERS.mdpointed at a canon path that moved.tri pointersdead count 12 → 10.§123 and errata in §§ earlier. 206 tests pass.
Refs #2161