sweep every checker, and say what the step actually proves (Refs #2161) - #2598
Merged
Conversation
`tri gate-sweep` selected files matching `check_*` or `*gate*` -- so it measured the naming convention. Thirty checkers live in tools/; thirteen had never been swept, including every verify_*, the fuzzer and the generators. It now takes every non-private .py in tools/ and lets the reader judge what belongs, which is the same call this campaign made for gate selection: choose by property, not by what someone called the file. The thirteen contained five correct skips (all fatal under --require, checked -- a skip that is not is a pass wearing another word), one generator raising FileNotFoundError on the catalog it generates from, and one claim. CRASH: 0 across all thirty. THE CLAIM. emit-bitexact-gate.yml ran a step called "Prove the trainer LEARNS (XOR 4/4 + nonlinear held-out >=90%, incl. deep 3-layer)" The tool it runs is pure Python. No t27c, no .t27 spec, no iverilog. It generates backprop microcode, runs it on a bit-faithful GF-T INTERPRETER, and asserts on the TEXT of the Verilog it emits. It passes unchanged in a directory containing nothing but itself, which is how the scope was noticed and is not something reading the step could reveal. The tool's own docstring was honest and precise. The overclaim lived entirely in the step name -- and the step name is the claim most people read. Nobody opens the tool; they read a green check and a sentence, and the sentence said the trainer learns, which a reader takes for the compiler, the RTL, or the board. Renamed to what it proves, with the reason above it, and a "what this does not establish" block added to the tool naming the three claims it is NOT: that the compiler produces this microcode, that the emitted Verilog simulates, that any of it works on silicon. Refs #2161
Contributor
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
Contributor
PR DashboardGenerated at: 2026-08-23 19:58:28 UTC
Summary
Seal Status
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
tri gate-sweepselected files matchingcheck_*or*gate*— so it measured the naming convention. Thirty checkers live intools/; thirteen had never been swept, including everyverify_*, the fuzzer and the generators. It now takes every non-private.pyand lets the reader judge what belongs — the same call this campaign made for gate selection: choose by property, not by what someone called the file.CRASH: 0 across all thirty.
The thirteen contained five correct skips (all fatal under
--require— checked, because a skip that is not is a pass wearing another word), one generator raisingFileNotFoundErroron the catalog it reads, and one claim.The claim
The workflow ran a step called:
The tool is pure Python. No
t27c, no.t27spec, noiverilog. It generates backprop microcode, runs it on a bit-faithful GF-T interpreter, and asserts on the text of the Verilog it emits. It passes unchanged in a directory containing nothing but itself — which is how the scope was noticed, and is not something reading the step could reveal.The tool's own docstring was honest and precise. The overclaim lived entirely in the step name — and the step name is the claim most people read. Nobody opens the tool; they read a green check and a sentence, and the sentence said the trainer learns, which a reader takes for the compiler, the RTL, or the board.
Renamed to what it proves, with the reason in a comment above it, and a what this does not establish block added to the tool naming the three claims it is not: that the compiler produces this microcode, that the emitted Verilog simulates, that any of it works on silicon.
An honest tool under an overclaiming name is the most durable kind of wrong number, because every individual artefact is accurate. §90 and §91.
Refs #2161