Repository navigation
tools: a check for shadowed top-level definitions, and the f16 narrowing corpus named and fingerprinted - #2246
Conversation
…s named and fingerprinted
`tools/hrx_f16_narrowing_check.py` defined `def self_test()` twice. The second wins, so the first
one's scan arms were dead code: the file imported, the surviving self_test printed twelve `ok` lines,
and three assertions could not fail because they never ran. Nobody noticed because nothing runs the
tool in CI -- only by hand.
Three things, in the order they matter:
tools/ci/dupdef_check.py an AST walk for a top-level def/class defined more than once, reported as
`name (lines N, M)`. It is scoped to tools/ and scripts/ and skips
third_party/build/node_modules. Five self-test arms, including the exact
shape it was written for and the control that a duplicated METHOD inside
a class is not a top-level shadow.
the dead copy removed its three scan arms (mul-updated / add-only / f32 control) already exist in
the surviving self_test, so this deletes 22 lines and loses no assertion.
the corpus is named `hrx_f16_narrowing_check.py <corpus-root>` now prints `CORPUS <abs path>`
and `FINGERPRINT <sha256[:16]> (<n> .loom files)`, hashing each file's
relative path and bytes so a move changes it too. Four arms: the file
count, the same corpus twice, a one-byte edit, and a rename.
Measured on this tree before the fix and after:
dupdef_check --check --root tools/hrx_f16_narrowing_check.py before rc 1 (self_test lines 163, 185)
after rc 0
dupdef_check --check (whole tools/ + scripts/) rc 0, 0 shadowed definitions
hrx_f16_narrowing_check.py --self-test 16 arms, rc 0 (was 12, all shadow-surviving)
Both are registered as ctest rows so they run without a ci.yml edit. ctest, configured fresh:
1/2 dupdef_check Passed 0.47 s 2/2 hrx_f16_narrowing_selftest Passed 0.08 s 100% passed
`ci_targets_check --check` still reports 9 baseline / none new, so the rows add no drift.
…ght.py expects `tools/copyright.py --check` failed on the new file in CI (the build job runs it before anything else), so this is the `--fix` output: the full Apache paragraph rather than the short SPDX-only header I wrote. `--check` rc 0 after; dupdef_check --check and --self-test unchanged.
PR Reviewer Guide 🔍(Review updated until commit bbacf7b)Here are some key observations to aid the review process:
|
|
Persistent review updated to latest commit bbacf7b |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at bbacf7b: READY
This catches a class that bit this fleet twice today, and neither time was it caught by anything automatic.
- #2086 registered the same
add_test(NAME bump_hrx_report …)twice — comment and all, in one hunk. CTest does not reject a duplicate name; it registers two tests with that name, so it passed CI and would have leftctest -Rrunning the fixture twice for ever, plus a duplicate row nobody would question. - #1790/#1794 both added the same implicit-zero-point block, so merging both applied the work twice.
Both are "a second definition that shadows the first and nothing notices", which is exactly what dupdef_check is for — and it is the same shape as the ratchet problem the fleet solved earlier tonight: a check can only see what it looks for, and a duplicate definition reads as a valid file to every other check.
Running it via python3 … --check with WORKING_DIRECTORY ${CMAKE_SOURCE_DIR} is right for a whole-tree scan, and "" in the title ("and the f16 …") suggests it found something already, which is the best sign a new checker is real.
|
VERDICT: READY at The finding is the best kind of process bug: a check whose arms were silently not running. The check is shaped to be checkable itself, which is the part I care about most:
Two asks:
Scope of my check: fresh configure rc 0, the |
…elf-test `add_python_test(hrx_f16_narrowing ...)` (CMakeLists.txt:418) has run `tools/hrx_f16_narrowing_check.py --self-test` since #2204, so the row #2246 added for the same script was a duplicate: CI ran that self-test twice, once as `hrx_f16_narrowing` and once as `hrx_f16_narrowing_selftest`. One script, one row. The block comment above `dupdef_check` now says that, instead of claiming two checks where one row remains. Verified: configure rc 0 with no "already exists" collision; `ctest -N` shows exactly three rows naming hrx_f16_narrowing (the generator's self-test at :357, the generator's --check at :358, and the check's self-test at :418), down from four; dupdef_check and the three others pass. Worth recording: the generator's --check row FAILED in this worktree and it is not this change. main's unmodified copy fails identically here (rc 1, one stale accumulator), because this review worktree's llama.cpp is at d8bd0ff60 while the pin is 4107043c - so the corpus is 124 .loom files instead of 612 and the baseline describes a different tree. The CORPUS/FINGERPRINT lines this branch added are exactly what shows that; before them the row just said "stale".
… and --write refuses a foreign one (#2249) * tools: the f16 narrowing generator names its corpus and fingerprints it, and --write refuses a foreign one The generator took the corpus from a module constant and never said which tree it read, so two runs that disagree are only comparable by reading the submodule sha by hand. Same fix as the check got: CORPUS /home/bcloud/wt/trel3/third_party/llama.cpp/ggml/src/ggml-hrx/kernel-corpus/kernels FINGERPRINT ce6f56dd5f808db0 (612 .loom file(s)) printed BEFORE the verdict, not after it -- a FAIL is when the identity of the tree matters most, and the first version printed it only on the pass path, so every failing run was unattributed. --corpus DIR scan a corpus other than the pinned checkout's (check() and write() both take it) fingerprint sha256 over sorted (relative path, bytes), so a move changes it, not only an edit And the footgun that this change made reachable, found by walking into it: `--write --corpus <foreign>` rewrote tools/ci/hrx_f16_narrowing.txt and hrx_f16_accumulators.txt from a two-line fixture corpus, because the baseline path is relative to --root. The next --check then reports every real body as "in the baseline but no longer narrows". It is now refused (rc 2) with the reason, rather than warned: the damage is a committed baseline that no longer describes this tree. The two files were restored (git checkout -- ...) before this commit; this commit touches only the generator. Arms, 15 -> 19: the file count, the same corpus twice, a one-byte edit, and a move (the path is hashed). Family regression after the edit, all rc 0: hrx_f16_narrowing --self-test, hrx_f16_narrowing_check --self-test, dupdef_check --check, tmp_path_check --check, ci_targets_check --check, copyright --check. * tools: drop the duplicate ctest row #2246 added for the f16 check's self-test `add_python_test(hrx_f16_narrowing ...)` (CMakeLists.txt:418) has run `tools/hrx_f16_narrowing_check.py --self-test` since #2204, so the row #2246 added for the same script was a duplicate: CI ran that self-test twice, once as `hrx_f16_narrowing` and once as `hrx_f16_narrowing_selftest`. One script, one row. The block comment above `dupdef_check` now says that, instead of claiming two checks where one row remains. Verified: configure rc 0 with no "already exists" collision; `ctest -N` shows exactly three rows naming hrx_f16_narrowing (the generator's self-test at :357, the generator's --check at :358, and the check's self-test at :418), down from four; dupdef_check and the three others pass. Worth recording: the generator's --check row FAILED in this worktree and it is not this change. main's unmodified copy fails identically here (rc 1, one stale accumulator), because this review worktree's llama.cpp is at d8bd0ff60 while the pin is 4107043c - so the corpus is 124 .loom files instead of 612 and the baseline describes a different tree. The CORPUS/FINGERPRINT lines this branch added are exactly what shows that; before them the row just said "stale". * scripts/pack-elf.sh: remove a duplicate pack_require_provenance -- the first copy was dead A shell function defined twice does what a Python def defined twice does: the last one wins and the first is dead. Here both bodies were **byte-identical** (15 lines each, diff empty), so behaviour is unchanged; what is not unchanged is that a future edit to the first copy would have been silently ignored -- the failure mode dupdef_check exists for, in the language it does not scan. Checked, not assumed: bash -n scripts/pack-elf.sh rc 0 . scripts/pack-elf.sh + declare -f of the five exported helpers all present pack_require_provenance on an identical stage/build pair 0 before, 0 after pack_require_provenance on a one-byte-changed pair 1 before, 1 after (and the same 'differs from the build' line) The surviving copy is the one with the fuller comment (why a missing implementation library must leave the package rather than ship broken), and pack_require_tools -- which sat between the two copies -- still has exactly one definition. **Shell is deliberately NOT added to tools/ci/dupdef_check.py.** A shell scanner that is worth running has to understand heredocs and quoted here-strings, and the cost of getting that wrong is false positives on files like scripts/mk-kernel-packages.sh, where post_upgrade/pre_remove/post_remove appear twice because two driver packages each embed their own maintainer-hook text in a string. Reporting those as duplicates would train people to ignore the check. The one real instance is fixed by hand and recorded here instead. --------- Co-authored-by: lane <lane@local>
Found while answering a review ask that repeated twice:
tools/hrx_f16_narrowing_check.pyshould print which corpus it read. Adding that, I hit the reason nobody had noticed the tool is not run in CI — the file definesdef self_test()twice.The defect
The second wins. The first is dead. Nothing warns: the file imports,
--self-testprints twelveoklines, and the three arms in the shadowed copy cannot fail because they never run. That is a check that cannot fail wearing the costume of two checks — the failure mode this tree has a rule about.I removed the dead copy (22 lines, no assertion lost: its three cases exist in the surviving body as
class 3: …) and added the thing that catches the next one.tools/ci/dupdef_check.pyAn AST walk for a top-level
def/classdefined more than once, reportingname (lines N, M)and the consequence. Scoped totools/andscripts/, skippingthird_party/build/node_modules/.venv— a duplicate in vendored code is upstream's business.Five self-test arms, and two of them are controls:
hrx_f16_narrowing_check.pyshape, reproduced as a fixture.The corpus is now named and fingerprinted
The original ask: a result is only comparable to another if the tree it read is identified.
sha256 over each file's relative path and bytes, sorted — so a moved file changes it, not just an edited one. Four arms: the file count, the same corpus twice (identical), a one-byte edit (different), a rename (different).
Both run in CI without touching ci.yml
add_testrows, since the workflow is not editable from a lane:hrx_f16_narrowing_check.py --self-testis now 16 arms rc 0 (was 12).ci_targets_check --checkstill reports 9 baseline / none new, so the new rows add no drift.One honest note on how I got here
My first attempt to add the
hashlibimport replaced a substring that did not exist and printed "hashlib imported" anyway — the--self-testthen failed withNameError. A print that reports work it did not do is the same defect as an arm that cannot fail, which is why the arms above assert the fingerprint against a real temporary corpus rather than that the function exists.