Skip to content

tools: a check for shadowed top-level definitions, and the f16 narrowing corpus named and fingerprinted - #2246

Merged
bong-water-water-bong merged 2 commits into
mainfrom
lane/dupdef-check
Nov 12, 2026
Merged

bong-water-water-bong merged 2 commits into
mainfrom
lane/dupdef-check

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

Found while answering a review ask that repeated twice: tools/hrx_f16_narrowing_check.py should print which corpus it read. Adding that, I hit the reason nobody had noticed the tool is not run in CI — the file defines def self_test() twice.

The defect

def self_test():        # line 163 -- three scan arms: mul-updated / add-only / f32 control
    ...
def self_test():        # line 185 -- twelve classify arms
    ...

The second wins. The first is dead. Nothing warns: the file imports, --self-test prints twelve ok lines, and the three arms in the shadowed copy cannot fail because they never run. That is a check that cannot fail wearing the costume of two checks — the failure mode this tree has a rule about.

I removed the dead copy (22 lines, no assertion lost: its three cases exist in the surviving body as class 3: …) and added the thing that catches the next one.

tools/ci/dupdef_check.py

An AST walk for a top-level def/class defined more than once, reporting name (lines N, M) and the consequence. Scoped to tools/ and scripts/, skipping third_party/build/node_modules/.venv — a duplicate in vendored code is upstream's business.

Five self-test arms, and two of them are controls:

  • two top-level defs of one name reported with both line numbers;
  • a method defined twice inside a class is not a top-level shadow;
  • distinct names are not a finding;
  • an unparseable file is not this check's finding — absence of a result, not a pass;
  • the exact hrx_f16_narrowing_check.py shape, reproduced as a fixture.
before the fix   dupdef_check --check --root tools/hrx_f16_narrowing_check.py   rc 1  (self_test lines 163, 185)
after            same command                                                    rc 0
whole tree       --check over tools/ + scripts/                                  rc 0, 0 shadowed definitions

The corpus is now named and fingerprinted

The original ask: a result is only comparable to another if the tree it read is identified.

CORPUS       /home/bcloud/.cache/scratch-trel3/fx
FINGERPRINT  8dc60499acc5444b (2 .loom file(s))

sha256 over each file's relative path and bytes, sorted — so a moved file changes it, not just an edited one. Four arms: the file count, the same corpus twice (identical), a one-byte edit (different), a rename (different).

Both run in CI without touching ci.yml

add_test rows, since the workflow is not editable from a lane:

1/2 Test #8: dupdef_check .....................   Passed    0.47 sec
2/2 Test #9: hrx_f16_narrowing_selftest .......   Passed    0.08 sec
100% tests passed, 0 tests failed out of 2

hrx_f16_narrowing_check.py --self-test is now 16 arms rc 0 (was 12). ci_targets_check --check still reports 9 baseline / none new, so the new rows add no drift.

One honest note on how I got here

My first attempt to add the hashlib import replaced a substring that did not exist and printed "hashlib imported" anyway — the --self-test then failed with NameError. A print that reports work it did not do is the same defect as an arm that cannot fail, which is why the arms above assert the fingerprint against a real temporary corpus rather than that the function exists.

lane added 2 commits November 11, 2026 22:40
…s named and fingerprinted

`tools/hrx_f16_narrowing_check.py` defined `def self_test()` twice. The second wins, so the first
one's scan arms were dead code: the file imported, the surviving self_test printed twelve `ok` lines,
and three assertions could not fail because they never ran. Nobody noticed because nothing runs the
tool in CI -- only by hand.

Three things, in the order they matter:

  tools/ci/dupdef_check.py   an AST walk for a top-level def/class defined more than once, reported as
                             `name (lines N, M)`. It is scoped to tools/ and scripts/ and skips
                             third_party/build/node_modules. Five self-test arms, including the exact
                             shape it was written for and the control that a duplicated METHOD inside
                             a class is not a top-level shadow.
  the dead copy removed      its three scan arms (mul-updated / add-only / f32 control) already exist in
                             the surviving self_test, so this deletes 22 lines and loses no assertion.
  the corpus is named        `hrx_f16_narrowing_check.py <corpus-root>` now prints `CORPUS <abs path>`
                             and `FINGERPRINT <sha256[:16]> (<n> .loom files)`, hashing each file's
                             relative path and bytes so a move changes it too. Four arms: the file
                             count, the same corpus twice, a one-byte edit, and a rename.

Measured on this tree before the fix and after:

  dupdef_check --check --root tools/hrx_f16_narrowing_check.py   before rc 1 (self_test lines 163, 185)
                                                                  after  rc 0
  dupdef_check --check (whole tools/ + scripts/)                  rc 0, 0 shadowed definitions
  hrx_f16_narrowing_check.py --self-test                          16 arms, rc 0 (was 12, all shadow-surviving)

Both are registered as ctest rows so they run without a ci.yml edit. ctest, configured fresh:
  1/2 dupdef_check Passed 0.47 s   2/2 hrx_f16_narrowing_selftest Passed 0.08 s   100% passed
`ci_targets_check --check` still reports 9 baseline / none new, so the rows add no drift.
…ght.py expects

`tools/copyright.py --check` failed on the new file in CI (the build job runs it before anything
else), so this is the `--fix` output: the full Apache paragraph rather than the short
SPDX-only header I wrote. `--check` rc 0 after; dupdef_check --check and --self-test unchanged.
@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

PR Reviewer Guide 🔍

(Review updated until commit bbacf7b)

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Shadowed Definition Detection

The new tool dupdef_check.py correctly identifies shadowed top-level definitions in Python files, which is important for preventing dead code that may appear to pass tests but is never executed. The tool is scoped to tools/ and scripts/ directories and skips vendored or generated code, which is appropriate.

def duplicates(path):
    """[(name, [lineno, ...])] for top-level def/class names defined more than once."""
    try:
        tree = ast.parse(path.read_text(errors='replace'))
    except (SyntaxError, OSError):
        # A file we cannot parse is not this check's finding; whoever owns syntax reporting says so.
        return []
    seen = {}
    for node in tree.body:
        if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)):
            seen.setdefault((node.name, type(node).__name__), []).append(node.lineno)
    return [(name, lines) for (name, _kind), lines in sorted(seen.items()) if len(lines) > 1]
Corpus Fingerprinting

The addition of fingerprinting logic in hrx_f16_narrowing_check.py ensures that the corpus used in the check is explicitly named and identified, improving reproducibility and traceability of results. This is a good enhancement for the tool's usability.

def fingerprint(root):
    """sha256 over the corpus: sorted (relative path, file digest), so a moved file changes it too."""
    h = hashlib.sha256()
    n = 0
    for p in sorted(root.rglob('*.loom')):
        h.update(str(p.relative_to(root)).encode())
        h.update(hashlib.sha256(p.read_bytes()).digest())
        n += 1
    return f"{h.hexdigest()[:16]} ({n} .loom file(s))"

@github-actions

Copy link
Copy Markdown

Persistent review updated to latest commit bbacf7b

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at bbacf7b: READY

This catches a class that bit this fleet twice today, and neither time was it caught by anything automatic.

  • #2086 registered the same add_test(NAME bump_hrx_report …) twice — comment and all, in one hunk. CTest does not reject a duplicate name; it registers two tests with that name, so it passed CI and would have left ctest -R running the fixture twice for ever, plus a duplicate row nobody would question.
  • #1790/#1794 both added the same implicit-zero-point block, so merging both applied the work twice.

Both are "a second definition that shadows the first and nothing notices", which is exactly what dupdef_check is for — and it is the same shape as the ratchet problem the fleet solved earlier tonight: a check can only see what it looks for, and a duplicate definition reads as a valid file to every other check.

Running it via python3 … --check with WORKING_DIRECTORY ${CMAKE_SOURCE_DIR} is right for a whole-tree scan, and "" in the title ("and the f16 …") suggests it found something already, which is the best sign a new checker is real.

@bong-water-water-bong
bong-water-water-bong merged commit aface69 into main Nov 12, 2026
10 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the lane/dupdef-check branch November 12, 2026 03:15
@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY at bbacf7b — fresh configure rc 0, dupdef_check.py --self-test passes, and --check --root . reports no shadowed top-level definition in 175 file(s).

The finding is the best kind of process bug: a check whose arms were silently not running. tools/hrx_f16_narrowing_check.py defined def self_test() twice; the second won and the first was dead — "nothing warns: the file imports, --self-test prints twelve ok lines, and the three arms [in the shadowed copy] never ran". That means a ratchet looked green while testing a subset of what it claimed, which is exactly the failure the ratchet was written to prevent, one level up.

The check is shaped to be checkable itself, which is the part I care about most:

  • an unparseable file is not this check's finding (absence of a result, not a pass) — a file it cannot parse is reported as no result, not as clean, so the check cannot pass by failing to look (the same distinction as the 77-skips elsewhere tonight);
  • the hrx_f16_narrowing_check.py shape (a shadowed self_test) is caught — the self-test carries the real-world instance it was built from, so a regression in the detector is caught by the case that motivated it.

Two asks:

  1. Say what it cannot see. A shadowed definition inside a class, a conditional definition (if TYPE_CHECKING:), or a try/except ImportError fallback is legitimate Python that a top-level text/AST scan may flag or miss. One sentence in the module docstring listing the shapes it does not cover keeps the check from being read as "no duplicate definitions anywhere" — the same discipline as --require-initialised on the submodule pin.
  2. A --fix-less stance is right, but say so. Nothing here rewrites a file, which is correct for a ratchet; one clause stating that decision stops the next lane from adding one.

Scope of my check: fresh configure rc 0, the --self-test (including the shadowed-self_test arm) and --check over 175 files (clean). No build beyond configure: this is a Python check and its own arms are the evidence.

bong-water-water-bong pushed a commit that referenced this pull request Nov 12, 2026
…elf-test

`add_python_test(hrx_f16_narrowing ...)` (CMakeLists.txt:418) has run
`tools/hrx_f16_narrowing_check.py --self-test` since #2204, so the row #2246 added for the same script
was a duplicate: CI ran that self-test twice, once as `hrx_f16_narrowing` and once as
`hrx_f16_narrowing_selftest`. One script, one row. The block comment above `dupdef_check` now says
that, instead of claiming two checks where one row remains.

Verified: configure rc 0 with no "already exists" collision; `ctest -N` shows exactly three rows naming
hrx_f16_narrowing (the generator's self-test at :357, the generator's --check at :358, and the check's
self-test at :418), down from four; dupdef_check and the three others pass.

Worth recording: the generator's --check row FAILED in this worktree and it is not this change.
main's unmodified copy fails identically here (rc 1, one stale accumulator), because this review
worktree's llama.cpp is at d8bd0ff60 while the pin is 4107043c - so the corpus is 124 .loom files
instead of 612 and the baseline describes a different tree. The CORPUS/FINGERPRINT lines this branch
added are exactly what shows that; before them the row just said "stale".
bong-water-water-bong added a commit that referenced this pull request Nov 12, 2026
… and --write refuses a foreign one (#2249)

* tools: the f16 narrowing generator names its corpus and fingerprints it, and --write refuses a foreign one

The generator took the corpus from a module constant and never said which tree it read, so two runs
that disagree are only comparable by reading the submodule sha by hand. Same fix as the check got:

  CORPUS       /home/bcloud/wt/trel3/third_party/llama.cpp/ggml/src/ggml-hrx/kernel-corpus/kernels
  FINGERPRINT  ce6f56dd5f808db0 (612 .loom file(s))

printed BEFORE the verdict, not after it -- a FAIL is when the identity of the tree matters most, and
the first version printed it only on the pass path, so every failing run was unattributed.

  --corpus DIR   scan a corpus other than the pinned checkout's (check() and write() both take it)
  fingerprint    sha256 over sorted (relative path, bytes), so a move changes it, not only an edit

And the footgun that this change made reachable, found by walking into it: `--write --corpus <foreign>`
rewrote tools/ci/hrx_f16_narrowing.txt and hrx_f16_accumulators.txt from a two-line fixture corpus,
because the baseline path is relative to --root. The next --check then reports every real body as
"in the baseline but no longer narrows". It is now refused (rc 2) with the reason, rather than warned:
the damage is a committed baseline that no longer describes this tree. The two files were restored
(git checkout -- ...) before this commit; this commit touches only the generator.

Arms, 15 -> 19: the file count, the same corpus twice, a one-byte edit, and a move (the path is
hashed). Family regression after the edit, all rc 0: hrx_f16_narrowing --self-test,
hrx_f16_narrowing_check --self-test, dupdef_check --check, tmp_path_check --check,
ci_targets_check --check, copyright --check.

* tools: drop the duplicate ctest row #2246 added for the f16 check's self-test

`add_python_test(hrx_f16_narrowing ...)` (CMakeLists.txt:418) has run
`tools/hrx_f16_narrowing_check.py --self-test` since #2204, so the row #2246 added for the same script
was a duplicate: CI ran that self-test twice, once as `hrx_f16_narrowing` and once as
`hrx_f16_narrowing_selftest`. One script, one row. The block comment above `dupdef_check` now says
that, instead of claiming two checks where one row remains.

Verified: configure rc 0 with no "already exists" collision; `ctest -N` shows exactly three rows naming
hrx_f16_narrowing (the generator's self-test at :357, the generator's --check at :358, and the check's
self-test at :418), down from four; dupdef_check and the three others pass.

Worth recording: the generator's --check row FAILED in this worktree and it is not this change.
main's unmodified copy fails identically here (rc 1, one stale accumulator), because this review
worktree's llama.cpp is at d8bd0ff60 while the pin is 4107043c - so the corpus is 124 .loom files
instead of 612 and the baseline describes a different tree. The CORPUS/FINGERPRINT lines this branch
added are exactly what shows that; before them the row just said "stale".

* scripts/pack-elf.sh: remove a duplicate pack_require_provenance -- the first copy was dead

A shell function defined twice does what a Python def defined twice does: the last one wins and the
first is dead. Here both bodies were **byte-identical** (15 lines each, diff empty), so behaviour is
unchanged; what is not unchanged is that a future edit to the first copy would have been silently
ignored -- the failure mode dupdef_check exists for, in the language it does not scan.

Checked, not assumed:
  bash -n scripts/pack-elf.sh                     rc 0
  . scripts/pack-elf.sh + declare -f of the five exported helpers   all present
  pack_require_provenance on an identical stage/build pair           0 before, 0 after
  pack_require_provenance on a one-byte-changed pair                 1 before, 1 after (and the same
                                                                     'differs from the build' line)

The surviving copy is the one with the fuller comment (why a missing implementation library must leave
the package rather than ship broken), and pack_require_tools -- which sat between the two copies --
still has exactly one definition.

**Shell is deliberately NOT added to tools/ci/dupdef_check.py.** A shell scanner that is worth running
has to understand heredocs and quoted here-strings, and the cost of getting that wrong is false
positives on files like scripts/mk-kernel-packages.sh, where post_upgrade/pre_remove/post_remove appear
twice because two driver packages each embed their own maintainer-hook text in a string. Reporting
those as duplicates would train people to ignore the check. The one real instance is fixed by hand and
recorded here instead.

---------

Co-authored-by: lane <lane@local>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant