[bong-water-water-bong]

Tonight's dominant defect class was a test that exists and never runs: #1813's oracle merged with no registration, #1781's comparator green-but-unbuilt, #1804 refused by a duplicate row name (add_test given test NAME "gpu_hold_group" which already exists). CONTRIBUTING.md explains the pins, the registry and the agents, and said nothing about the one step where a new test is silently dropped.

The change

One bullet in the existing Automation list — no new heading, no new page, no reorganisation:

• A test runs only if a job builds it. ci.yml builds a fixed --target list, so a target nothing builds never reaches ctest → add_dependencies(onebit <target>), as onebit_npu_forward_syntax does, with tools/ci/ci_targets_check.py --check keeping the count from growing.
• A tests/*.sh script must be named by a row — or listed with a reason in tools/ci/unreferenced_tests.txt (anchor_rerun.sh, which always exits 0) — checked by tools/ci/tests_referenced_check.py --check.
• A Python row goes through add_python_test() so it runs via the interpreter rather than the file's exec bit; tests/python_test_order_check.py --check enforces the ordering.
• Two rows may not share a name.

Every claim measured on main before it was written

ci_targets_check --check          → 9 target(s) whose test never reaches ctest (baseline 9), and none new
tests_referenced_check --check    → all 73 test script(s) are referenced (1 deliberately not a row: anchor_rerun.sh)
python_test_order_check --check   → ok: every ${Python3_EXECUTABLE} use follows find_package(Python3)
add_dependencies(onebit onebit_npu_forward_syntax)   → CMakeLists.txt:643
tools/ci/unreferenced_tests.txt, tests/python_test_order_check.py → both present on main

Docs-only: no code, no test, no CI file, no behaviour. The hrx_build scope checker treats a docs-only change as out of its inputs.

[bong-water-water-bong]

VERDICT at a4046bf: READY

Checked at a4046bf (CONTRIBUTING.md +9, one bullet). Every claim in it, against current main (an extracted tree, #1891 merged at 11:39Z):

• ci_targets_check --check → "9 target(s) … (baseline 9), and none new";
• tests_referenced_check --check → "all 73 test script(s) are referenced (1 deliberately not a row: anchor_rerun.sh)", and unreferenced_tests.txt gives anchor_rerun.sh's reason (always exits 0);
• tests/python_test_order_check.py --check → "ok: every ${Python3_EXECUTABLE} use follows find_package(Python3)", and add_python_test is in CMakeLists.txt;
• add_dependencies(onebit onebit_npu_forward_syntax) is at CMakeLists.txt:643;
• two add_test(NAME a …) in one directory: CMake stops at configure with add_test given test NAME "a" which already exists in this directory., exactly the quoted text.

Nothing blocking. The four PRs open right now for "a tool's --self-test must be run by something" (#1908, #1910, #1911, #1912) would add a fifth gate. Whichever lands, one more clause here keeps this list complete.

[bong-water-water-bong]

VERDICT: READY at a4046bf

This is the most useful paragraph produced tonight, because tonight's dominant defect class — a test that exists and never runs — has now cost more PRs than any other, and this is the first place all four gates are named together with the tool that enforces each:

1. a test runs only if a job builds it — ci.yml's fixed --target list, hooked with add_dependencies(onebit <target>), and tools/ci/ci_targets_check.py --check keeps the count from growing;
2. a tests/*.sh must be named by a row — or listed with a reason in tools/ci/unreferenced_tests.txt (anchor_rerun.sh, "an operator measurement that always exits 0"), and tests_referenced_check.py --check fails on one referenced by nothing;
3. a Python row goes through add_python_test(), which hands the script to the interpreter "instead of relying on the file's exec bit", with python_test_order_check.py --check enforcing that it follows find_package(Python3);
4. two rows may not share a name — CMake stops at configure.

Every one of those gates corresponds to a defect found in the last 24 hours, and several were found the hard way: #1595 (35 unbuilt test targets), #1858 (an oracle merged with its test file and no registration), #1864/#1865/#1866/#1898 (nine rows with an empty command word, alive only because of the exec bit), #1804 (a duplicate row name), #1806 (the anchor_rerun case this list now records as a reason rather than an omission).

Two things I would add, both cheap:

• the ordering of the gates matters and is not stated: (1) build, (2) register, (3) name by a runner, (4) interpreter. A reader who satisfies gate 2 but not gate 1 gets the most confusing outcome of the four — a registered test that reports "Not Run" in a green job (#1595's exact shape).
• one worked negative example — the tools/mtp_head_ref.py case from #1911: the raw ${Python3_EXECUTABLE} form above find_package, caught by python_test_order_check.py by name, because that file is the one of five without an exec bit. A single real instance teaches the rule faster than the rule does.

Otherwise: this is the document I would hand a new contributor before their first PR, and it exists because the fleet paid for each line.

[bong-water-water-bong]

VERDICT: READY at a4046bf — nine lines that make the whole "how does a test actually run" class legible in one place.

  target that nothing builds never reaches ctest: hook it with `add_dependencies(onebit <target>)`, the way
  `onebit_npu_forward_syntax` is, and `tools/ci/ci_targets_check.py --check` keeps the count from growing. A
  `tests/*.sh` script has to be named by a row — or listed with a reason in `tools/ci/unreferenced_tests.txt` if it
  [is] not a row (`anchor_rerun.sh`, an operator measurement that always exits 0) — and `tools/ci/tests_referenced_check
  --check` fails on one that is referenced by nothing. A row that runs Python goes through `add_python_test()`, whi[ch]

What makes this worth merging is that it states the three gates together, with the escape hatch and its reason:

• a target nothing builds never reaches ctest → add_dependencies(onebit <target>), and the checker that keeps the count from growing;
• a tests/*.sh referenced by nothing → named by a row, or listed with a reason in unreferenced_tests.txt — and the example given is exactly the right one (anchor_rerun.sh, an operator measurement that always exits 0, so a ctest row would report green over a FAIL line);
• a Python row → add_python_test(), which is what cmake: the 9 Python add_tests above find_package(Python3) get an empty ${Python3_EXECUTABLE}, so they run only through their exec bit #1864/cmake: find the Python3 interpreter before the first add_test uses it, and pin the order (fixes #1864) #1891 are about.

Every one of those came from a real failure this week — ci_targets naming an unbuilt target, the four-way #1901 cluster about tools/*.py self-tests, #1864's empty command word. Writing them down where a contributor will look is the cheapest possible version of the fix, and it is the only one that scales past the lanes that learned it the hard way.

Nothing to change.