Repository navigation
tests: register the seven shell tests that nothing ran (fixes #1800) - #1804
Closed
bong-water-water-bong wants to merge 1 commit into
Closed
bong-water-water-bong wants to merge 1 commit into
bong-water-water-bong wants to merge 1 commit into
Conversation
Eight shell tests under tests/ were referenced by nothing: not an add_test, not a workflow, not a helper of another test. They exist, they pass by hand, and a green job never touched them - engine #1595 in the opposite direction (there, add_test targets that were never built; here, tests that were never registered). Each command line below was run against this tree before it was written down, and each exited 0: bash tests/base_modes_test.sh <root> -> base_modes: ok bash tests/gpu_hold_group_test.sh -> gpu_hold_group: ok bash tests/gpu_hold_hold_dir_test.sh <root>/scripts/gpu-hold -> ok (3 directory states) bash tests/gpu_hold_mem_abort_test.sh <root>/scripts/gpu-hold -> gpu_hold_mem_abort: ok bash tests/gpu_hold_survivors_test.sh <root>/scripts/gpu-hold -> gpu_hold_survivors: ok bash tests/gpu_hold_ticket_lost_test.sh -> PASS (removed ticket exits 75 in 1s) bash tests/gpu_hold_tree_test.sh -> gpu_hold_tree: ok anchor_rerun.sh stays unregistered on purpose: `anchor_rerun.sh <1bit> <outdir> <device> <alias>:<gguf>` needs a built binary and a device, so it is a manual instrument, not a CI gate; registering it would add a test that can only skip. The reason is in a comment beside the block. The six gpu_hold_* arms are the regression set for the orphan family (#1620, #1626, #1633, #1640) - the class that put a 14 h llama-server and 51 GiB of GTT into another lane's ticket. Their arms existed and passed; nothing enforced them. No worktree was created for this: disk is at 110 G (below the 175 G gate), so the commit was built with git plumbing (read-tree/update-index/write-tree/commit-tree) against origin/main.
Collaborator
Author
|
@pi-prs ready for review at |
This was referenced Nov 10, 2026
Closed
bong-water-water-bong
added a commit
that referenced
this pull request
Nov 10, 2026
…ide it (#1906) Tonight's dominant defect was a test that existed and never ran: #1813's oracle merged with no registration, #1781's comparator green-but-unbuilt, #1804 refused by a duplicate row name. CONTRIBUTING.md explains the pins, the registry and the agents, and said nothing about the one step where a new test is silently dropped. One bullet under Automation, each claim checked against the tree on main: ci_targets_check --check 9 targets never reach ctest (baseline 9), none new tests_referenced_check --check all 73 test script(s) referenced; 1 allow-listed python_test_order_check --check every ${Python3_EXECUTABLE} use follows find_package add_dependencies(onebit onebit_npu_forward_syntax) CMakeLists.txt:643 No new heading and no new page: this is the existing Automation list, where a reader already looks before touching CI. Co-authored-by: t <t@e.com>
bong-water-water-bong
deleted the
pi-docs/register-unreferenced-tests
branch
November 12, 2026 21:08
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1800. Eight shell tests under
tests/were referenced by nothing — noadd_test, no workflow, noscripts/caller, no helper relationship — so a green job never ran any of them. Seven are registered here; the eighth is documented as deliberately excluded.This is #1595 in the opposite direction: there,
add_testtargets were never built, so tests reported "Not Run" inside a green job; here, the tests were never registered, so the job did not know they existed.Every command line was run before it was written down
Not inferred from the files' headers — executed against this tree, each exiting 0:
base_modestests/base_modes_test.sh ${CMAKE_SOURCE_DIR}base_modes: okgpu_hold_grouptests/gpu_hold_group_test.shgpu_hold_group: okgpu_hold_hold_dir…/gpu_hold_hold_dir_test.sh ${CMAKE_SOURCE_DIR}/scripts/gpu-holdgpu_hold_mem_abort…/gpu_hold_mem_abort_test.sh …/scripts/gpu-holdgpu_hold_mem_abort: okgpu_hold_survivors…/gpu_hold_survivors_test.sh …/scripts/gpu-holdgpu_hold_survivors: okgpu_hold_ticket_lost…/gpu_hold_ticket_lost_test.shPASS— a removed ticket exits 75 in 1sgpu_hold_tree…/gpu_hold_tree_test.shgpu_hold_tree: okEach takes no argument or a path, which is why none of them needs
$<TARGET_FILE:onebit>— unlike theserve-shaped tests atCMakeLists.txt:192-203.anchor_rerun.shis excluded, in a comment beside the blockanchor_rerun.sh <1bit> <outdir> <device> <alias>:<gguf>needs a built binary and a device. Registering it would add a test that can only ever skip, which is how a suite grows green rows that mean nothing. The comment says so, so the next sweep does not "fix" it.Why the six
gpu_hold_*matter mostThey are the regression arms for the orphan family — #1620, #1626, #1633, #1640 — the class that put a 14 h
llama-serverholding 51 GiB of GTT into another lane's ticket. Every one of those arms existed and passed by hand; nothing enforced them. After this PR, a future edit that re-breaks the teardown fails CI instead of passing review.Still open on #1800 (not in this PR)
The third item in the issue: a check that would prevent the next unregistered test — assert every
tests/*_test.shappears inCMakeLists.txt. That needs a naming convention decision (what counts as a test vs a helper), so it belongs in its own change rather than being guessed here.Verified
git diff --stat origin/main→ CMakeLists.txt, +15 lines, 0 deletions; parentheses balanced;add_testcount 185 → 192.read-tree/hash-object/update-index/write-tree/commit-tree) againstorigin/main, andrefs/heads/pi-docs/register-unreferenced-tests= local2fb14d5.