ci(bootstrap): ratchet the test suite over failing names, with --no-fail-fast

[gHashTag]

Closes #2382

The measurement

On e53b9d048, with a warm target dir:

command	targets run	passed	failed
cargo test -p t27c --tests	1	1621	13
same, --no-fail-fast	73	2031	383

cargo test stops after the first failing target. The unit-test target fails — those are
the 13 named in #2292 — so the other 72 compile and never run. Every integration test
under bootstrap/tests/ is in that group, including every regression guard added this
week: #2363, #2003, #2006, #1977, #1985. They were not broken and not missing. They were
unreachable. The guard for #2363 passes when actually run.

This also explains the --bins observation on #2363: that measurement missed the guard by
a different mechanism, and the recorded "13 failures" in corpus-ratchet.yml's comment is
the fail-fast artefact.

The 383 is not 383 problems

• 61 of 73 targets are completely clean.
• 358 failures are one target — tests/icarus_lowerable.rs, 0 passed of 358.
  iverilog is installed, so not a missing tool: attributable by shape to the emitter
  defect gen-verilog: 26 of 32 modules reference struct fields under a prefix declared nowhere — no single prefix can be correct #2325, counted once per spec.
• 13 are the known unit failures.
• 12 across 10 targets have never been examined.

What landed

scripts/ci/test_ratchet.py, scripts/ci/test-baseline.txt (383 entries), and
.github/workflows/bootstrap-tests.yml — its own workflow with its own check name
(test-ratchet), deliberately not inside corpus-ratchet.yml, which is red on master
and where a new failure would be indistinguishable from the standing one.

Three design choices, each against a specific failure this repository has already had:

• A set of target<TAB>test keys, not a count. 383 still passes when one failure is
  fixed and a different one appears.
• A ratchet, not a gate. corpus-ratchet.yml already records why: "a gate that lands
  red gets disabled rather than obeyed."
• Degenerate logs refuse rather than certify. A one-target log — the exact condition
  that hid 72 targets — exits 2 with NOT evaluated, as do an empty and a missing log.
  Absence never reads as a clean set.

A baselined test that starts passing is reported, not failed on, so the baseline
cannot rot into a list nobody can tell is stale.

Evidence

TRUE — unmodified log against its own baseline: 73 targets ran; 383 failing test(s) (baseline 383) / No new failures. Baseline holds. exit 0.

BITING, on a real rebuild and rerun, not a doctored log — bootstrap/tests/on_clock_plain_assign.rs:85
altered to assert a string the emitter never produces:

73 targets ran; 384 failing test(s) (baseline 383)

**FAILED — these tests are newly failing:**
  - `on_clock_emits_assignment_whose_rhs_does_not_reference_the_target`   (tests/on_clock_plain_assign.rs)

exit 1. Restored afterwards; the tree is byte-identical to master apart from the three new files.

Degenerate — fail-fast log, empty log, missing log: all exit 2. Exit codes were measured
without a pipeline, since $? after a pipe reports the last command's status.

Cost: 1 min 49 s wall, warm cache.

What this does not establish

• The 12 unexamined failures were not diagnosed and their age was not established.
  Born-failing, long-failing and just-regressed need different responses. One of them,
  dma_local_addr_autoincrement_both_paths, sits in the file fix(bitnet): pair the DMA local write address with its own beat (Closes #2003) #2345 changed — a lead, not
  a regression claim.
• The 358 are attributed to gen-verilog: 26 of 32 modules reference struct fields under a prefix declared nowhere — no single prefix can be correct #2325 by shape, not by reading each failure message.
• Baseline measured on macOS/arm64. A Linux runner may differ; the first CI run will
  show whether it does, and if so the baseline must be regenerated from a runner log rather
  than patched by hand.
• This job is not a required context (build and L1 TRACEABILITY can fail without blocking a merge — neither is a required context #2376), so it blocks no merge. Making it one is
  the repository owner's decision.
• It fixes nothing. It stops the next regression from being invisible.

[github-actions]

📓 NotebookLM Notebook linked to this PR

• Notebook ID: b83263109fb055dc
• URL: https://notebooklm.google.com/notebook/b83263109fb055dc

This notebook contains session context, decisions, and artifacts for this work.

[github-actions]

PR Dashboard

Generated at: 2026-08-22 07:31:03 UTC

• ci(bootstrap): ratchet the test suite over failing names, with --no-fail-fast #2383 ci(bootstrap): ratchet the test suite over failing names, with --no-fail-fast | Created: 2026-08-22 | Updated: 2026-08-22 | 15 failing / 24 total
• corpus: freeze the damage, patch one class at a time, and say where one rule stops #2161 corpus: freeze the damage, patch one class at a time, and say where one rule stops | Created: 2026-08-14 | Updated: 2026-08-21 | 0 failing / 9 total
• Unbreak FPGA CI and correct two unbacked hardware claims #2081 Unbreak FPGA CI and correct two unbacked hardware claims | Created: 2026-08-11 | Updated: 2026-08-20 | 5 failing / 27 total

Summary

Status	Count
Total Open PRs	3
PRs with Failing Checks	1
PRs with All Checks Green	2
READY	1
FAILING	1
PENDING	0

Seal Status

• ⚠️ STALE -- sha256(compiler.rs)=fba627661c36 != manifest seal=87e5cbd3ad94.
  The committed NMSE numbers were certified against an older compiler.rs.
  Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).