ci(bootstrap): ratchet the test suite over failing names, with --no-fail-fast

.github/workflows/bootstrap-tests.yml

@@ -0,0 +1,99 @@
+# Ratchet the bootstrap test suite over a baseline of failing test NAMES.
+#
+# Why this exists (#2382): `cargo test -p t27c --tests` stops after the FIRST failing
+# target. The unit-test target fails, so the other 72 targets compile and never run —
+# every integration test under bootstrap/tests/ among them. That is why the regression
+# guards added for #2363, #2003, #2006 and others were dead: not because CI lacked a
+# test step, but because the step everyone had in mind would have run 1 target of 73.
+# Hence `--no-fail-fast`, and hence the ratchet refuses a log with only one target.
+#
+# Why a ratchet and not a gate: master has 383 failing tests today. A plain gate would
+# land red, and corpus-ratchet.yml already records what happens then — "a gate that
+# lands red gets disabled rather than obeyed".
+#
+# Why names and not a count: 383 still passes when one failure is fixed and a different
+# one appears. Only a set catches that.
+#
+# This job is NOT in the required-context set, so it does not block a merge (see #2376).
+# That is the repository owner's decision to make, not this workflow's.
+
+name: Bootstrap Test Ratchet
+
+on:
+  push:
+    branches: [master]
+    paths:
+      - 'bootstrap/**'
+      - 'Cargo.toml'
+      - 'Cargo.lock'
+      - 'scripts/ci/test_ratchet.py'
+      - 'scripts/ci/test-baseline.txt'
+      - '.github/workflows/bootstrap-tests.yml'
+  pull_request:
+    # No `branches:` filter, deliberately — scripts/ci/check_pr_branch_filters.py
+    # enforces its absence so stacked PRs are not silently ungated.
+    paths:
+      - 'bootstrap/**'
+      - 'Cargo.toml'
+      - 'Cargo.lock'
+      - 'scripts/ci/test_ratchet.py'
+      - 'scripts/ci/test-baseline.txt'
+      - '.github/workflows/bootstrap-tests.yml'
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+permissions:
+  contents: read
+
+jobs:
+  test-ratchet:
+    name: test-ratchet
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Cache cargo
+        uses: actions/cache@v4
+        with:
+          path: |
+            ~/.cargo/registry
+            ~/.cargo/git
+            target
+          key: ${{ runner.os }}-cargo-bootstrap-${{ hashFiles('Cargo.lock') }}
+
+      # The suite has 383 known failures, so cargo exits non-zero on a healthy run.
+      # The exit code is recorded rather than obeyed, and the RATCHET is the verdict.
+      # This is not `|| true`: a build failure produces a log with zero targets, which
+      # the ratchet reports as "NOT evaluated" and exits 2 on. Absence never reads as a
+      # clean set.
+      - name: Run bootstrap tests (--no-fail-fast)
+        id: tests
+        run: |
+          set -o pipefail
+          cargo test -p t27c --tests --no-fail-fast 2>&1 | tee test-log.txt \
+            || echo "cargo_exit=$?" >> "$GITHUB_OUTPUT"
+          echo "targets=$(grep -cE '^[[:space:]]+Running' test-log.txt || true)" >> "$GITHUB_OUTPUT"
+
+      - name: Ratchet the failing set
+        if: always()
+        run: |
+          set -eux
+          python3 scripts/ci/test_ratchet.py \
+            --log test-log.txt \
+            --baseline scripts/ci/test-baseline.txt
+
+      - name: Summary
+        if: always()
+        run: |
+          {
+            echo "### Bootstrap test ratchet"
+            echo ""
+            echo "- targets run: \`${{ steps.tests.outputs.targets }}\`"
+            echo "- cargo exit: \`${{ steps.tests.outputs.cargo_exit || '0' }}\` (recorded, not obeyed — see the workflow header)"
+            echo "- baseline: \`scripts/ci/test-baseline.txt\`"
+            echo ""
+            echo "A run reporting one target means \`--no-fail-fast\` was lost; the ratchet fails on it."
+          } >> "$GITHUB_STEP_SUMMARY"

docs/now/2026-08-22-bootstrap-test-ratchet.md

@@ -0,0 +1,61 @@
+# NOW — cargo test ran 1 of 73 targets
+
+Last updated: 2026-08-22
+
+## Ratchet the bootstrap suite over failing test names (Closes #2382)
+
+- Branch: `fix/2382-bootstrap-test-ratchet`
+- Issue: #2382
+
+### Что легло
+
+`scripts/ci/test_ratchet.py`, `scripts/ci/test-baseline.txt` (383 entries), and
+`.github/workflows/bootstrap-tests.yml` (check name `test-ratchet`).
+
+The measurement that motivated it, on `e53b9d048`:
+
+| command | targets | passed | failed |
+|---|---|---|---|
+| `cargo test -p t27c --tests` | **1** | 1621 | 13 |
+| same, `--no-fail-fast` | **73** | 2031 | **383** |
+
+`cargo test` stops after the first failing target. The unit target fails — those are the
+13 named in #2292 — so the other 72 compile and never run. Every regression guard added
+this week (#2363, #2003, #2006, #1977, #1985) lives in one of them. They were not broken
+and not missing; they were unreachable. The guard for #2363 **passes** when actually run.
+
+Of the 383: **61 of 73 targets are completely clean**; 358 failures are one target
+(`tests/icarus_lowerable.rs`, 0 passed of 358) attributable by shape to the emitter
+defect #2325; 13 are the known unit failures; **12 across 10 targets have never been
+examined**.
+
+Baseline is a set of `target<TAB>test` keys, not a count — 383 still passes when one
+failure is fixed and another appears. A baselined test that starts passing is reported,
+not failed on, so the set cannot rot unnoticed.
+
+### Границы честности (BINDING)
+
+- **The 12 unexamined failures were not diagnosed and their age was not established.**
+  Born-failing, long-failing and just-regressed need different responses. One of them,
+  `dma_local_addr_autoincrement_both_paths`, sits in the file #2345 changed — **that is
+  a lead, not a regression claim**.
+- The 358 are attributed to #2325 by shape (whole-target, 0 passed, `iverilog` present),
+  not by reading each failure.
+- Baseline measured on **macOS/arm64** at `e53b9d048` with a warm target dir; a Linux
+  runner may differ, and the first CI run will show whether it does. If it does, the
+  baseline must be regenerated from a runner log, not patched by hand.
+- This job is **not a required context**, so it blocks no merge (#2376). Making it one is
+  the repository owner's decision.
+- It does not fix anything. It stops the next regression from being invisible.
+
+### Evidence
+
+Three bars, measured. TRUE: unmodified log against its own baseline exits 0. BITING: with
+`bootstrap/tests/on_clock_plain_assign.rs:85` altered to assert a string the emitter never
+produces, a real rebuild + rerun gave 73 targets, 384 failures, and the ratchet exited 1
+naming `on_clock_emits_assignment_whose_rhs_does_not_reference_the_target`. Degenerate
+inputs — a one-target (fail-fast) log, an empty log, a missing log — all exit 2 with
+"NOT evaluated" rather than reading absence as a clean set. Exit codes were measured
+without a pipeline, since `$?` after a pipe reports the last command's status.
+
+Cost: **1 min 49 s** wall on a warm cache.