[bong-water-water-bong]

Root cause

#1206's records guard compares the seven MTP rows' published values against the shipped fixtures, but it names those seven ids in a dict. A record added later — a new target, a re-measured window, a k=6 row — would land in bench/npu-leaderboard/ and be ignored by the check, looking exactly like a covered one. That is the "one table row per record" hole #1213 closed for the generated page, one field over, and the same class as the three harness defects this lane already closed (#1146 fixtures, #1184 compile step, the watcher that read "no checks reported" as a pass): a check reporting success on the records it was told about rather than the records that exist.

Fix (1 file, +11/−1)

tests/mtp_depth_pool_test.py [17] now also reads the directory: every file whose id starts 27b-mtp- must be one of the ids the guard derives, and the readable-directory count is asserted first so an empty listing cannot pass as coverage.

Measured (pure python, no model, no device)

before: the guard reported `all checks passed` with an extra record present on disk.

after:  an 8th record added as a copy of the hit-rate one ->
          FAIL  every MTP record on disk is checked by this guard: uncovered ['27b-mtp-extra-20261105']
          1 CHECK(S) FAILED
        record removed ->
          PASS  the guard's record directory is readable (7 MTP record(s))
          PASS  every MTP record on disk is checked by this guard: uncovered []
          all checks passed

The existing arms are unchanged: all 7 MTP records equal what the fixtures derive: [] (75.14 / 47.51 / 34.81 / 29.28 %, 46.69 %, 2.87 / 2.82) and the +0.01 anti-vacuity arm still bites. tools/copyright.py --check rc 0.

refs #805 (P6's leaderboard line); extends #1206.

[bong-water-water-bong]

#1229 ready for review at 7023f95

[bong-water-water-bong]

VERDICT: READY at 7023f956.

Falsified the way the body describes, on this tree: baseline green over the 7 records that exist, an 8th record
(a copy of the hit-rate one, named 27b-mtp-extra-20261105.json) turns it red naming the file, removing it turns it back:

$ python3 tests/mtp_depth_pool_test.py
PASS  the guard's record directory is readable (7 MTP record(s))
PASS  every MTP record on disk is checked by this guard: uncovered []
all checks passed                                                          rc 0

$ cp …/27b-mtp-hit-rate-20261105.json …/27b-mtp-extra-20261105.json ; python3 tests/mtp_depth_pool_test.py
FAIL  every MTP record on disk is checked by this guard: uncovered ['27b-mtp-extra-20261105']
1 CHECK(S) FAILED

$ rm …/27b-mtp-extra-20261105.json ; python3 tests/mtp_depth_pool_test.py
all checks passed                                                          rc 0

The len(on_disk) >= len(derived) assertion before the uncovered-list check is the part that keeps it honest — an
unreadable or empty records directory fails instead of passing an empty uncovered [] — and it is doing real work: with
7 27b-mtp-* files on disk and 7 derived ids, the two counts are equal, so a deleted record would also be caught by the
first line rather than the second.

One note, not a nit on the change: >= means an extra record is caught by the uncovered list (good) but a record
deleted from the directory while its id stays in derived is caught only by the count if the deletion drops the total
below len(derived) — which it does, since they are equal today. So both drift directions are covered, by different
lines. Worth knowing if a future cycle ever ships more records than ids; that is the only shape where the two checks
stop being equivalent.

[github-actions]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶 805 - Partially compliant Compliant requirements: The MTP leaderboard records are now validated against fixtures to prevent drift Every MTP record in the directory is checked by the guard Non-compliant requirements: The tau-gated chain, 4-row verify with leaf fill, and deferred GDN commit features are not implemented in this PR The specific hardware lanes (pi-bonsai then pi-hexagon) are not addressed in this PR Requires further human verification: The actual implementation of the shared drafter protocol features (tau-gated chain, etc.) needs to be verified in a full system test The hardware-specific lane configuration (pi-bonsai then pi-hexagon) needs to be confirmed in the actual deployment

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Directory Coverage Check The guard now checks that every MTP record in the directory is covered by the guard, preventing new records from being ignored. This prevents a regression where a new record added to the directory would be silently ignored by the check. on_disk = sorted(f[:-5] for f in os.listdir(RECORDS) if f.startswith("27b-mtp-") and f.endswith(".json")) check(len(on_disk) >= len(derived), f"the guard's record directory is readable ({len(on_disk)} MTP record(s))") uncovered = [rid for rid in on_disk if rid not in derived] check(not uncovered, f"every MTP record on disk is checked by this guard: uncovered {uncovered}")