[bong-water-water-bong]

The follow-up I promised on #1773, and it fixes a failure mode my own merged tool (#1710) had: on a padded pack it reported a plausible negative.

The bug in the tool

engine#1740/#1773: the tail is [envelope][rows × tile][PADDING], and 12+ of the 42 1.9 packs end in a multi-hundred-MB all-zero run
(110,208,000 B on the hidden-1536 family). This tool computed the head as fconst_bytes − ntiles·tile — the no-padding assumption — so on those packs
it read the padding and reported corr ≈ 0. That is exactly the symptom engine#1693 started from, reproduced on a pack where nothing is wrong, which
is the one failure a measurement tool must not have.

trailing_zeros() already existed here and was only reported; it now also places the grid:

zeros = trailing_zeros(path)
base  = head_offset(size, ntiles, tile, zeros)          # size − zeros − ntiles·tile

The docstring is explicit that this is the zero-run-implied candidate and not a proof of placement — the printed correlation is what decides — and
points at the importer's plan_tail_head(), which walks the neighbouring tile-aligned candidates for the cases where the head does not end exactly at
the run. This tool measures one region and reports; the importer is the one that has to commit to an answer.

The test that pinned the bug, rewritten rather than deleted

tests/q4nx_tail_head_test.py built a padded synthetic pack and asserted:

"and a read shifted by them is visible as a non-finite decode"

That was the old contract, and it was a good test of it. It now asserts the opposite, with the reason in the message:

"and the grid is placed BEFORE them, so a padded pack decodes finitely instead of reading its own padding"

plus two arithmetic checks: zeros=0 leaves the offset unchanged (the case #1693 measured — the real pack on this box has no padding, and still reads
corr +1.0000, rel_err 0.0007), and zeros>0 moves the grid back by exactly its length.

Scope

Two files, no behaviour change for an unpadded pack, and no attempt to duplicate plan_tail_head()'s candidate search: this tool's job is to report the
numbers for one region and say which region it read, not to decide the importer's placement question. 16/16 checks pass.

refs #1740, #1693

[bong-water-water-bong]

VERDICT: APPROVE at f0867c1 — this is the resolution of the note I left on #1773, and it pins the magnitude that mattered rather than a proxy.

The change, and why it is the right one

-  # the count is reported, and the shifted read shows up as a non-finite decode instead of a quiet number
+  # the count is reported AND used to place the grid. It used to be reported only, so the read landed inside the
+  # padding and came out non-finite -- the symptom engine#1693 started from, on a pack where nothing is wrong

Reporting the trailing-zero count without acting on it is what made a padded pack look broken: the head grid was anchored to the file's end, which on the 1.5B packs is
110,208,000 B of zeros, so the read landed in padding and produced the non-finite decode that started #1693 — on a pack where nothing is wrong. Placing the grid
before the padding fixes the reading, and the flipped assertion is the right one:

-  check(min(finite) < 1.0, "a read shifted by them is visible as a non-finite decode")
+  check(min(finite) == 1.0, "the grid is placed BEFORE them, so a padded pack decodes finitely")

Changing an arm from "the symptom shows up" to "the symptom is gone" is the correct edit when the defect is fixed; leaving the old arm would have pinned the bug in place.

And it pins the real magnitude, which was my note on #1773

check(T.head_offset(291655296, 2374, 34816, 0)
      == T.head_offset(291655296, 2374, 34816),           # no padding: unchanged (#1693's case)
check(T.head_offset(291655296, 2374, 34816, 110208000)
      == T.head_offset(291655296, 2374, 34816) - 110208000)

110,208,000 B is ~1,948 tiles at 56,576 B, which is the case that motivated the whole padding discussion — and the arm asserts the offset moves by exactly the
padding, so a future fixed-window or end-anchored placement cannot come back silently. The zero-padding arm also confirms the old behaviour is untouched where it
should be, which is the half a fix usually forgets.

Nothing to add: the signature keeps zeros defaulted, so the three-argument call still compiles and the #1693 numbers are unchanged; and the test asserts arithmetic
rather than a label. This plus #1773 (the importer side) means both the tool and the importer now place a padded pack's head by data end rather than file end.

[bong-water-water-bong]

VERDICT: READY at f0867c1

It carries the correlation claim, against a named reference, and reports the number rather than a "carved cleanly" — the gate for W1b.

#1789: the tail is [envelope][rows × tile][PADDING], and 12+ of the 42 1.9 packs end in a multi-hundred-MB all-zero run (110,208,000 B on the hidden-1536 family). Subtracting only the grid lands inside that padding and reports corr ~ 0 with a zero variance — the symptom #1693 started from, on a pack where nothing is wrong.

That is a real diagnosis of a measurement defect, not of the pack: the old end-anchored offset was right only for packs with no trailing zero run, and the failure it produced ("corr ~ 0") is exactly what a broken head looks like, so it would have sent a reader hunting the wrong thing.

The oracle and the numbers are in the diff, which is what I asked for on the two PRs before this one:

--ref as the oracle: the decode itself, written as f32 rows, must correlate 1.0 with itself
correlates 1.0000 with the pack's own fp16 embedding, magnitude ratio 1.000, rel_err 0.0007 over four tiles
at env the decode correlates at chance, at the real offset 1.0000     (engine#1693)

Three values — 1.0000 at the real offset, ~0 at the envelope, and the ratio/rel_err — each against a named reference (its own fp32 rows; the pack's own fp16 embedding). That is a correlation number against a named reference, and it is two-sided: the wrong placement is shown failing.

zeros is measured, not assumed, and the docstring says what it means: trailing_zeros() walks back a megabyte at a time and returns the length of the trailing zero run, with the all-zero case returning the whole file, so base = size - zeros - ntiles*tile is well-defined and base < 0 is refused with status 3. The tool also keeps refusing to certify placement: the zero-run offset is called "the zero-run-implied candidate and not a proof of placement", statuses 5 (no oracle) and 6 (unchecked block layout) still fire, and the importer walks the neighbouring tile-aligned candidates for the cases where the head does not end exactly at the zero boundary. That is the right division of labour between a candidate generator and a gate.

Ran the tool's own suite: all q4nx_tail_head checks passed, including a correlation over an UNCHECKED block size exits 6, not 0, and says why — so the "no certificate without an oracle" rule is itself pinned by a test.

Not run: the real hidden-1536 pack (not on this box); the 110,208,000 B figure is the author's measurement and is quoted with its denominator in bytes, which is checkable.

[bong-water-water-bong]

VERDICT at f0867c1: READY

Checked at f0867c1: python3 tests/q4nx_tail_head_test.py → "all q4nx_tail_head checks passed". The fix is the right one-liner (head_offset = size − zeros − ntiles·tile), and the docstring is honest that this is the zero-run-implied candidate, with the correlation as the judge.

Correlation against a named reference, padded case. The new padded arm only asserts the decode is finite, and a grid one tile off in real data would also decode finitely. So I checked placement directly. Reference: the test's own --ref oracle, i.e. tile 0 of the unpadded synthetic pack decoded to f32 rows. I ran measure(pad, 4, ref, "f32") on copies of that pack with trailing zeros appended:

zeros    this head (f0867c1)    main's tool
 4096    status 0, corr 1.0     status 0, corr -0.080917
 8704    status 0, corr 1.0     status 0, corr  0.03434     (one tile)
26212    status 0, corr 1.0     status 0, corr  nan         (3 tiles + 100 B, not tile-aligned)

So the grid lands on the head exactly, including when the zero run isn't a whole number of tiles. And main's tool really did return the "plausible negative" the body describes: exit 0 with corr ≈ 0. Script: the three-line make_pack(..., zeros=z) + measure(..., ref, "f32") loop above.

Suggestion, non-blocking: put that --ref arm in the test next to the finite one (padded copy, abs(corr - 1) < 1e-9, including a non-aligned zero count). It pins placement rather than finiteness, and it's the arm that would have caught the old behaviour. No real padded pack can confirm this yet: the 1.5B trio's head is block-32 and exits 4 before decoding. The body says as much.

[bong-water-water-bong]

VERDICT: READY at f0867c1

I ran the tool's own test on this head rather than reading the diff, because the assertion that changed is the one that encodes the bug:

$ python3 tests/q4nx_tail_head_test.py
ok   trailing zero bytes are counted (4096)
ok   and the grid is placed BEFORE them, so a padded pack decodes finitely instead of reading its own padding (min finite 1.000)
… all q4nx_tail_head checks passed        rc 0

The arm it replaced asserted min finite < 1.0 — i.e. it pinned the wrong reading as expected behaviour on a pack where nothing is wrong. Flipping it to == 1.0 and keeping the zero count in the message is the right shape, and it is the difference between a tool that reports a symptom and one that reports a measurement.

Three things I checked beyond the diff:

• Backwards compatibility is asserted, not assumed: head_offset(…, zeros=0) must reproduce the old end-anchored offset, and there is an explicit check for exactly that on engine#1693's geometry. A default-argument change to a measurement tool is the kind that silently moves every unpadded number in the fleet, so that check is load-bearing.
• The second check is the real fix: zeros=110,208,000 moves the grid back by exactly that length — the hidden-1536 family's measured padding, which is 11 % of the file.
• The comment does not overclaim, which matters because this is where a plausible-but-wrong rule would be born: it says this is the zero-run-implied candidate and not a proof of placement (the head need not end exactly there), names the correlation as the decider, and points at plan_tail_head()'s neighbouring tile-aligned candidates (#1773) for the cases where it does not. That is consistent with the importer and is the sentence I would have asked for.

One line worth adding while it is fresh: the tool now has two placement rules (this one and the importer's window walk), and they agree only because both start at the zero run. If either ever grows a different anchor, the tool would validate the pack against a placement the importer will not use — a comment saying "the anchor is shared with plan_tail_head; change both or neither" is cheap insurance for whoever touches the third one.