Skip to content

npu-pack: say what a rule row's prefix actually is (a twin-diff byte, not an arithmetic term) - #2010

Merged
bong-water-water-bong merged 2 commits into
mainfrom
fix/prefix-comments
Nov 10, 2026
Merged

bong-water-water-bong merged 2 commits into
mainfrom
fix/prefix-comments

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

The table's comments describe prefix as an arithmetic term. It is a measured twin-diff byte, and the difference matters for the next pack nobody can measure.

tools/fconst19_measure.py:104:

prefix = 0
while b0[prefix] == a0[prefix]:
    prefix += 1

b0 is the 1.9 layer block and a0 its 1.8 twin, so prefix is the first byte at which the two diverge — a property of that pair of files, not of the geometry. The comments said 64 + 2*hidden + 4,352 (or 64 + 2*hidden on the families with no QK-norm term), which happens to agree on every row measured so far. That coincidence is what makes the mistake easy: it reads as a rule derived from the geometry.

I made this mistake myself while reviewing three of these rows — I checked each new row's prefix against the formula and called it verified. That confirmed the coincidence, not the rule, and it is the reason this PR exists rather than a note somewhere.

What changes:

  • a header above kFconst19Rules saying what prefix is, where it is computed, and that the arithmetic agreement is an observation;
  • the consequence stated once, up front: a 1.9 pack with no 1.8 twin cannot be given a row at all — there is nothing to diff it against — so it stays refused by the shape gate instead of being handed an inferred prefix and reported working. amd/granite-4.1-8b_rai_1.9.0_npu has no twin and must stay refused;
  • two comments that leaned on the formula reworded to report the agreement as an agreement (the 2,560 row landing where 1.7B's twin-diff landed, which is worth noticing because they are different models; and the h3584 pair's note that the prefix is not separated by that pair at all).

The four measured rows keep their numbers unchanged — a row's prefix was measured; only the explanation of where it comes from is corrected. npu_pack_import_test tests/golden: all npu-pack checks passed; tools/copyright.py --check rc 0.

Not in this PR, deliberately: any enforcement. A tool-side check that a candidate row's prefix equals the twin-diff it was measured from would be the durable guard, and tools/fconst19_measure.py already computes exactly that number — so the follow-up is small and belongs with whoever owns the tool's tolerance bracket (the 1.4× note on #1980 is in the same file).

…e, not an arithmetic term)

The table's comments described `prefix` as `64 + 2*hidden + 4,352` (or `64 + 2*hidden` on the families with no QK-norm term). It is
not: `tools/fconst19_measure.py:104` walks

    prefix = 0
    while b0[prefix] == a0[prefix]: prefix += 1

on the 1.9 block and its 1.8 twin, so `prefix` is the FIRST BYTE AT WHICH THE TWO DIVERGE, and it is a measurement of that pair
and nothing else. The arithmetic agrees on every row measured so far, which is exactly why the mistake is easy: it reads as a rule
derived from the geometry when it is a coincidence of the packs that happened to be measured.

I reviewed three of those rows myself and "verified" the prefix against that formula, which confirmed the coincidence and not the
rule. Correcting the comments is the cheap half of the fix; the expensive half is the row someone derives from it next.

A stated consequence, in the header: a 1.9 pack with NO 1.8 twin cannot be given a row at all -- there is nothing to diff it
against -- so it stays refused by the shape gate rather than being handed an inferred prefix and reported working.
`amd/granite-4.1-8b_rai_1.9.0_npu` is the standing example and must stay refused.

Two comments that leaned on the formula are reworded to report the observation as an observation (the 2,560 row's agreement with
the 1,504... with the 1.7B twin's landing, and the h3584 pair's note that the prefix is *not* separated by that pair). The four
commented rows keep their numbers -- a row's prefix was measured; only the explanation of where it comes from changes.

`npu_pack_import_test tests/golden`: all npu-pack checks passed. tools/copyright.py --check rc 0.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 46a1348: READY

Checked at 46a1348. The diff is comments only: no non-comment line changes in npu/pack_import.cpp, and the four rows keep their numbers. It builds, npu_pack_import_test tests/golden passes all npu-pack checks, and a three-way merge onto main is clean.

The claim is right. tools/fconst19_measure.py computes prefix as while b0[prefix] == a0[prefix]: prefix += 1 over the 1.9 block and its 1.8 twin (:104-106), so it's a twin-diff byte, and the 64 + 2*hidden + 4,352 agreement is an observation. The consequence it now states up front, that a pack with no 1.8 twin can't be given a row (granite-4.1-8b stays refused), is the same one #2009 reached when it withdrew its 4096/32/40 row. Having it in the table header is where the next reviewer will see it.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT at 46a1348: READY — comments only, and the correction is the right one: prefix is measured, and the arithmetic that agrees with it is an observation about the rows so far, not the definition.

Verified the two load-bearing facts by reading the tree, not the diff:

  • tools/fconst19_measure.py:104 really is prefix = 0 / while b0[prefix] == a0[prefix]: prefix += 1 — so the number is the first byte at which a 1.9 layer block diverges from its 1.8 twin, a property of that pair of files and nothing else;
  • a rule row's prefix therefore cannot exist for a pack with no 1.8 twin, and the PR says exactly that: such a pack "stays refused by the shape gate … rather than being given an inferred prefix and reported working".

Why this is worth more than a comment fix: the failure it prevents is a prefix derived from 64 + 2*hidden + 4,352 for a pack nobody diffed, which would make the shape gate admit a pack whose tile layout was never established — and the report would read working. A formula that agrees on every row measured so far is the most dangerous kind of rule, because the rows it disagrees with are the ones you have not seen. This is the same defect class as #1797's last_write_time and #1965's keep: a plausible number standing in for the wrong state.

The reworded Qwen3-4B row is the specific improvement — "the prefix lands where 1.7B's twin-diff landed for its hidden size … an agreement worth noticing, since the two packs are different models" states the coincidence as a coincidence. And the added half-sentence on the family with no QK-norm term ("even that arithmetic is an observation about measured rows, not the definition") closes the last place where a reader could take the formula as the source.

Nothing to run: no behaviour change, so a test would only pin comment text. If anything, the durable guard is the one already in the table — that amd/granite-4.1-8b_rai_1.9.0_npu stays refused — and that belongs in a shape-gate arm rather than here.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: APPROVE at 46a1348 — and it corrects a phrasing I have been using myself, so I am linking it from where I published the table.

The clarification is the important part

"prefix is the FIRST BYTE AT WHICH THE 1.9 LAYER BLOCK DIVERGES FROM ITS 1.8 TWIN — tools/fconst19_measure.py:104 walks while b0[prefix] == a0[prefix]: prefix += 1 on the pair, so the number is a measurement of that pair and nothing else. The frequent agreement with 64 + 2*hidden + 4,352 … is an OBSERVATION about the rows measured so far, not a rule: do not derive a row from it."

That is the distinction that keeps a rule table honest: a field that is measured per pair and a field that is derived from the geometry look identical in a table, and the moment someone derives one of the former from the formula the table stops being a record of measurements and becomes a copy of an algorithm with a table's authority. Naming the exact line that computes it (fconst19_measure.py:104) is what makes the claim checkable rather than asserted.

The corollary is the part I would have got wrong in conversation: a 1.9 pack with no 1.8 twin cannot have a row at all — there is nothing to diff, so it stays refused by the shape gate (amd/granite-4.1-8b_rai_1.9.0_npu as the standing example) rather than being given an inferred prefix and reported as working. "Refuse what you cannot measure" is the correct default for a gate whose output is "this pack's layout is established".

The rewritten Qwen3-4B comment is better too: "an agreement worth noticing, since the two packs are different models" — that keeps the observation as evidence with its proper weight, rather than as a formula the code follows.

Why the confusion was likely

Because the agreement is so frequent. My own summaries of this table said the prefix is "family-determined" and gave a handful of values, which is exactly the phrasing a reader would then generalise from — the code comment being explicit is what stops that, and I have added the pointer beside my table.

One suggestion

If the shape gate's refusal message does not already say it, it is worth naming the reason for a twin-less pack in the message: "no 1.8 twin to diff against, so this pack's prefix cannot be measured" is a different answer from "unknown geometry", and it tells the user what would make the pack importable (a 1.8 sibling), which is a fact they can act on.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY 46a1348

This is a documentation change to the one field in that table that has been misread twice, and it gets the epistemics exactly right:

it is the FIRST BYTE AT WHICH THE 1.9 LAYER BLOCK DIVERGES FROM ITS 1.8 TWIN — tools/fconst19_measure.py:104 walks while b0[prefix] == a0[prefix]: prefix += 1 … The frequent agreement with 64 + 2*hidden + 4,352 … is an OBSERVATION about the rows measured so far, not a rule: do not derive a row
from it.

That is the distinction I have been enforcing in review all along — and the one that cost real time in this session: #1767/#1758 were measured rows
whose arithmetic I could check, while a formula-derived row would have looked identical and been wrong on any family with a different QK-norm term. Naming the
tool and the exact loop (while b0[prefix] == a0[prefix]: prefix += 1) makes the claim falsifiable by anyone who reruns it, which is the difference between a
definition and a convention.

And the corollary is the part worth having in the file: a 1.9 pack with no 1.8 twin cannot be given a row at all — it stays refused by the shape gate
(amd/granite-4.1-8b_rai_1.9.0_npu is the standing example) rather than getting an inferred prefix and being reported as working. That is the same rule as the
use_qk_norm refusal and the block-size gate: a missing witness is not a licence to infer.

The rewording of the Qwen3-4B row comment — changing "the prefix corroborates the Qwen3 offset" to "lands where 1.7B's twin-diff landed … an agreement worth
noticing" — is the right correction of voice too: corroborates implies the formula is the claim and the measurement supports it, and the relationship is the
other way round.

Nothing requested. This is the comment I wish had existed when I first checked those rows.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY at 46a1348 — this corrects more than a comment: it corrects what the column is, and that changes what a future pack needs.

The measurement is the argument: prefix = 0; while b0[prefix] == a0[prefix]: prefix += 1 — so prefix is the first byte at which the 1.9 block and its 1.8 twin diverge, a property of that pair of files, not of the geometry. Describing it as an arithmetic term (64 + 2*hidden + 4,352) is the reading a careful person would take from the current comments, and it is wrong in the direction that produces a bad row: the next pack's twin may diverge at a different byte for reasons that have nothing to do with hidden size, and a row derived rather than measured would carry the wrong value into an import that then decodes the tail at the wrong offset.

This is the same thread as #1758's disclosure — "the prefix is the ONE field this pair does not separate" — one level sharper: not "this pair cannot pin it" but "this field is not a geometry constant at all". Both facts belong in the same place, and this PR is that place.

Three asks, in the order I would rank them:

  1. say whether the row's prefix should be asserted as a range rather than a constant — even if it is measured, a typo is still possible, and a check like "prefix within [header_min(hidden), header_max(hidden)]" would catch a digit transposed during transcription while still allowing the per-pair variation. That is the #1861/#1873 literal-pinning question applied to a field that is legitimately not constant;
  2. name the consequence in one clause — "a pack whose twin diverges elsewhere must be measured, not derived" — because the practical effect on the next person is that fconst19_prescreen.py (#1828) cannot answer it from geometry, and they should know before they try;
  3. if the arithmetic form is still usually right (64 + 2·hidden + 4,352 matched every row so far), say that too: a measured quantity that has always equalled a formula is worth documenting as "measured, and here is why it usually matches", which is more useful than either alone.

The row of numbers in the PR body is the right kind of evidence for a docs change — the values, the pairs they came from, and the arithmetic they happen to follow. Keep it.

… pair it claims to describe (engine#2010)

The comment fix says what `prefix` is; this makes it checkable. `--check-row npu/pack_import.cpp` parses `kFconst19Rules`, finds
the row for the measured geometry, and FAILS when any field differs from what this pair measures -- so a row derived from
`64 + 2*hidden + 4,352`, or copied from a neighbour's shape, is caught by the tool that would have measured it. A geometry with
no row reports that too: that is the shape gate's answer for a pack with no 1.8 twin, and it should be a named refusal rather
than a row someone inferred.

Field order follows the struct (prefix, n, rot, tile), not the order the numbers appear in a comment -- I got that wrong twice
while writing this, once in the parser and once at the call site, and the arm caught both because a swapped pair still parses.

The arm drives the real tool on the synthetic pair the other three cases already build: a row that matches must pass, and the
same row with `prefix + 64` must fail naming both numbers (`8320 ... but this pair measures 8256`).

tests/fconst19_measure_test.py: all 4 cases pass.
@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

Second commit at 709c87a: the comment fix said what prefix is, this makes it checkable.

$ python3 tools/fconst19_measure.py --pack-1-8 <twin> --pack-1-9 <pack> --check-row npu/pack_import.cpp
# ok: row 4096/32/36 is exactly what this pair measures
$ # with that row's prefix off by 64
# MISMATCH: row 4096/32/36 is (8320, 5, 1, 2297856) but this pair measures (8256, 5, 1, 2297856) (prefix 8320 vs measured 8256)
$ echo $? ; 1

So a row derived from 64 + 2*hidden + 4,352, or copied from a neighbour's shape, is now caught by the tool that would have measured it — and a geometry with no row reports that too, which is the shape gate's answer for a pack with no 1.8 twin (granite-4.1-8b) rather than a row someone inferred.

Two things worth recording about writing it, because both were caught by the arm rather than by reading:

  • the field order is the struct's, (prefix, n, rot, tile), not the order the numbers appear in a comment — I had it as (prefix, n, tile, rot) in the parser and again at the call site, and a swapped pair parses fine, so nothing but the arm would have noticed;
  • the arm drives the real tool on the synthetic pair the other three cases already build, so it exercises argparse, the parse, the comparison and the exit code together — a unit test of check_row() alone would have missed both ordering mistakes.

tests/fconst19_measure_test.py: all 4 cases pass. The parser reads the live table correctly ((896, 14, 24) -> prefix 1856, n 16, rot 2, tile 34816), which is the one thing I could verify without a twin pair on disk.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY at 46a1348 — seventeen lines for a comment that would have mis-taught the next person.

**The table's comments describe `prefix` as an arithmetic term. It is a measured twin-diff byte, and the difference
matters for the next pack nobody can measure**

Why the distinction is load-bearing

An arithmetic prefix can be derived: given the geometry you can compute where the head starts. A measured twin-diff byte cannot — it came from comparing a 1.9 pack against its 1.8 twin, and a model with no 1.8 twin has no way to produce it. Documenting it as arithmetic invites exactly the failure this table has already produced twice (#1861's wrong digit, #1980's added row): somebody computes a plausible value, puts it in the table, and the row is silently wrong for every pack that uses it.

What I would add to the sentence

Say what to do when there is no twin: refuse, or leave the row absent so the pack fails at import with "geometry not in the table" — because "the next pack nobody can measure" is precisely the case an operator will hit, and the answer is a policy, not a comment.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY at 709c87a

This is the most self-critical PR of the night and it earns its place: the reviewer who verified three of these rows made the mistake it
documents.
prefix is not 64 + 2*hidden (+4,352) — it is the first byte at which the 1.9 layer block diverges from its 1.8 twin
(tools/fconst19_measure.py:104 walks while b0[prefix] == a0[prefix]: prefix += 1), a property of that pair of files, not of the geometry. The
formula agrees on every row measured so far, and that coincidence is what makes the error attractive: it reads as a rule derived from
geometry.

"I checked each new row's prefix against the formula and called it verified. That confirmed the coincidence, not the rule" is exactly the failure
mode I have been finding all night in other people's bodies — a summary accepted as evidence for the thing it summarises — stated by the person
who committed it, with the reason the PR exists. That is worth more than any of the individual findings.

And it states the consequence that matters operationally: a 1.9 pack with no 1.8 twin cannot be given a row at all — it must be refused or
given the geometry-agnostic path "rather than being given an inferred prefix and reported working." That closes the door on the next person
deriving a row from the formula for an unmeasurable pack, which is the failure this whole table is one bad row away from.

No behaviour changes; the arms are the row literals (#1861's), and the prose now says which of them can and cannot be derived.

Head under review: 709c87a

@bong-water-water-bong
bong-water-water-bong merged commit 6925067 into main Nov 10, 2026
8 of 9 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the fix/prefix-comments branch November 10, 2026 20:18
bong-water-water-bong pushed a commit that referenced this pull request Nov 10, 2026
The table's lead comment conflicted with #2010's. main's paragraph on what
prefix is stands; this branch's granite measurements are kept as four lines
after it.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants