From 46a13480dcae6580e408a318af3a70756dce4e5a Mon Sep 17 00:00:00 2001 From: bong-water-water-bong Date: Tue, 10 Nov 2026 15:39:06 -0400 Subject: [PATCH 1/2] npu-pack: say what a rule row's `prefix` actually is (a twin-diff byte, not an arithmetic term) The table's comments described `prefix` as `64 + 2*hidden + 4,352` (or `64 + 2*hidden` on the families with no QK-norm term). It is not: `tools/fconst19_measure.py:104` walks prefix = 0 while b0[prefix] == a0[prefix]: prefix += 1 on the 1.9 block and its 1.8 twin, so `prefix` is the FIRST BYTE AT WHICH THE TWO DIVERGE, and it is a measurement of that pair and nothing else. The arithmetic agrees on every row measured so far, which is exactly why the mistake is easy: it reads as a rule derived from the geometry when it is a coincidence of the packs that happened to be measured. I reviewed three of those rows myself and "verified" the prefix against that formula, which confirmed the coincidence and not the rule. Correcting the comments is the cheap half of the fix; the expensive half is the row someone derives from it next. A stated consequence, in the header: a 1.9 pack with NO 1.8 twin cannot be given a row at all -- there is nothing to diff it against -- so it stays refused by the shape gate rather than being handed an inferred prefix and reported working. `amd/granite-4.1-8b_rai_1.9.0_npu` is the standing example and must stay refused. Two comments that leaned on the formula are reworded to report the observation as an observation (the 2,560 row's agreement with the 1,504... with the 1.7B twin's landing, and the h3584 pair's note that the prefix is *not* separated by that pair). The four commented rows keep their numbers -- a row's prefix was measured; only the explanation of where it comes from changes. `npu_pack_import_test tests/golden`: all npu-pack checks passed. tools/copyright.py --check rc 0. --- npu/pack_import.cpp | 25 +++++++++++++++++-------- 1 file changed, 17 insertions(+), 8 deletions(-) diff --git a/npu/pack_import.cpp b/npu/pack_import.cpp index 02fe8c996..fcb1da214 100644 --- a/npu/pack_import.cpp +++ b/npu/pack_import.cpp @@ -380,6 +380,13 @@ constexpr double kTieCorr = 0.99; // Qwen3's per-head QK norm gains, but Llama-3.2-1B -- which has no QK norms at all -- carries the same +4,352 at hidden // 2,048, so the extra is a property of the geometry, not of the architecture. Measured anyway, because a plausible // explanation that survives one model is how this table would get a wrong row. +// WHAT `prefix` IS, because it has been misunderstood twice (engine#2010): it is the FIRST BYTE AT WHICH THE 1.9 LAYER +// BLOCK DIVERGES FROM ITS 1.8 TWIN -- `tools/fconst19_measure.py:104` walks `while b0[prefix] == a0[prefix]: prefix += 1` +// on the pair, so the number is a measurement of that pair and nothing else. The frequent agreement with +// `64 + 2*hidden + 4,352` (or `64 + 2*hidden` on the families with no QK-norm term) is an OBSERVATION about the rows +// measured so far, not a rule: do not derive a row from it. A 1.9 pack with NO 1.8 twin cannot be given a row at all -- +// there is nothing to diff it against, so it stays refused by the shape gate (amd/granite-4.1-8b_rai_1.9.0_npu is the +// standing example) rather than being given an inferred prefix and reported working. constexpr Fconst19Rule kFconst19Rules[] = { // Qwen2.5-0.5B-Instruct: normalises to its 1.8 twin byte for byte, whole file, all 24 blocks. {896, 14, 24, 1856, 16, 2, 34816}, @@ -401,9 +408,10 @@ constexpr Fconst19Rule kFconst19Rules[] = { // Qwen-2.5_1.5B_Instruct measured to the SAME bundle and rotation as the Coder pack -- byte for byte over all 28 // layer blocks -- so the two packs share one row now that the key is geometry, not the pack name. {1536, 12, 28, 3136, 7, 1, 226304}, - // Qwen3-4B: 2,933 of 2,208,061,248 bytes differ (1.3e-6), again content not layout. The prefix corroborates the - // Qwen3 offset seen at 1.7B -- 64 + 2*hidden + 4,352, so 8,512 at hidden 2,048 and 9,536 at 2,560 -- and the tile - // count differs from 1.7B's 3, which is why this is measured per pack and never derived. + // Qwen3-4B: 2,933 of 2,208,061,248 bytes differ (1.3e-6), again content not layout. The prefix lands where 1.7B's + // twin-diff landed for its hidden size (8,512 at 2,048 here, 9,536 at 2,560) -- an agreement worth noticing, since the + // two packs are different models -- and the tile count differs from 1.7B's 3, which is why this is measured per pack and + // never derived. {2560, 32, 36, 9536, 5, 1, 1462272}, // Llama-3.2-1B-Instruct: hidden 2,048 with 32/16 heads and no QK norms -- the first non-Qwen family here. Same // prefix as Qwen3-1.7B at this hidden size (8,512), a different bundle (5 tiles of 591,872, not 3 of 1,183,744), @@ -427,9 +435,10 @@ constexpr Fconst19Rule kFconst19Rules[] = { // 28,271. // The prefix is the ONE field this pair does not separate: `prefix+64` differs in 33,291 bytes, which is under the tool's // 38,306-byte tolerance -- so `prefix = 7,232` here rests on the two models agreeing and on the plain Qwen2.5 arithmetic - // (64 + 2*hidden, with no QK-norm term, unlike the Qwen3 line's +4,352), not on the pair alone. Said plainly because a row - // whose comment implies all seven fields were equally pinned would be the same kind of overstatement this table exists to - // avoid. + // (64 + 2*hidden, with no QK-norm term, unlike the Qwen3 line's +4,352), not on the pair alone -- and note that even that + // arithmetic is an observation about measured rows, not the definition: `prefix` is where the twins first diverge. Said + // plainly because a row whose comment implies all seven fields were equally pinned would be the same kind of + // overstatement this table exists to avoid. // The block's arithmetic, from the measurement rather than from a formula: prefix 7,232 + 9 x tile 1,009,664 = 9,094,208, // which is exactly the `header_off` the tool reports for this pair -- so the payload after the prefix is NINE tile slots, // of which the rule rotates eight and the ninth is the gap the post-attention header sits behind. (An earlier version of @@ -445,7 +454,7 @@ constexpr Fconst19Rule kFconst19Rules[] = { {2048, 16, 36, 4160, 9, 1, 295936}, // Mistral-7B-Instruct-v0.2 at hidden 4,096 / 32 heads / 32 layers: measured with tools/fconst19_measure.py against // amd/Mistral-7B-Instruct-v0.2_rai_1.8.0_npu_4K, 1,954 of 3,843,419,200 bytes differ (5.08e-05) -- content, not - // layout. The prefix follows the Qwen3 line (64 + 2*hidden + 4,352 = 12,608), the bundle is 5 tiles of 2,297,856 + // layout. The prefix is 12,608 (the first twin-diff byte), the bundle is 5 tiles of 2,297,856 // (11,489,280 B, which is NOT header_off minus the prefix: the header sits after the prefix and the bundle follows // it, so this row is one the arithmetic would have got wrong), and the header stride 117,862,720 is uniform over // all 32 blocks. This is the largest family on #1503 (31 packs). Measured by another lane on main, whose @@ -453,7 +462,7 @@ constexpr Fconst19Rule kFconst19Rules[] = { {4096, 32, 32, 12608, 5, 1, 2297856}, // Llama-3.2-3B (and -Instruct, same geometry) at hidden 3,072 / 24 heads / 28 layers: measured with // tools/fconst19_measure.py against amd/Llama-3.2-3B_rai_1.8.0_npu_4K, 1,090 of 1,756,930,368 bytes differ - // (6.20e-07 of the file, which the tool prints as 6.20e-05 %) -- content, not layout. The prefix is the Qwen3 line's 64 + 2*hidden + 4,352 = 10,560 at this hidden + // (6.20e-07 of the file, which the tool prints as 6.20e-05 %) -- content, not layout. The prefix is 10,560 here (the first twin-diff byte at this hidden // size, but the bundle is FOUR tiles of 1,740,800 (6,963,200 B) where the Phi/h3072 row has two of 5,222,400 -- // which is the whole reason a row is measured per geometry and never derived from the hidden size. {3072, 24, 28, 10560, 4, 1, 1740800}, From 709c87ad4f05c10717662d7169df16b2c047f4f8 Mon Sep 17 00:00:00 2001 From: bong-water-water-bong Date: Tue, 10 Nov 2026 16:01:10 -0400 Subject: [PATCH 2/2] fconst19_measure: --check-row, so a table row can be held against the pair it claims to describe (engine#2010) The comment fix says what `prefix` is; this makes it checkable. `--check-row npu/pack_import.cpp` parses `kFconst19Rules`, finds the row for the measured geometry, and FAILS when any field differs from what this pair measures -- so a row derived from `64 + 2*hidden + 4,352`, or copied from a neighbour's shape, is caught by the tool that would have measured it. A geometry with no row reports that too: that is the shape gate's answer for a pack with no 1.8 twin, and it should be a named refusal rather than a row someone inferred. Field order follows the struct (prefix, n, rot, tile), not the order the numbers appear in a comment -- I got that wrong twice while writing this, once in the parser and once at the call site, and the arm caught both because a swapped pair still parses. The arm drives the real tool on the synthetic pair the other three cases already build: a row that matches must pass, and the same row with `prefix + 64` must fail naming both numbers (`8320 ... but this pair measures 8256`). tests/fconst19_measure_test.py: all 4 cases pass. --- tests/fconst19_measure_test.py | 33 +++++++++++++++++++++++++++- tools/fconst19_measure.py | 40 ++++++++++++++++++++++++++++++++++ 2 files changed, 72 insertions(+), 1 deletion(-) diff --git a/tests/fconst19_measure_test.py b/tests/fconst19_measure_test.py index beb7a4f88..2fa505bae 100755 --- a/tests/fconst19_measure_test.py +++ b/tests/fconst19_measure_test.py @@ -139,12 +139,43 @@ def case_refuse(root): return ok, f"rc={rc} err={err.strip().splitlines()[-1][:90] if err.strip() else ''!r}" + +def case_check_row(root): + """engine#2010: a row must BE the measurement. `prefix` is the first byte where the twins diverge, so a row + whose prefix came from `64 + 2*hidden + 4,352` (or from a neighbour's shape) is wrong in a way no geometry + check sees. The guard is `--check-row `; here the source file is a two-line stand-in.""" + a = b"".join(block_18(L) for L in range(LAYERS)) + b"TAIL" * 16 + b = b"".join(block_19(L) for L in range(LAYERS)) + b"TAIL" * 16 + p18 = write_pack(root, "p18", a) + p19 = write_pack(root, "p19", b, declared_18=False) + rc, out, err = run((p18, p19), ("--json",)) + if rc != 0: + return False, f"the pair itself did not measure: rc={rc} {err.strip()[:80]}" + row = json.loads([l for l in out.splitlines() if l.startswith("{")][0]) + good = (f" {{{row['hidden']}, {row['heads']}, {row['layers']}, {row['prefix']}, {row['n_tiles']}, " + f"{row['rot']}, {row['tile']}}},") # the struct's order: prefix, n, rot, tile + src = os.path.join(root, "good.cpp") + with open(src, "w") as f: + f.write("constexpr Fconst19Rule kFconst19Rules[] = {\n" + good + "\n};\n") + rc1, out1, err1 = run((p18, p19), ("--check-row", src)) + # and a row whose prefix is off by one tile-length, which is what a derived prefix looks like + bad = good.replace(f", {row['prefix']},", f", {row['prefix'] + 64},", 1) + src2 = os.path.join(root, "bad.cpp") + with open(src2, "w") as f: + f.write("constexpr Fconst19Rule kFconst19Rules[] = {\n" + bad + "\n};\n") + rc2, out2, err2 = run((p18, p19), ("--check-row", src2)) + ok = rc1 == 0 and rc2 == 1 and "is exactly what this pair measures" in (out1 + err1) \ + and "MISMATCH" in (out2 + err2) and f"prefix {row['prefix'] + 64} vs measured" in (out2 + err2) + return ok, f"match rc={rc1}; mismatch rc={rc2}; detail={(out2 + err2).strip()[-90:]!r}" + + def main(): root = tempfile.mkdtemp(prefix="fconst19-test.") try: results = [("exact pair (and the refusal path executes)", case_exact(root)), ("a few hundred codes differ: accepted with residual", case_tolerance(root)), - ("a scrambled tile: refused", case_refuse(root))] + ("a scrambled tile: refused", case_refuse(root)), + ("--check-row: the table's row must be this pair's measurement", case_check_row(root))] finally: shutil.rmtree(root, ignore_errors=True) bad = [n for n, (ok, _) in results if not ok] diff --git a/tools/fconst19_measure.py b/tools/fconst19_measure.py index 386b1b14f..49c6c5658 100755 --- a/tools/fconst19_measure.py +++ b/tools/fconst19_measure.py @@ -37,6 +37,7 @@ """ import argparse import json +import re import os import sys @@ -139,11 +140,45 @@ def normalise(blob, layers, p, hdr_len): return bytes(res) +ROW_RE = re.compile(r"^\s*\{(\d+), (\d+), (\d+), (\d+), (\d+), (\d+), (\d+)\},", re.M) + + +def table_rows(source_text): + """The `kFconst19Rules` rows in a pack_import.cpp, as tuples in the tool's own field order.""" + out = {} + for m in ROW_RE.finditer(source_text): + hidden, heads, layers, prefix, n, rot, tile = (int(g) for g in m.groups()) + out[(hidden, heads, layers)] = (prefix, n, rot, tile) + return out + + +def check_row(source_text, measured): + """engine#2010: a row must BE the measurement. `prefix` is the first byte where the twins diverge, so a row + derived from `64 + 2*hidden + 4,352` (or from a neighbour's shape) is wrong in a way no geometry check sees -- + and a 1.9 pack with no 1.8 twin cannot be given a row at all, because there is nothing to diff. Returns + (ok, message).""" + hidden, heads, layers, prefix, n, rot, tile = measured + rows = table_rows(source_text) + key = (hidden, heads, layers) + if key not in rows: + return False, f"no kFconst19Rules row for {hidden}/{heads}/{layers}: the table cannot describe this pack" + got = rows[key] + want = (prefix, n, rot, tile) # the struct's field order + if got != want: + return False, (f"row {hidden}/{heads}/{layers} is {got} but this pair measures {want} " + f"(prefix {got[0]} vs measured {prefix})") + return True, f"row {hidden}/{heads}/{layers} is exactly what this pair measures" + + def main(): ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) ap.add_argument("--pack-1-8", required=True, help="the 1.8 pack of the same model") ap.add_argument("--pack-1-9", required=True, help="the 1.9 pack to measure") ap.add_argument("--json", action="store_true", help="print the result as JSON instead of the table line") + ap.add_argument("--check-row", metavar="PACK_IMPORT_CPP", + help="after measuring, find this geometry's row in PACK_IMPORT_CPP and FAIL if any field differs " + "from what this pair measures. The guard for a row derived from arithmetic (or from a " + "neighbour's shape) instead of from the twin-diff this tool computes: engine#2010") ap.add_argument("--probe-wrong", action="store_true", help="having found the rule, perturb it one field at a time and print each residual, so the " "tolerance has a measurement behind it instead of an argument") @@ -196,6 +231,11 @@ def main(): else: print(f"// verified: {note} ({len(a)} B)") print(f"{{{hidden}, {heads}, {layers}, {prefix}, {n}, {rot}, {tile}}},") + if args.check_row: + ok, why = check_row(open(args.check_row).read(), (hidden, heads, layers, prefix, n, rot, tile)) + print(f"# {'ok' if ok else 'MISMATCH'}: {why}", file=sys.stderr if not ok else sys.stdout) + if not ok: + return 1 if args.probe_wrong: # What a WRONG rule costs, in the same units as the tolerance: perturb one field of the accepted # rule at a time, normalise again, and report the residual. A tolerance is only defensible when the