From f3d77037493d274da70f3571a1f3e27d5a86787c Mon Sep 17 00:00:00 2001 From: t Date: Tue, 10 Nov 2026 14:05:54 -0400 Subject: [PATCH 1/3] npu-pack: a short epsilon vote no longer wins over the anchor residues (Qwen3-1.7B imports again, #1985) engine#1985: amd/Qwen3-1.7B_rai_1.9.0_npu (2.660 GB = the census size, so the pack is not the variable) imports with gaps 0 on 91815b27b and refuses with 280 on main -- "attention payload holds 64 tiles; q/k/v/o at 64 rows each need 96", qk_norm false, tiles_measured 0. The cause is that a *successful* vote can still be SHORT. On this pack the eps-anchor vote records one in-block offset for a block that has three (Qwen3 opens a per-head k_norm and q_norm section before o_proj), and #1838's early return -- right for a vote that found the whole list -- then kept that one, so the 2-header arithmetic at :3135 refused every attention projection. The two witnesses answer different questions, so this makes them work together rather than choosing one: * the vote settles the STRIDE (it needs a dominance test); * the residue list settles the COUNT (a real section boundary repeats in every block); * when the residues CONTAIN the vote's offset and say there are more, the residue list wins -- and the change is reported on stderr so a surprising answer is visible; * when they disagree outright, the vote's answer stands exactly as before. Verified on the pack, not by reading: `npu-pack info` -> gaps 0, qk_norm true, tiles_measured 64 (the same figures 91815b27b gives); `import --weight-layout b8` -> attention 2688/2688 tiles + MLP 3584/3584 groups byte-exact, 112 + 84 projections, scale/zero 100.00%; and it **serves on the device** against the regenerated kernels-qwen17-b8 family: 3/3 identical requests, 6 s per 16 tokens. npu_pack_import_test: all checks pass. Not fixed here: the same class may affect the other 4-header Qwen3 packs (Qwen3-4B, -2507, Qwen3-8B). They are OOB rows from before the change and each is a 5.4 GB re-run; the arithmetic above says they take the same path, and the same fix applies, but that is a claim for the re-runs rather than for this commit. --- npu/pack_import.cpp | 26 ++++++++++++++++++++++++-- 1 file changed, 24 insertions(+), 2 deletions(-) diff --git a/npu/pack_import.cpp b/npu/pack_import.cpp index ad46af92a..def37f589 100644 --- a/npu/pack_import.cpp +++ b/npu/pack_import.cpp @@ -1650,11 +1650,33 @@ FconstGeom locate_fconst_with_header(const std::string& tok, const std::string& // engine#1831: only when the vote did NOT record the block's headers. When it did, its list is the measurement // (`plan()` reads 2-header vs 4-header from `hdr.size()`, and `post_attn_hdr()` is `hdr.back()`), and rewriting it // to one entry turned every Qwen3 4-header 1.9 block into a 2-header one whose "post-attention" header was k_norm's. - if (!g.hdr.empty()) return g; + // engine#1985: a vote that succeeded can still be SHORT. On amd/Qwen3-1.7B_rai_1.9.0_npu the epsilon-anchor vote records + // one in-block offset for a block that has three (Qwen3 opens a per-head k_norm and q_norm section before o_proj), and + // engine#1838's early return -- correct for a vote that found the whole list -- then keeps that one and every attention + // projection refuses with "attention payload holds 64 tiles; q/k/v/o at 64 rows each need 96". The two witnesses answer + // different questions: the vote settles the STRIDE (it needs a dominance test), the residue settles the COUNT (a real + // section boundary repeats in every block). So when the residues contain the vote's offset and say there are more, the + // residue list wins; when they disagree outright, the vote's answer stands untouched. const std::vector pos = eps_header_positions(tok, size_t(1) << 30, dims.eps_bits); + const std::vector residue = fconst_block_headers(pos, stride8, 3); + if (!g.hdr.empty()) { + const bool residue_contains_vote = + std::find(residue.begin(), residue.end(), g.h1) != residue.end(); + if (residue_contains_vote && residue.size() > g.hdr.size()) { + FconstGeom better = g; + if (fconst_set_block_headers(better, residue)) { + std::fprintf(stderr, + "note: the epsilon vote recorded %d in-block header(s) at %lld B stride, the anchor residues " + "repeat at %d; using the residue list (%d headers per block)\n", + int(g.hdr.size()), (long long)g.stride, int(residue.size()), better.period); + g = better; + } + } + return g; + } // The scan window is the first GiB (about 8 blocks at h3584), so a real section boundary repeats ~8 times against // 1-2 for a payload coincidence; 3 separates them (requiring `layers/2` was too strict and dropped the real one). - const std::vector inner = fconst_block_headers(pos, stride8, 3); + const std::vector& inner = residue; if (!fconst_set_block_headers(g, inner)) { g.why = "the 1.9 header gives stride " + std::to_string(stride8) + ", but the epsilon anchors repeat at " + std::to_string(inner.size()) + " in-block offset(s), and a layer block has 1 (2-header) or 3 (4-header); " From f4a925e2ea5e52d198197a93e71c72bf860da9d7 Mon Sep 17 00:00:00 2001 From: bong-water-water-bong Date: Tue, 10 Nov 2026 15:45:26 -0400 Subject: [PATCH 2/3] npu-pack: the two-witness arbitration is a pure function, and a seven-arm table pins it Review of #1986, which asked for exactly this: twenty-four lines of arbitration between the epsilon vote and the anchor residues, with nothing in the tree pinning either direction. Both directions matter -- taking the residue list too eagerly is a wrong layout on a pack the vote got right, and never taking it is engine#1985 (Qwen3-1.7B refused with 64 tiles where the block has 96). `prefer_residue_headers(vote_h1, vote_count, residue)` is now a pure exported function; `locate_fconst_with_header()` calls it. It keeps the rule the PR's comment states: the residue list wins only when it CONTAINS the vote's offset and reports MORE offsets -- a vote that found part of the list is short, not wrong, while a residue list that contradicts it is a different answer and leaves the vote alone. Arms (7, no pack, no file, no configure): * a short vote yields to the residue list (engine#1985's shape: 1 offset against 3); * the vote's offset may sit anywhere in the residue list; * equal-length lists keep the vote (nothing to arbitrate, and no churn in the note); * a LONGER vote keeps the vote (the residues lost a boundary); * a residue list that does not contain the vote's offset keeps the vote (disagreement, not shortness); * an empty vote and an empty residue list never prefer the residue. Mutant check -- drop the containment test so the longer list always wins: FAIL prefer residue: a residue list that does not contain the vote's offset keeps the vote 1 check(s) failed restored -> all npu-pack checks passed. Two linkage slips, both mine and both the same as fconst19_header's earlier: the definition landed inside the anonymous namespace, so the header's external declaration made the call ambiguous ("2 candidates", identical signatures). Moved out with a scoped namespace close, which is the same fix that was needed last time -- the constructor of that mistake is defining a test seam next to the code it belongs with. Refs #1986, #1985 --- npu/pack_import.cpp | 13 ++++++++++--- npu/pack_import.h | 7 +++++++ tests/npu_pack_import_test.cpp | 23 +++++++++++++++++++++++ 3 files changed, 40 insertions(+), 3 deletions(-) diff --git a/npu/pack_import.cpp b/npu/pack_import.cpp index def37f589..83e303612 100644 --- a/npu/pack_import.cpp +++ b/npu/pack_import.cpp @@ -493,6 +493,15 @@ const Fconst19Rule* select_rule_in(const Fconst19Rule* rules, int count, int hid // The header a 1.9 block carries: 00 00 7f ff once per head. Its length being the head count is what makes // it a test rather than a magic number. 0x00007fff is the big-endian reading of those four bytes; the bytes // are what is compared, because the little-endian reading of the same bytes is 0xff7f0000. +} // namespace (closed so prefer_residue_headers has external linkage: the arm drives it directly, engine#1986) +bool prefer_residue_headers(int64_t vote_h1, size_t vote_count, const std::vector& residue) { + if (vote_count == 0 || residue.empty()) return false; + if (residue.size() <= vote_count) return false; // the vote already has the longer list + return std::find(residue.begin(), residue.end(), vote_h1) != residue.end(); +} + +namespace { + std::vector fconst19_header(int heads) { std::vector h(size_t(heads) * 4, 0); for (int i = 0; i < heads; ++i) { @@ -1660,9 +1669,7 @@ FconstGeom locate_fconst_with_header(const std::string& tok, const std::string& const std::vector pos = eps_header_positions(tok, size_t(1) << 30, dims.eps_bits); const std::vector residue = fconst_block_headers(pos, stride8, 3); if (!g.hdr.empty()) { - const bool residue_contains_vote = - std::find(residue.begin(), residue.end(), g.h1) != residue.end(); - if (residue_contains_vote && residue.size() > g.hdr.size()) { + if (prefer_residue_headers(g.h1, g.hdr.size(), residue)) { FconstGeom better = g; if (fconst_set_block_headers(better, residue)) { std::fprintf(stderr, diff --git a/npu/pack_import.h b/npu/pack_import.h index dcd0aa86a..12e9826ad 100644 --- a/npu/pack_import.h +++ b/npu/pack_import.h @@ -524,6 +524,13 @@ std::vector fconst_block_headers(const std::vector& pos, int64 // engine#1831: set `hdr`/`h1`/`period` from that list the way locate_fconst() does -- one inner header is a 2-header block, // three is a 4-header block. Returns false, and changes nothing, for any other count. bool fconst_set_block_headers(FconstGeom& g, const std::vector& inner); +// engine#1985: which in-block header list to keep, given the epsilon vote's and the anchor residues'. The two witnesses answer +// DIFFERENT questions -- the vote settles the STRIDE (it needs a dominance test), the residue settles the COUNT (a real section +// boundary repeats in every block) -- so a vote that found only part of the list is not wrong, just short, and the longer +// consistent list wins. `residue` wins only when it CONTAINS the vote's first offset and reports more; an outright disagreement +// is not a licence to take the longer list, so the vote stands. Pure and exported so this decision table is pinned with no pack, +// no file and no configure (engine#1986 review). +bool prefer_residue_headers(int64_t vote_h1, size_t vote_count, const std::vector& residue); // One measured Ryzen AI 1.9 Token .fconst block shape (engine#1503). Every field is measured against the 1.8 pack of // the same model by tools/fconst19_measure.py; none of it is derivable from the pack, which is why a pack whose shape diff --git a/tests/npu_pack_import_test.cpp b/tests/npu_pack_import_test.cpp index 1a91f2cf9..d5a0fcdaa 100644 --- a/tests/npu_pack_import_test.cpp +++ b/tests/npu_pack_import_test.cpp @@ -4090,6 +4090,28 @@ void test_fconst_variable_stride(const std::string& root) { // engine#1740 (W1a): the tail-head placement arithmetic, pinned without a pack, a device or a fixture. The rows, the // tile size and the envelope are the hidden-896 pack's, so the numbers here are the real ones: 1,856 B envelope, // 2,374 tiles of 122,880 B, tail 1,856 + 2,374 x 122,880 = 291,653,952 B. +// engine#1986 review: the arbitration between the two witnesses, pinned as a decision table. Twenty-four lines of "which list +// wins" had no arm, and both directions matter: taking the residue list too eagerly is a wrong layout on a pack the vote got +// right, and never taking it is engine#1985 (Qwen3-1.7B refused with 64 tiles where the block has 96). +void test_prefer_residue_headers() { + using V = std::vector; + // the vote found ONE in-block offset at 640 B; the residues repeat at three -- engine#1985's shape + check(pi::prefer_residue_headers(640, 1, V{640, 1280, 1920}), "prefer residue: a short vote yields to the residue list"); + // ...and the same three offsets with the vote's one among them in a different position + check(pi::prefer_residue_headers(1920, 1, V{640, 1280, 1920}), "prefer residue: position in the list does not matter"); + // equal lists: nothing to arbitrate, and no churn in the note + check(!pi::prefer_residue_headers(640, 3, V{640, 1280, 1920}), "prefer residue: an equal-length list keeps the vote"); + // the vote found MORE than the residues: the vote wins, because the residues lost a boundary + check(!pi::prefer_residue_headers(640, 4, V{640, 1280, 1920}), "prefer residue: a longer vote keeps the vote"); + // OUTRIGHT DISAGREEMENT: the residue never mentions the vote's offset, so this is not "short", it is "different" + check(!pi::prefer_residue_headers(704, 1, V{640, 1280, 1920}), + "prefer residue: a residue list that does not contain the vote's offset keeps the vote"); + // degenerate inputs: an empty vote is handled by the caller's own branch, an empty residue never wins + check(!pi::prefer_residue_headers(640, 0, V{640, 1280, 1920}), "prefer residue: an empty vote never prefers the residue"); + check(!pi::prefer_residue_headers(640, 1, V{}), "prefer residue: an empty residue list never wins"); + std::printf(" prefer residue: 7 arms (short vote yields, equal/longer/disagreeing keep the vote, degenerate inputs)\n"); +} + void test_tail_head_placement() { // The real hidden-896 geometry: group_bytes(896, 64) = 8 x 68 x 64 = 34,816 B per head tile, V/64 = 2,374 tiles, // 1,856 B envelope, so the tail is 1,856 + 2,374 x 34,816 = 82,655,040 B -- which is the 0.5B pack's measured tail @@ -4271,6 +4293,7 @@ int main(int argc, char** argv) { test_encode_tile_code_range(); test_qk_norm_refusal(); test_qk_gain_offsets(); + test_prefer_residue_headers(); test_tail_head_placement(); const std::string golden_dir = argc > 1 ? argv[1] : ""; const std::string pack = argc > 2 ? argv[2] : ""; From 9e593b4440f82ac220cd3df034a199cfe784a89c Mon Sep 17 00:00:00 2001 From: t Date: Tue, 10 Nov 2026 19:51:38 -0400 Subject: [PATCH 3/3] npu-pack test: remove the fixture files my own `git add -A` swept into this branch The CI "Copyright notice on every file" step went red on this branch and it was my fault, not the checker's: I ran npu_pack_import_test in the repo root while the branch predated #2024 (which makes the q4nx-container arm remove its fixture directory), and then committed with `git add -A`, so five fixture blobs travelled into the branch: q4nx-container/{container-genre/model,huge,meta,short,two}.q4nx They are test output, they belong to no commit, and the copyright checker is right to refuse a binary blob with no header. `git ls-tree origin/main | grep q4nx-container` is empty -- this exists only here. Removing them returns this branch's diff to the three files it should touch: npu/pack_import.{cpp,h} and tests/npu_pack_import_test.cpp. Verified locally: `python3 tools/copyright.py --check` is clean. --- q4nx-container/container-genre/model.q4nx | Bin 1340 -> 0 bytes q4nx-container/huge.q4nx | Bin 8 -> 0 bytes q4nx-container/meta.q4nx | Bin 5224 -> 0 bytes q4nx-container/short.q4nx | Bin 5158 -> 0 bytes q4nx-container/two.q4nx | Bin 820685 -> 0 bytes 5 files changed, 0 insertions(+), 0 deletions(-) delete mode 100644 q4nx-container/container-genre/model.q4nx delete mode 100644 q4nx-container/huge.q4nx delete mode 100644 q4nx-container/meta.q4nx delete mode 100644 q4nx-container/short.q4nx delete mode 100644 q4nx-container/two.q4nx diff --git a/q4nx-container/container-genre/model.q4nx b/q4nx-container/container-genre/model.q4nx deleted file mode 100644 index 010a7b9d9898fcb8c363ddb9c0d4a389eea8fe7a..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 1340 zcmdnO00GrXx%nxnIeMwNNvSFECHdK@dBu9=shQ~+B}!J+N+~6k1*u9_N=|NuW=c9r z#TkhpZnT++jzz4FQc7Y;VtjsDT5)PgF;LV%$I!sYB(_!uYGzJiWol8eo`GI&PJv!Z wetBMeK~a7d7SlZ~kj%EwF*L(88)UMMp`oc!Y;CR5C>RZa(GVC7fdL8u07F>ROaK4? diff --git a/q4nx-container/huge.q4nx b/q4nx-container/huge.q4nx deleted file mode 100644 index b786843b6f0fc9cf328ec334a1487fa9a650e351..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 8 NcmZQz00Kq^1^@sH00IC2 diff --git a/q4nx-container/meta.q4nx b/q4nx-container/meta.q4nx deleted file mode 100644 index 457e118667f518277dc85085593e227bd2ac8d2b..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 5224 zcmeIuK?=e!6h%?j^2Qm|l!|l*u0cu+G*t)EV!i_{*}Fz}A@|Mxg+Do`&2OthxFw5Y zv>1Z=Rp5c2Qz8Ojp eLjVB;5I_I{1Q0*~0R#|0009ILKmdWi6?g-ZrN*%U diff --git a/q4nx-container/short.q4nx b/q4nx-container/short.q4nx deleted file mode 100644 index 193af377155c0f95bb656e259f712733771f4941..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 5158 zcmeIup$ULc07X$}dD)~z1P5>i(+35kphf=jB5on)wp-8CeASeUHS2XBTgJ>o+MIiN eXX(3$009C72oNAZfB*pk1PBlyK!5;&-wHg<$FUUv diff --git a/q4nx-container/two.q4nx b/q4nx-container/two.q4nx deleted file mode 100644 index 6b2e734b669d476cb74ddae156e0e2fc758f7c4a..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 820685 zcmeIuOKQSU6oBEb<&G~Qib!zSL7{6X5(0_Uk{3xtL|k6?RimRm23^2+X6Fq41K;P< zV=kfT%BoK4;#>`5k|u|?KTi35ZM&gKPSxRkKZVUDl+(Gd!X`YwEY=~0abL{q+j^C5 zV