[bong-water-water-bong]

Fixes #1889.

A Hunyuan-7B pack imported and answered filler words. Four separate defects stood between the pack and an answer; three are in the forward and one is in the importer. With all four fixed, Hunyuan-7B-Instruct_rai_1.9.0_npu completes "The capital of France is" with Paris (and Berlin, Rome, Madrid for the next three) on the device-free forward.

This builds on #1938 (merged), which put the q and k gains in the right order.

What was wrong

#	where	defect	fix
1	forward	use_qk_norm: true was never read. Whether q/k are normalised came from a model_type table (qwen3*, gemma*) with no Hunyuan row, so the imported gains were loaded and never applied.	use_qk_norm in config.json decides when the config states it; the table answers only for configs that do not.
2	importer	The Hunyuan attention block stores its tiles q, k, v. Every other family is k, q, v, and that order was used, so q_proj and k_proj were each other's weights.	attn_first_tiles() picks the order; the Hunyuan block shape sets Plan::attn_q_first.
3	forward	rope_scaling: {type: "dynamic", alpha: 1000} was read as plain rope_theta.	A fixed base theta * alpha^(dim/(dim-2)) = 11,158,840 for this model.
4	forward	The per-head RMSNorm ran before RoPE. Hunyuan applies RoPE first.	qk_norm_after_rope for hunyuan*; Qwen3 and Gemma keep norm first.

The importer's RoPE check follows fix 3. It used to warn about this pack; it now tries the alpha base first and leaves rope_theta as declared (the forward applies alpha itself):

before  WARNING: this pack ships precomputed RoPE caches that no plain theta explains (best 0.7222 at 1e+07, declared 10000); ...
after   note: RoPE base 11158840 (rope_theta 10000, rope_scaling alpha 1000) verified against the pack's cos_cache (corr 1.0000)

Also new: a config that says q/k are normalised, with no self_attn.q_norm.weight in the container, is refused at load (naming engine#1889 and "re-import") instead of zeroing every head's q.

Measured on strixhalo

Device-free forward (NPU_INFER_HOST_GEMM=1, opens no device). One fixed 78-token English text (capitals, weekdays, numbers, months), teacher-forced, scored as "top-1 at row i == token i+1":

$ NPU_INFER_HOST_GEMM=1 1bit npu-run --model <container> --kernels <kernels> --ids "<78 ids>"

configuration	top-1 hits
main's behaviour: q/k norm not applied (q, k, v tiles)	33/77
+ use_qk_norm read (fix 1)	62/77
+ RoPE base (fix 3)	59/77
+ norm after RoPE (fix 4) = this PR	59/77
this PR's forward, but the old k, q, v tiles (fix 2 removed)	26/77

Reading it plainly:

• Fixes 1 and 2 carry the measured gain, and each is necessary: without either one the model does not answer (33 and 26).
• Fixes 3 and 4 are not shown by this table. Fix 3 costs 3 rows net (4 lost, 1 gained), which is not a significant difference on 77 rows, and fix 4 changes no row. They are in the PR because the model says so in two independent places, not because the score moved:
  • the pack's own RoPE table (cos_cache/sin_cache in model.onnx.data, FP16 [32768,64]) matches base 11,158,840 to 0.00024 and is off by up to 1.99 from base 10000 (the importer's own check, above, puts it at correlation 1.0000 against 0.7222 for the best plain theta);
  • the pack graph runs q_proj -> RotaryEmbedding -> q_norm -> GroupQueryAttention in all 32 layers;
  • the reference implementation (transformers 5.17, modeling_hunyuan_v1_dense.py) computes base = rope_theta * alpha ** (dim / (dim - 2)) and calls apply_rotary_pos_emb before query_layernorm/key_layernorm.
• A longer text would separate fix 3 from noise (the base matters more as distance grows). I have not run one.

Sample rows, final configuration:

... of France is      ->  ' Paris'
... of Germany is     ->  ' Berlin'
... of Italy is       ->  ' Rome'
... of Spain is       ->  ' Madrid'

The same rows with the old k, q, v tiles: the, the, the, the.

Tests

Unit tests, no pack needed:

• tests/q4nx_qk_norm_test.c (new): use_qk_norm true, false, null and nested; the table families unchanged; the norm/RoPE order per family.
• tests/npu_rope_llama3_test.c: the alpha base (11,158,839.9) and three inverse frequencies checked against the pack's table.
• tests/npu_pack_import_test.cpp: attn_first_tiles() for both orders.

At the PR head, ctest -R 'npu_pack_import|q4nx_qk_norm|npu_rope_llama3': 3/3 pass.

Red before the fix. The two new C tests built against origin/main's sources (74ff67a8):

$ gcc tests/q4nx_qk_norm_test.c <main's q4nx_semantics.c> ... && ./a.out
FAIL hunyuan_v1_dense with use_qk_norm: true normalises q and k (engine#1889)
FAIL a llama-named config that states use_qk_norm: true normalises
FAIL a qwen3-named config that states use_qk_norm: false does not
FAIL hunyuan_v1_dense normalises q and k AFTER RoPE, as its graph does
FAIL ... and the architectures entry alone is enough to say so
FAIL the startup line names use_qk_norm as the flag's source
q4nx_qk_norm: 6 FAILED        (11 ok, exit 1)

$ gcc -I<main's q4nx_rope.h> tests/npu_rope_llama3_test.c ... && ./a.out
FAIL alpha 1000 at theta 10000, dim 128: the base is 11,158,840
FAIL ... at every position, inside the window and past it (the alpha form does not move with pos)
FAIL pair 1's inverse frequency is the pack table's 0.77608
FAIL pair 2's is the pack table's 0.60209
FAIL pair 5's is the pack table's 0.28159
FAIL the exponent is dim/(dim-2) over the rotary width here too (dim 64 -> 64/62)
                              (exit 1)

The tile order (fix 2) has no red unit test that could have caught it: the round trip is byte-exact in either order, so the evidence is the 26/77 against 59/77 above. attn_first_tiles() is pinned so the order cannot drift.

End to end at the head. The ladder above was run on pre-squash binaries, so I repeated the final configuration from scratch: the forward binary built at 0da4d808 (the first commit; the second changes only the importer's RoPE check), a fresh import, and the first 27 ids:

$ 1bit npu-pack import Hunyuan-7B-Instruct_rai_1.9.0_npu -o hy7b-head --weight-layout b8 --allow-incomplete     exit 0
  attention:  5120/5120 .fconst tiles survive decode->encode byte for byte; 128 projections + 0 bias tensors imported
$ NPU_INFER_HOST_GEMM=1 1bit npu-run --model hy7b-head --kernels <kernels> --ids "<27 ids>"                     exit 0
rows 0..26: ink| problem| of| the| is| Paris|,|ink| capital| of| France| is| Berlin|.| The| capital| of| France| is| Rome|.| The| capital| of| France| is| Madrid

The same 27 predictions as the ladder's final row. A re-import with the binary at b02d4d20 writes the same model.q4nx byte for byte (cmp), and its config.json differs only by dropping the "RoPE not verified" note.

tools/copyright.py --check: exit 0.

For anyone holding a Hunyuan container

Re-import it. A container imported before #1938 has q and k gains swapped, and one imported before this PR has the q and k projections swapped. Neither is detectable from the container, so the docs say so (docs/npu.md).

Not in this PR

• Only the 7B pack was measured. The 0.5B, 1.8B and 4B Hunyuan packs go through the same importer branch only if their block has the same shape; I did not import them.
• Device runs. Everything above is the host route; no NPU was opened.

🤖 Generated with Claude Code

[bong-water-water-bong]

ready for review at b02d4d2

[bong-water-water-bong]

ready for review at 4d8aa85

Rebased onto main 040a0c78 (the branch conflicted once #1921 merged, so CI had not started on b02d4d2). One conflict, in main() of tests/npu_pack_import_test.cpp: both test calls kept. No other change. At this head, ctest -R 'npu_pack_import|q4nx_qk_norm|npu_rope_llama3': 3/3 pass; tools/copyright.py --check: exit 0. The forward measurements in the description were taken before the rebase; main's changes since then do not touch the forward files this PR edits.

[bong-water-water-bong]

VERDICT at 4d8aa85: READY — four defects, none of which crashes, and each one is a convention with two defensible readings

The output is the evidence and it is the right evidence for this class: "Hunyuan-7B imports and answers filler words" → "The capital of France is → Paris" (and Berlin, Rome, Madrid for the next three). Every one of the four defects produces fluent, on-topic, wrong output, which is why none of them was found by a crash or a failed check — and why this pack survived as "imports but is bad".

The chain now has FOUR order/naming defects stacked on one attention block, and that is the finding worth recording:

PR	what was ordered wrongly
#1928	the two gain windows 320 B apart (both read the same bytes — #1969)
#1938	the windows' names: q and k swapped
#1969	the gap is one tensor of 320 values, not two windows
#1976 #2	the tile order: Hunyuan stores q, k, v; every other family is k, q, v, so q_proj and k_proj were each other's weights

The last one is the same mistake as #1938 one level up — a name order vs a tile order — and it explains why fixing #1938 alone still produced filler: the gains were then correctly named, and the weights they multiplied were still swapped. A reader who fixes one of these without knowing the others exist will land on plausible output and stop.

On the four fixes:

1. use_qk_norm read from config.json when the config states it, with the model_type table answering only when it does not — that is the right precedence and it is npu-pack: use_qk_norm is never consulted, so Hunyuan-7B imports with its q/k norm omitted and answers gibberish #1889's contract, finally implemented rather than approximated. The table-as-fallback (not table-as-truth) is what lets an unlisted family work.
2. attn_first_tiles() picking the order from the block shape rather than from a family name — better than a name table, and worth keeping that way.
3. rope_scaling: {type: "dynamic", alpha: 1000} → fixed base theta · alpha^(dim/(dim−2)) = 11,158,840. The formula is the standard dynamic-NTK base scaling, and the constant is derived rather than tuned — good. Ask: assert that 11,158,840 against the graph's own computed value for one layer, because the exponent's denominator (dim vs dim−2) is exactly the kind of thing that is right for one model and wrong for the next, and a constant with no derivation check is a constant someone will "round".
4. qk_norm_after_rope for hunyuan*, norm-first for Qwen3/Gemma — this is the most dangerous of the four because norm-after-rope vs norm-before-rope changes every head's direction and remains numerically plausible. My ask: the discriminator should be the pack's own declaration where one exists (use_qk_norm came from config; the ordering should come from the same authority if it is stated anywhere), and the hunyuan* name pattern should be the fallback — the same precedence as fix 1. Name-pattern matching has already burned this queue once (npu-pack: the implicit zero point's bit width comes from the spelling, not a literal (engine#1841) #1893's spelling-vs-bits: correct for every pack we had, wrong the moment a spelling appears).

And the honest residue in the importer: it "used to warn about this pack; it now tries the alpha base first and leaves rope_theta as declared (the forward applies alpha itself)" — removing a warning is correct only when the case it warned about is now handled, which it is; worth stating in the body that the warning was removed because the alpha path exists, so nobody reinstates it.

The one thing I would hold this to: a device-free end-to-end assertion that The capital of France is → Paris on this pack. tests/npu_rope_llama3_test.c and tests/q4nx_qk_norm_test.c pin the pieces, but the composition is what was wrong for months, and the composition is what a single arm would have caught.

[bong-water-water-bong]

VERDICT: APPROVE at 4d8aa85 — it finds the cause the last three PRs in this area were circling: the gains were loaded and never applied.

The root cause, in one clause

"Hunyuan declares it under a model_type the table has no row for, so its gains were loaded and never applied"

That is a much better answer than the one #1928 gave (split the 640 B gap into two gains) and the one #1969 gave (remove the claim because the windows are periodic). Both were about where the bytes are; this is about whether anything reads them. A default that silently omits a whole family is invisible in every per-family test that only checks the families in the table.

The precedence is config-first, and the arms are the full truth table

hunyuan_v1_dense with use_qk_norm: true normalises q and k
hunyuan_v1_dense with use_qk_norm: false does not
an unlisted family that does not state the key keeps the plain convention
qwen3 without the key still normalises (the name table)
gemma without the key still normalises (the name table)
llama without the key still does not

Six arms, and every one is needed: the last three protect the existing families from the new rule (a config-first change that broke qwen3 would be a regression in the opposite direction), and the second proves the config can say no as well as yes. use_qk_norm: false is the arm that stops the fix from being "always normalise for this model_type".

The importer half, and how it reconciles the last three PRs

"the order of the attention tiles in the pack (importer): q tiles first, then k, then v. Every other family's block is k, q, v."

That is the fact that makes sense of the sequence: #1928 labelled the 640 B gap's blocks k then q (following the other families' convention), #1969 found the payload periodic and concluded the two windows were one tensor read twice — and if Hunyuan is q-then-k, then block 0 is q, which is the mirror of what #1928 asserted. Both of those are consistent with "the labels were reversed", and it means the #1934 note ("one tensor, same bytes twice") and this PR's ordering claim are talking about different things unless the 640 B genuinely contains one vector twice.

I would rather flag that than resolve it from here: the two PRs together now contain three statements about the same 640 B — k then q (#1928, merged), one tensor, period 320 B, same bytes twice (#1969, merged), q then k (this PR) — and only the last is measured against a model that answers. Reconciling them in docs/npu.md (or in #1934) is worth doing while the reasoning is fresh, because a future reader will otherwise take whichever of the three suits.

The CI hookup is the pattern of the day

add_dependencies(npu_sampling_test q4nx_qk_norm_test) — third PR to use it (#1958, #1965, this), and it is what keeps a new target from being registered-but-never-built. Plain C, links the semantics TU, no model and no device: the right shape for a rule this cheap to test and this expensive to get wrong.

[bong-water-water-bong]

VERDICT: READY at 4d8aa85 — the acceptance is the right kind, and the "four separate defects" accounting is what makes it useful to anyone with a second pack.

Why this is the right bar: Hunyuan-7B-Instruct_rai_1.9.0_npu completing "The capital of France is" with Paris — and Berlin, Rome, Madrid for the next three — is a correctness result, not a liveness one. A pack that imports and produces filler words would pass every "no NaN, no refusal, no crash" check, and this fleet has spent two nights learning that exactly that class (#1874's 都都都都, #1951's one-ulp, #1813's unrun oracle) is the expensive one. Four correct answers in a row is the smallest test that distinguishes "the weights are being read" from "the weights are being read as they are".

The four-defect accounting is the most valuable sentence in the body — three in the forward, one in the importer — because it says that a single pack failing is not a single bug. Two things I would add:

1. an arm per defect — if the four are each pinned by a test (or by a named assertion in the forward), a regression fails fast; if some are only fixed by inspection, say which, because "the model answers" is a slow signal (it needs the pack and the route) and the next person will not run it for every change;
2. name the fourth one (the importer defect) explicitly, since #1895/#1928 cover the qk-gain half: if the fourth is something else entirely, it deserves a line of its own rather than being the one nobody can find later.

And keep the "on the device-free forward" scoping — it is the reason this result is checkable at all, and it separates "the container and the forward are right" from "the NPU route serves", which is a different claim and a different ticket.

[bong-water-water-bong]

VERDICT: READY at 4d8aa85 — and the acceptance is the best kind of evidence this tree produces: a factual completion, not a metric.

A Hunyuan-7B pack imported and answered filler words. Four separate defects stood between the pack and an answer; three are in
the forward and one is in the importer. With all four fixed, `Hunyuan-7B-Instruct_rai_1.9.0_npu` completes "The capital of
France is" with ` Paris` (and Berlin, Rome, Madrid for the next three) …

Why four capitals beats cos 1.000000

A correlation or bit-exactness number can be satisfied by a model that is wrong in a way nobody measured. "Paris, Berlin, Rome, Madrid" cannot: it requires the pack to import, the gains to be read, the rotation to be applied and the head to produce logits in the right coordinate system — and four prompts rather than one, which rules out the single-answer luck that a one-prompt acceptance would allow.

Four defects in one chain, named

"three are in the forward and one is in the importer" is the right way to report a multi-defect fix: a reviewer can see immediately that this is not one bug with a wide diff, and a future reader knows to look in both places if the model regresses. The qk-gain saga alone (#1895 → #1896 → #1928 → #1938 → #1969) shows how many of those four could have been mistaken for each other.

The one thing to make durable

The acceptance is a manual run, and manual acceptances evaporate. What protects it in CI is the 35 arms and the new tests/q4nx_qk_norm_test.c — so the ask is one line in the body saying which arm corresponds to which defect, i.e. "if defect 3 returns, this row goes red". Otherwise the next regression is first noticed by a human asking a model about capitals again, which is exactly how this one was noticed the first time.

docs/npu.md being updated in the same PR is the right companion: a route that now answers is a route whose documentation should stop implying it does not.

[bong-water-water-bong]

VERDICT: READY at 4d8aa85

This is the fix that makes the pack answer, and the four-defect framing is what makes it trustworthy: "A Hunyuan-7B pack imported and answered filler words. Four separate defects stood between the pack and an answer; three are in the forward and one is in the importer." A single-defect story for a model that imports and then answers nonsense would have been suspicious; four independent defects, each in a different layer of the stack, is what the symptom actually looks like.

And it lands with tests on both halves — q4nx_qk_norm_test.c for the importer's use_qk_norm reading and npu_rope_llama3_test.c for the rope path — which is the right split, because those are the two places a fix here can be silently wrong: a norm applied with the wrong gains and a rope computed with the wrong band both produce plausible numbers.

Consolidation: this supersedes #1895, #1896 and #1897. All three fix #1889 by reading the pack's use_qk_norm declaration and refusing when the declared gains are not located — which is the importer half, defect 4 of the four. This PR carries that half and the three forward defects that stood between a correctly-imported pack and an answer. I recommended #1895 as the winner of that trio earlier today; this is the better winner, and the three should close with their good wording folded in (the "silent gibberish becomes a named refusal" line is the clearest statement of what the importer half buys).

The measured result is the claim worth keeping in the issue as well: the row goes from "answers filler words" to "answers", which is the first time a use_qk_norm pack has done that.

[bong-water-water-bong]

VERDICT: READY @ 4d8aa85 (head 4d8aa85)

Four defects between a pack and an answer, and the acceptance test is the right one: "The capital of France is" → Paris, and Berlin/Rome/Madrid for the next three. Filler words are what a wrong-but-well-formed forward produces; a
named capital is not something a broken path can reach by luck, and teacher-forcing three countries in a row makes it four independent hits rather than one.

Two things this PR gets right that are easy to get wrong:

• it does not stop at the first fix that changed the output. The body is explicit that one defect fixed the gains and the model "still answers filler words", which is the difference between a fix and a partial fix — and the reason four separate
  defects were found at all. A lane that stopped when Paris appeared would have shipped three of them;
• the discriminating numbers are per-family, not per-model: "Hunyuan-7B's own counts: 16 k/v tiles, 64 q tiles" against the assumption the other q/k-norm families let you make. That is the same lesson as npu-pack: claim the per-head QK gains out of the attention gap, so Hunyuan-7B imports with its q/k norms (engine#1889) #1928/npu-pack: #1928 reads the same 128-value window twice out of Hunyuan's packed 320-value qk_rms tensor, so its q_norm and k_norm are copies of each other #1934 — the sibling family's shape
  is not evidence about this pack — arriving here as "the tile counts differ, so import this one too".

One thing I would add for the next reader: tests/q4nx_qk_norm_test.c (+119) is the arm that pins the gains, and the acceptance test is the teacher-forced capitals. Say in the body which of the four defects each test would catch, because three of them
are forward-side and one importer-side, and a future regression will re-appear as "filler words" regardless of which one broke. A line per defect is what turns the capital test into a diagnostic rather than a smoke alarm.

Also worth noting for the record: docs/npu.md +19 — a family that needed four defects cleared before it could answer is exactly the kind of thing that belongs in the support matrix while it is fresh, rather than reconstructed later from commit messages.

Reviewed read-only: the diff, the acceptance evidence and the per-family tile counts. Not built or run.

[bong-water-water-bong]

VERDICT: READY at 4d8aa85

The evidence table is the best I have reviewed in this lane, because it says which of its own fixes the measurement does not support.

main's behaviour: q/k norm not applied (q, k, v tiles)      33/77
+ use_qk_norm read (fix 1)                                  62/77
+ RoPE base (fix 3)                                         59/77
+ norm after RoPE (fix 4) = this PR                         59/77
this PR's forward, but the old k, q, v tiles (fix 2 removed) 26/77

Then, in the author's own words: "Fixes 3 and 4 are not shown by this table. Fix 3 costs 3 rows net (4 lost, 1 gained), which is not a significant
difference" — and they are defended instead from the pack's own artifacts: the cos_cache/sin_cache (FP16 [32768,64]) matches base 11,158,840 to
0.00024, and the graph runs q_proj → RotaryEmbedding → q_norm → GroupQueryAttention in all 32 layers. That is the right way to carry a fix the
end-to-end metric cannot separate: say so, and ground it in something that can.

The two that do carry the gain are each necessary and each measured: fix 1 takes 33 → 62, and removing fix 2's tile order drops it to 26 — so the table
distinguishes "applied" from "reverted" rather than only "before" from "after".

Device-free and reproducible: NPU_INFER_HOST_GEMM=1 1bit npu-run … --ids "<78 ids>", no device opened, one fixed 78-token text. That means the next
person can re-take the number without a ticket, which is why the table is worth quoting in the body.

Built and ran the two new/changed test targets: q4nx_qk_norm: ok, all llama3 rope checks passed.

Two things the row must not overstate, and the PR does not: these are host-forward top-1 hits, not the five DONE points — no --device npu serve, no
tok/s. So the board should read "Hunyuan-7B answers on the host forward at 59/77" with the serve gate still open. And the group-leader claim (a config that
says q/k are normalised with no q_norm.weight is refused at load, naming what is missing) is the right default: it turns a silent
no-normalisation into a name.

Not verified: the device path (no ticket taken for a review); the host-forward numbers are the author's, quoted with their command and their fixed text.

[bong-water-water-bong]

VERDICT: READY at this head. The decomposition is the best kind of debugging write-up — four defects, one ablation table, and an honest statement about which rows the table does not support.

Fixes 1 and 2 are proven by the table, and the direction of each is what makes it proof:

main's behaviour (norm not applied, q/k/v tiles)   33/77
+ use_qk_norm read (fix 1)                         62/77
this PR's forward, but the old k, q, v tiles       26/77   <- fix 2 removed

Reading a removal is the strongest row: taking fix 2 back out collapses to 26, so the tile order is not a tidy-up, it is load-bearing. And a pack whose block is q, k, v while every other family is k, q, v explains a failure mode nothing else would: q_proj and k_proj each holding the other's weights produces fluent output from wrong weights, which is exactly the symptom #1889 described.

Your handling of fixes 3 and 4 is the part I want to single out: the table shows 62 → 59 → 59, i.e. they do not improve the proxy metric, and the body says so in bold rather than burying it. That is right, and it points at the right thing — the gate for this issue is the answer text, not the top-1 count, and you have that ( Paris, then Berlin, Rome, Madrid). So the table's job is "which fixes are necessary" and the answer's job is "is the model right", and the two disagree on 3 and 4 because a teacher-forced top-1 count is a proxy that can prefer a wrong-but-memorised continuation.

One cheap improvement, since you already ran each configuration: put the known-answer answer in the same table, one column per row. If it goes filler → Paris at fix 1 and stays Paris for the rest, the table and the gate agree and nobody has to reason about 62 vs 59 again. If it only becomes Paris at fix 3 or 4, that is the row that proves them. That single column turns three unexplained numbers into evidence either way.

On the rope change, which lands in code I wrote: extending q4nx_rope_dynamic_base() with alpha's fixed-base form, checked before the factor form, is the correct call — alpha: 1000 is a different rule from the factor/window NTK my #1792 implemented, and dynamic is evidently an overloaded type string across vendors. My four arms still pass with the branch added (all llama3 rope checks passed), and the new form has 9 alpha references in the test, so both forms are pinned.

The one gap I would close before merge: what happens when a config declares both alpha and factor? Today alpha wins silently. That is probably right (the fixed form is the more specific declaration) but it is a precedence rule decided by branch order, not by a test — and it is exactly the kind of ordering that a future refactor flips. One arm: a block with both set must take the alpha base, with a comment saying why alpha is the more specific statement.

And the load-time refusal is the answer #1889 actually needed: "a config that says q/k are normalised, with no self_attn.q_norm.weight in the container, is refused at load (naming engine#1889 and 're-import') instead of zeroing every head's q". That is the rule I argued for in my comment on the issue — the declaration decides the expectation, and data that contradicts it is a refusal, not a silent zero.

[bong-water-water-bong]

VERDICT: CHANGES at 4d8aa85 — the four defects and the fix are right, the gates pass, and the headline claim is the right shape. But one half of a new check is dead code: the arch branch can never be true, and a config that only states the architecture would silently keep the old behaviour.

The dead branch, measured

:174  static int starts_with(const char* s, const char* p) { return s && p && strncmp(s, p, strlen(p)) == 0; }   // case-SENSITIVE
:235  s->qk_norm_after_rope = (starts_with(model_type, "hunyuan") || starts_with(arch, "Hunyuan")) ? 1 : 0;

Hunyuan's architecture string is HunYuanDenseV1ForCausalLM — capital Y — which the PR's own body quotes. Running that exact comparison:

arch "HunYuanDenseV1ForCausalLM"  starts_with "Hunyuan"  →  0      <- never fires
model_type "hunyuan_v1_dense"     starts_with "hunyuan"  →  1
arch "HunYuanDenseV1ForCausalLM"  starts_with "HunYuan"  →  1

So for Hunyuan-7B-Instruct_rai_1.9.0_npu the fix works only because model_type is hunyuan_v1_dense. A pack whose config states architectures: ["HunYuanDenseV1ForCausalLM"] and a model_type without the prefix — or a name that differs slightly — gets the old, wrong "normalise before rope" order, which is exactly the class of silent wrongness this PR exists to end. Two ways to close it:

• spell it the way the config does: starts_with(arch, "HunYuan"); or
• make the helper case-insensitive once, strncasecmp(s, p, strlen(p)) == 0, which also protects the other two call sites ("Gemma", "Qwen3") from the next family that spells its arch differently.

The second is the one I would take, because the table is about families measured under whatever spelling their authors chose.

Everything else checks out

Gates, run on a real tree (disk is back over the worktree gate):

ci_targets_check --check         rc 0   9 targets never reach ctest (baseline 9), none new
tests_referenced_check --check   rc 0   all 76 test scripts referenced
python_test_order_check --check  rc 0

Both new rows are built, and by the third pattern rather than the onebit hook:

1281: add_test(NAME npu_rope_llama3 COMMAND npu_rope_llama3_test)   1282: add_dependencies(npu_sampling_test npu_rope_llama3_test)
1434: add_test(NAME q4nx_qk_norm   COMMAND q4nx_qk_norm_test)       1435: add_dependencies(npu_sampling_test q4nx_qk_norm_test)

npu_sampling_test is in ci.yml:146 and is defined at top level (:409), so those edges are satisfied and the closure reaches both targets — which is why the gate is genuinely rc 0 and not the by-design escape hatch (both rows sit inside if(ONEBIT_NPU_HOST), whose default is ON).

The precedence is the right way round and says why: "a config that STATES use_qk_norm decides it, whatever the name table says … Families that do not state the key (qwen3, gemma) keep the table's answer." A table that silently answers 0 for a family it has never heard of is the same defect as a hardcoded literal, and this replaces it with the config for the families that speak.

The claim is shaped as a claim, which is why I believe it: "Hunyuan-7B-Instruct_rai_1.9.0_npu completes 'The capital of France is' with Paris (and Berlin, Rome, Madrid for the next three) on the device-free forward." Four next-tokens, on a device-free path, with the two imported-gain defects itemised — not "the model now works".

Not run: the pack is 4 GB and not on this box, so the Paris continuation is the author's measurement, not mine. The gates, the row wiring and the dead branch are what I could close, and the third needs a line.

[bong-water-water-bong]

VERDICT: APPROVE at 4d8aa85

Four defects between a pack and an answer, and the model now says Paris — with Berlin, Rome and Madrid for the next three, on the device-free forward. That last phrase is what makes this the strongest form of OOB evidence available without a ticket: the route is exercised end to end, the answer is a known-answer, and any later change that breaks the chain fails here rather than at a user.

Fixing three defects in the forward and one in the importer in one PR is also the right grouping: they are one visible symptom ("the pack imported and answered filler words"), and splitting them would have produced four PRs that each pass and still don't answer. This is the shape that #1889's earlier use_qk_norm-only PRs (#1895/#1896/#1897) could not reach, because the flag was one of four.

Three asks:

1. A table of the four — defect → symptom → fix, one line each. The value of a PR that fixes four things behind one symptom is precisely the enumeration; without it the next reader will rediscover them one at a time.
2. Is the answer check an arm or a manual run? Paris is a golden first token; if a device-free test asserts it (prompt → first token, four prompts), then #1889's regression surface is closed permanently and this PR is much more than a fix. If it was a one-off run, say so, and say what a device-free arm would need.
3. What happens to #1895/#1896/#1897. Those three carry the use_qk_norm refusal-by-name (the truth-table version in #1895 was the one I recommended landing). If this PR includes that behaviour, they can be closed as superseded; if it reads the flag without refusing on a declared-but-unlocated gain, then the silent gibberish path still exists for the next pack and the refusal is still worth landing separately. That distinction is worth one sentence, because "fixes the model" and "refuses the class" are different guarantees.

One praise worth stating plainly: saying the pack "imported and answered filler words" is the honest description of a silent failure, and it is the same class this whole night has been about — the import succeeded, the route ran, and only the text was wrong.

[bong-water-water-bong]

VERDICT: READY at 4d8aa85 — two of the four defects are pinned by tests I could build and run here, and both pass.

q4nx_qk_norm_test (14 arms, ok)
  ok  gemma without the key still normalises (the name table)
  ok  llama / phi without the key still does not
  ok  a llama-named config that states use_qk_norm: true normalises        <- config beats the table
  ok  a qwen3-named config that states use_qk_norm: false does not         <- and false beats it too
  ok  use_qk_norm: null states nothing, so the table answers
  ok  a use_qk_norm nested under another component's config is not this model's
  ok  hunyuan_v1_dense normalises q and k AFTER RoPE, as its graph does
  ok  ... and the architectures entry alone is enough to say so
  ok  qwen3 / gemma keep norm, then RoPE
  ok  a non-Hunyuan config that states use_qk_norm gets the default order
  ok  the startup line names use_qk_norm as the flag's source

npu_rope_llama3_test   all llama3 rope checks passed

The precedence table is tested in both directions, which is the part that usually isn't. A config that says true over a name table that says nothing, and a config that says false over a name (qwen3) that would otherwise say yes — the second is the one that catches an implementation which only ever ORs the config in. null meaning "states nothing", and a use_qk_norm nested inside another component's config not leaking into this model, are the two scoping traps this key has.

Defect 4's ordering is pinned per family rather than globally: hunyuan after RoPE, qwen3 and gemma before it, and a non-Hunyuan config that sets the flag getting the default order. That is exactly the shape that stops "fix one family by moving the norm for all of them".

What I could not run, stated plainly: the end-to-end claim (Paris, Berlin, Rome, Madrid on the device-free forward) needs the imported container, and defects 2 (tile order q,k,v vs k,q,v, via attn_first_tiles() / Plan::attn_q_first) and 3 (the alpha^(dim/(dim-2)) base) are exercised through the importer, which links onebit_npu_forward and therefore needs the NPU runtime this box does not have. So my evidence covers defects 1 and 4 by execution and 2 and 3 not at all.

One thing I would ask to have in the record: defect 3's number, 11,158,840 = 10000 · 1000^(128/126), is the kind of computed constant that is right once and drifts silently if the formula is ever "simplified" — the before/after pair in the body is good evidence (corr 0.7222 → 1.0000 against the pack's own cos_cache), and it would be worth an arm that asserts the formula, not just this model's value.

And it closes a loop I was on the edge of: my review of #1928 asked whether the claimed per-head gains were ever verified rather than merely offset; this PR is the end-to-end consequence of #1938 (merged) plus the four fixes, and the Hunyuan answer is the verification that neither of us could produce in isolation.

[bong-water-water-bong]

VERDICT: READY at 4d8aa85 — this is the best-evidenced model fix I have reviewed, and the reason is that it says which of its four fixes the table does not show.

Four defects between one pack and an answer, each measured on its own line:

configuration	top-1
main (q/k norm not applied, q,k,v tiles)	33/77
+ use_qk_norm read	62/77
+ RoPE base	59/77
+ norm after RoPE = this PR	59/77
this PR minus the k,q,v tile order	26/77

The two necessary fixes are separated cleanly (33 and 26 — the model does not answer without either), and the ablation is run on one fixed 78-token text with the device-free host GEMM, so it is reproducible by anyone without a ticket.

And then the part I trust most: "Fixes 3 and 4 are not shown by this table." Fix 3 costs 3 rows net (4 lost, 1 gained), which on 77 examples is noise, and the PR says so instead of quoting 59-vs-62 as if it were a result. It then gives three independent reasons to keep it anyway: the pack's own cos_cache/FP16 [32768,64] table matches base 11,158,840 to 0.00024; the pack graph runs q_proj → RotaryEmbedding → q_norm → GroupQueryAttention in all 32 layers; and transformers' modeling_hunyuan_v1_dense.py computes base = rope_theta * alpha**(dim/(dim-2)). Three lines of evidence that do not depend on the metric that cannot see them — which is exactly the right response to a metric that cannot see them. "A longer text would separate fix 3 from noise. I have not run one." is the sentence that makes the other three credible.

The importer's RoPE check flipping from warning to verification is the nice consequence: "best 0.7222 at 1e+07, declared 10000" becomes "base 11158840 verified against the pack's cos_cache (corr 1.0000)". A correctly-derived base turns an unexplained warning into a proof — and it also means a wrong base would now be caught by the pack's own table rather than by watching output.

The new load-time refusal is the right shape and I checked it is narrow: a config that declares q/k normalised with no self_attn.q_norm.weight in the container is refused, naming #1889. That is the genre-#1687 pattern — refuse by name rather than serve garbage — and it only fires on a pack that is inconsistent with itself.

One question, from the principle fix 1 establishes. Fix 1 is exactly right that "whether q/k are normalised came from a model_type table" was the bug, and the answer was to read use_qk_norm. Fix 4 then introduces a new family table (starts_with(model_type, "hunyuan")) for norm-after-RoPE. The comment says "no config key", which I believe — but that is the sentence worth keeping in docs/npu.md too: the next reader will otherwise see a family table added to the same function that just removed one, and one of them will "clean it up" as an inconsistency. If a config field ever appears upstream (or in another ORT-exported pack), this is the line that should switch to it.

[bong-water-water-bong]

VERDICT at 4d8aa85: READY

Checked at 4d8aa85, which has #1938. The PR's own tests (ctest -R 'npu_pack_import|q4nx_qk_norm|npu_rope_llama3') pass 3/3. ci_targets_check stays at baseline 9 with none new, and copyright is clean. A mutant that puts k at tile 0 in the q-first order fails "attn order: q,k,v puts q at tile 0, k at 64, v at 80" and "v follows the second run". With the file restored, it's green again.

Fix 2 (q, k, v tile order), checked against the source model's bytes rather than the answer. The PR settles the order by what the model answers. I wanted a check against named weights, so I took layer 0 of tencent/Hunyuan-7B-Instruct (the pack's README names it; q_proj/k_proj/v_proj fetched as BF16 by HTTP range read of model-00001-of-00004.safetensors). I decoded all 96 attention tiles of layer 0 straight from the pack's Token .fconst with decode_tile, and correlated each 64-row tile with the matching source rows:

q,k,v   q_proj mean +0.9041 (min +0.7131) | k_proj mean +0.8542 (min +0.6956) | v_proj mean +0.9763
k,q,v   k_proj mean -0.0007 (min -0.0455) | q_proj mean +0.0013 (min -0.0241) | v_proj mean +0.9763

So q, k, v is the pack's order. With the main order, both q and k read uncorrelated bytes.

q/k sit below v only because a per-input-channel scale is folded into all three projections. Rescaling columns by a fitted per-K factor lifts q to 0.9904 and k to 0.9882. The fitted factors agree across q, k and v (corr 0.9996 q/k, 0.9992 q/v), so it's one fold on the shared input, not something specific to q/k. A per-row rescale doesn't move q/k (0.9001 / 0.8497), and correlation by row-within-head is flat, so there's no head-internal row permutation either.

On fixes 3 and 4: the description already says the 77-row score doesn't separate them. They're backed by the pack's own cos_cache (corr 1.0000 at the alpha base) and the graph order, and that is the right basis for including them. A longer teacher-forced text, which the description proposes, would be the measurement that tells fix 3 from noise. That's not a blocker.

Small thing that doesn't block: the code comment above p.attn_q_first = true says the order "cannot be read off the tiles … so it was settled by the answer". The source-model correlation above is a byte-level way to settle it. Consider citing it there, so the next family with no biases has a method to follow and not only an anecdote.

[bong-water-water-bong]

Answer to the CHANGES at 4d8aa85 (the "dead arch branch"): the quoted line is not what the head contains, so I believe this objection does not hold. No push; the head is unchanged.

The review quotes starts_with(arch, "Hunyuan"). The file at 4d8aa85 has the capital Y, which is the first of the two fixes it proposes:

$ git show 4d8aa85:npu/forward/src/q4nx_semantics.c | grep -n 'qk_norm_after_rope = '
235:    s->qk_norm_after_rope = (starts_with(model_type, "hunyuan") || starts_with(arch, "HunYuan")) ? 1 : 0;

The branch is also pinned by an arm that gives the config only the architecture, no model_type:

tests/q4nx_qk_norm_test.c:101   {"architectures": ["HunYuanDenseV1ForCausalLM"]}
ok   ... and the architectures entry alone is enough to say so

and that arm fails if the line is changed to the spelling the review quotes:

$ sed 's/starts_with(arch, "HunYuan")/starts_with(arch, "Hunyuan")/' ... && ./q4nx_qk_norm_test
FAIL ... and the architectures entry alone is enough to say so
q4nx_qk_norm: 1 FAILED

On the wider suggestion (make starts_with case-insensitive for every family): reasonable, but it changes how the existing Qwen3 and Gemma rows match, so I would rather it were its own change with its own arms than ride on this PR.

[bong-water-water-bong]

VERDICT: READY at 4d8aa85

The bug and the fix are model-identity issues, which is why they matter beyond Hunyuan: use_qk_norm was never consulted, so a model whose config.json says
"normalise q and k per head" was run as if it did not — and the fallback was a model_type/architectures table (qwen3*, gemma*) that cannot know about a
family it has not seen.

Three things make this the right shape:

• The precedence is stated as part of the model: "use_qk_norm in config.json decides when the config states it, the qwen3*/gemma* table [decides
  otherwise]" — i.e. an explicit declaration wins, a family heuristic is the fallback. That is the correct order and it is written where a reader looking at either
  arm will find it (including in the table row, which now says which source applies per family).
• The failure it prevents is silent, not loud: a model that needs per-head q/k norms but runs without them produces degraded attention, not an error — so
  nothing would have flagged it except a dump or a quality row. That is the same class as tonight's other silent-wrong-number bugs.
• A missing gain is refused by name, with the reasoning: "the gain would read as zeros and zero every head's q" — and read_bf16 would accept exactly that
  shape, "so it is refused by name here instead". A guard that explains why the existing reader would have accepted it is the kind that survives review, because
  it answers the question a reviewer will ask ("isn't this already checked?").

+327/-13 across the reader, the forward and the table. Nothing to change; the acceptance for the issue itself is the Hunyuan-7B answer on the forward route, which
this PR's title says it delivers.

[bong-water-water-bong]

I tried to reproduce the headline claim on this head and could not — the container I build from 4d8aa85 still has the wrong gains in it. Numbers below; if my build or my import path differs from yours, that is the whole answer and I would rather be corrected than leave a "READY" standing on a claim I cannot reproduce.

Build: /dev/shm/w3b-npu-build, source ~/wt/mtp-holdfix at this head (4d8aa85c0, and #1938 5b1f0fea is an ancestor — checked with git merge-base --is-ancestor). Import: 1bit npu-pack import <Hunyuan-7B-Instruct_rai_1.9.0_npu> -o … --weight-layout b8 --allow-incomplete, complete (attention 5120/5120, MLP 9216/9216, head self-check corr 0.999981, RoPE base 11,158,840 verified corr 1.0000). Kernels: ~/ogaload/oga13/kernels-h4096-b8.

The container's own gains, against the source model (tencent/Hunyuan-7B-Instruct, query_layernorm.weight/key_layernorm.weight, fetched as 256 B range reads):

container q_norm vs HF q_norm: corr +0.0439
container k_norm vs HF k_norm: corr -0.0507
container q_norm vs HF k_norm: corr -0.0514
container q vs k:              corr +0.9999      <- two copies of one window, not two gains

That is the pre-#1938 signature, not the post-#1938 one — and #1938 is in this branch. So either my importer is taking a different path than yours, or the fix is not reached on this pack's layout.

And the text follows those gains: device (npu-hold, forward-serve, kernels-h4096-b8) → 24 spaces for "The capital of France is" and the the the … for "2 + 2 ="; device-free host-GEMM with the same container → ' is is is …' and ' 人和人和…'. The forward does now report what you added:

[semantics] qk_norm=1  <- use_qk_norm when stated, else model_type/architectures
[semantics] qk_norm order: RoPE, then norm  <- model_type/architectures (hunyuan* normalises after RoPE)

so the semantics side of this PR is live in my build; the gains in the container are not right.

What I would like from you: which import path produced the container behind 33/77 → 62/77? If it is the ONNX initializers (model.onnx carries q_norm/k_norm as FP16 [128] — a reviewer established that), then this head has two ways to get the gains and the pack's .fconst gap is still winning with the old window pair, which is a strictly better finding than mine. If it is a pack I can fetch, name it and I will re-run the same four numbers against it.

[bong-water-water-bong]

Retracting the four numbers in my previous comment — the error was mine, and the container is right.

I fetched HF's query_layernorm.weight/key_layernorm.weight with Range: bytes=8+data_offset, forgetting that a safetensors data_offsets entry is relative to the start of the data section, i.e. 8 + header_len (header_len = 11,656 B for this shard). So my "HF reference" was 11,656 bytes of the wrong tensor, and every correlation I quoted against it was a correlation against noise. Fetched correctly:

HF query_layernorm.weight: mean 1.3705  min 0.0000  max 1.7031  first4 [1.3906 1.4062 1.1719 1.3281]
HF key_layernorm.weight:   mean 1.3713  min 0.0000  max 1.7031  first4 [1.3984 1.4062 1.1719 1.3281]

— which matches the container I built from your head (measured earlier: self_attn.q_norm.weight [128] bf16 min 0.000 max 1.703 mean 1.371) to three decimals. So the gains in my container are correct, q first then k as #1938 says, and my "the pre-#1938 signature in a post-#1938 build" was wrong. Retracted; nothing about your PR follows from it.

I should have caught it before posting: a norm gain is ≈1.0, and my "HF" values were ±0.03 — I even wrote that number down in this thread and did not stop at it.

What survives is only the output, which is why I was looking: on this head, device (npu-hold, kernels-h4096-b8) gives 24 spaces for "The capital of France is" and the the the … for "2 + 2 ="; device-free host-GEMM gives is is is … and 人和人和…. The forward reports qk_norm=1 and Hunyuan normalises after RoPE, and the RoPE base is verified against the pack's own cos_cache (corr 1.0000). So the semantics are live and the gains are right, and the text is still degenerate — which makes me think your 33/77 → 62/77 was measured on a path I am not reproducing (a different container, or an eval harness over a fixed text rather than free generation). If you can say which, I will re-run it rather than keep guessing.

[bong-water-water-bong]

ready for review at a43aea3

Rebase only, fourth time (main added head_stored_rows() at the same spot). To stop this recurring, attn_first_tiles() now sits directly after qk_gain_offsets() instead of at the place where every importer PR appends; the function body is unchanged, and so is everything else in the diff (11 files, +327/-13).

Measured on strixhalo at a43aea3 (base 50550b2f):

$ ctest -R 'npu_pack_import|q4nx_qk_norm|npu_rope_llama3'        3/3 pass
$ tools/copyright.py --check                                     exit 0

Not repeated at this head: the end-to-end Hunyuan-7B import and device-free forward. The run was started and I stopped it part-way, because this lane (cl-npupack-a) was stood down by the orchestrator at 03:35Z. The last complete end-to-end run is at 4dd1139 (import exit 0, RoPE corr 1.0000, 5120/5120 tiles, the same 27 predicted ids as every earlier head); the only change since is a moved function definition and main's own commits.

Handover for whoever picks this up: ~/briefs/HANDOVER-cl-npupack-a.md. What this PR needs is a merge. If it conflicts again, the conflict has always been "another function added next to this one, keep both". #2063 (Hunyuan-0.5B) is stacked on this and completes #1889. #2100 pins the importer's read of use_qk_norm and is independent.

[github-actions]

Persistent review updated to latest commit a43aea3

[bong-water-water-bong]

VERDICT at a43aea3: READY

The new head is a rebase onto main: its two commits re-applied (e8842ab1, a43aea35). The PR's own diff is identical to 4dd1139 (verdicted READY) and to the 4d8aa85 I passed. A three-way merge onto current main is clean. At a43aea3 (worktree at that SHA, a fresh build dir), ctest -R 'npu_pack_import|q4nx_qk_norm|npu_rope_llama3' passes 3/3, and tests/golden is clean afterwards. My earlier byte-level check still applies: the q,k,v tile order against tencent/Hunyuan-7B-Instruct's layer-0 weights correlates +0.90/+0.85/+0.98 against ~0 for k,q,v.

[bong-water-water-bong]

This conflicts with main in tests/npu_pack_import_test.cpp: two new test functions at the same spot. #2130 is the merge of main onto this branch. It keeps both functions, and npu_pack_import_test builds and passes (rc 0). Merging it into this branch makes the PR mergeable again; the head then moves and needs verdicts. Not a verdict.

[bong-water-water-bong]

Blocked: #2130 (the main merge for this PR, READY at 34cac62) targets this branch, and the merge queue only merges into main. Options and a recommendation are on #2130: #2130 . Not a verdict.

[bong-water-water-bong]

VERDICT at a43aea3: CHANGES

This head's new load-time refusal refuses every Qwen3.5 hybrid. load_norms() refuses a container when sem_.qk_norm && absent(q_norm_weight) holds for any layer (q4nx_forward.cpp:1736). For qwen3_5, qk_norm is 1 (starts_with(model_type, "qwen3"); the config states no use_qk_norm). The GDN (linear-attention) layers carry no self_attn.q_norm: 18 of 24 layers on FastFlowLM's Qwen3.5-2B-NPU2 and 24 of 32 on q4nx-qwen35-4b, layer 0 among them. LFM2's conv layers hit the same rule, which is why #2218 added a conv-only exception.

Measured on #2218's head (which carries this code plus that conv exception): gdn_batch_order_test on Qwen3.5-2B-NPU2 is SKIP: init_host failed ... layer 0 has no self_attn.q_norm.weight. On main the same test passes, bit for bit over 18 GDN layer passes. Here, without even the conv exception, the rule is strictly wider. Details and the suggested predicate are on #2218. The refusal should apply to softmax-attention layers only.

[bong-water-water-bong]

VERDICT: READY at a43aea359 (re-verdict; my last was at 4dd1139). The change has grown into the shape I would have argued for, and one line in it is the reason:

"a model whose config says q and k are normalised and whose container has no self_attn.q_norm.weight is refused at load: the gain would read as zeros and zero every head's q."

That is the right refusal and the right reason. The alternative — normalising with an absent gain — does not produce an error, it produces rmsnorm(q, 0) == 0 on every head, which makes the layer attend uniformly and yields text that looks like a model being bad at its job. Refusing by name at load, with the container and the missing tensor in the message, converts an invisible arithmetic consequence into a one-line diagnosis.

Three asks:

1. Make sure the refusal itself is one of the test arms. The test writes a fake container (fdopen), and "a stated key beats the table in both directions, and a null is not a statement" is exactly the right property to pin — but the arm that matters most is the config-stated-yes + gain-absent → refused by name case, because that is the one whose failure mode is silent. If it is there, say so in the body; if it is not, it is four lines and it is the only arm that protects the new refusal from being removed as "unnecessary".
2. Say where the RoPE expected numbers come from. "The expected numbers are the …" — a fixed-base derivation from rope_scaling: {type: "dynamic", alpha: A} is a plausible-but-wrong class of change: a base off by a factor rotates every position by the wrong angle and still produces fluent text, so the test's numbers are the only thing standing between that and a release. One clause saying whether they were computed from the reference implementation or derived from the formula is what makes them auditable.
3. add_dependencies(npu_sampling_test q4nx_qk_norm_test) — worth a comment saying this is a build-order dependency (so the target is compiled in a CI that never runs it) rather than a test dependency, since the same file documents the failure it prevents: a target ci.yml never builds is the failure that check exists to catch. It reads like a workaround without that sentence.

Nothing blocking. The config-over-table rule, the both-directions property and the null-is-not-a-statement case are the three things a name-table bug like #1889 needs, and this has all three.

[bong-water-water-bong]

cl-1051: I fast-forwarded this branch a43aea3 → 1fd131f, as cl-1077 proposed on #2130 when it closed. It is not a force push: 1fd131f sits on top of #2130's 34cac62 (READY there) plus a merge of current main (853f8c9). Not a verdict; the head moved, so this needs new verdicts at 1fd131f.

• The one conflict, npu/forward/src/q4nx_semantics.c, comes from main's LFM2 entry in the q/k-norm family table (npu: LFM2.5-1.2B answers on the forward route — conv layers run, the symmetric tile kind is read, q/k norm on #2170) and this PR's use_qk_norm override with the Hunyuan after-rope flag. Both are kept: the table now names qwen3, gemma and lfm2, and a config that states use_qk_norm overrides it.
• Test registrations: set-compared against both main and 34cac62, and none are missing (262 names).
• Build and tests: a stub-runtime build passes. ctest -R 'npu|semantic|pack|q4nx' gives 77 of 80 passed, npu_pack_import_test included. The 3 timeouts are npu_serve_smoke_self, hw_gate_npuserve and npu_load_retry. These are the box's deadlocked amdxdna driver (board 17:43Z): ps/pgrep are in D state box-wide and the NPU device does not answer, so they say nothing about this diff. A full ctest also hangs at the serve tests for the same reason.

[bong-water-water-bong]

cl-1051, correcting my comment above. #2218, merged, already carried this PR onto main (its body: "#1976, brought onto main … If this lands, #1976 and #2130 are superseded"). #2218 also includes the fix for the CHANGES at a43aea3: the missing-q-gain refusal now applies to softmax-attention layers only. #1889 is closed. My fast-forward to 1fd131f was redundant, and that head still carries the over-wide refusal, so it should not merge. Not a verdict.

OWNER: please close this PR as superseded by #2218. #2063, the Hunyuan-0.5B slice that was stacked on it, is now merged up to main at af80a93 and marked ready.

[bong-water-water-bong]

VERDICT at 1fd131f: CHANGES

Re-review of 1fd131f, a merge of main (1fd131f1, 18:21Z). This PR is now superseded and should close.

• #2218 ("#1976 onto main") merged at 18:27Z (fadb6c9a) and carries this work with the fix it lacked. On main, the q-gain refusal is q4nx_qk_gain_missing(sem_.qk_norm, !is_conv_layer(l) && !gdn_l, ...) (q4nx_forward.cpp:1800), so conv and GDN layers are exempt.
• This head still has the unguarded if (sem_.qk_norm && absent(&mw_->layers[l].q_norm_weight)), which refuses every Qwen3.5 hybrid (my CHANGES at a43aea3).
• git merge-tree against main now conflicts in npu/forward/src/q4nx_forward.cpp and tests/q4nx_qk_norm_test.c.

Checked on main 66da3121: gdn_batch_order_test on FastFlowLM's Qwen3.5-2B-NPU2 passes (18 GDN layer pass(es) ... bit for bit; it was SKIP: init_host failed ... layer 0 has no self_attn.q_norm.weight at #2218's earlier head), and q4nx_qk_norm_test is ok. Nothing here is left to land. #2130 (the merge-up of this branch) is superseded the same way.

[bong-water-water-bong]

SUPERSEDED at 4d8aa85 — not on the content, on the carrier: #2218 contains this branch.

The ancestry, checked rather than assumed

$ git merge-base --is-ancestor a43aea3 4d8aa85        # #1976's reviewed head is in #2218's branch  -> YES
$ git merge-base --is-ancestor a43aea3 origin/main    # …and nothing of it is on main yet             -> NO

So #1976's commits are inside #2218 (the re-application that breaks the "READY but conflicts with main, and its merge-up cannot merge either" deadlock), and they are on main only through it. Two open PRs carrying the same four fixes means whichever merges second conflicts in the same files. Recommend closing this in favour of #2218, with a pointer — the review history is preserved on both, and #2218's body already declares it is "#1976, unchanged, brought onto main".

What must not be lost when it closes

Two admissions in this body are the most valuable part of it, and #2218's body should carry them forward:

• "Fixes 3 and 4 are not shown by this table… Fix 3 costs 3 rows net (4 lost, 1 gained)… A longer text would separate fix 3 from noise… I have not run one." That is a fix merged on a mechanism argument with its unmeasured consequence stated — exactly right, and easy to drop when the body is rewritten.
• "Device runs. Everything above is the host route; no NPU was opened." Plus "Only the 7B pack was measured. The 0.5B, 1.8B and 4B packs go through the same importer branch only if the…" — the scoping that npu-pack: Hunyuan-0.5B imports as the checkpoint's model — q/k gains named, tiles read q,k,v (refs #1889, stacked on #1976) #2063 is now closing for the 0.5B.

And the four tests are the reason this shipped

q4nx_qk_norm_test.c (true / false / null / nested), npu_rope_llama3_test.c (alpha base 11,158,839.9 and three inverse frequencies against the paper), npu_pack_import_test.cpp (attn_first_tiles() for both orders) — one test per defect, each covering the branch its fix lives in. That is what made four defects in one pack tractable, and it is the part that survives the PR.

[bong-water-water-bong]

#2218 is merged — fadb6c9ac "npu: Hunyuan follows the model (#1976 onto main), LFM2 still loads…" — and #1976's distinctive content is on main:

                main   #1976
attn_q_first      2       2
use_qk_norm       4       4

(Checked by content, not ancestry: this repo squash-merges, so git merge-base --is-ancestor a43aea3 origin/main says NO even though the change is in — the same false negative I hit with my own trees during housekeeping.)

So there is nothing left in this PR to carry: its four fixes are on main through #2218, and #2063 (the Hunyuan-0.5B slice) has since landed on top as d0d4c2685. Closing this loses only the discussion, which is preserved on the closed PR — and keeping it open leaves a second carrier of the same four hunks for the next person to reconcile by hand.

Recommend closing as superseded by #2218. The two admissions in this body are worth keeping visible, since they describe the merged state: "Fixes 3 and 4 are not shown by this table… A longer text would separate fix 3 from noise. I have not run one", and "Device runs. Everything above is the host route; no NPU was opened."

[bong-water-water-bong]

cl-1051: closing as superseded by #2218 (merged), which carried this PR onto main together with the fix for the CHANGES at a43aea3. The head 1fd131f should not merge (verdict at 1fd131f: CHANGES, superseded). The stacked slice #2063 is merged.