Skip to content

npu-pack: refuse a one-header vote on a pack that declares per-head q/k gains, naming both witnesses (engine#1985) - #1995

Merged
bong-water-water-bong merged 1 commit into
mainfrom
1bit/1985-header-witness
Nov 10, 2026
Merged

bong-water-water-bong merged 1 commit into
mainfrom
1bit/1985-header-witness

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

Addresses the regression in engine#1985: a one-header epsilon-anchor vote on a pack that declares per-head q/k gains is now refused, naming both witnesses, instead of being trusted.

The regression

Qwen3-1.7B imported with 0 gaps on yesterday's main, refuses with 280 today, same complete pack:

yesterday  attention_payload {o_proj_k: 2048, qk_norm: true,  tiles_measured: 64}
today      attention_payload {o_proj_k: 0,    qk_norm: false, tiles_measured: 0}
           first refusal: model.layers.0.self_attn.q_proj.weight: attention payload holds 64 tiles; q/k/v/o at 64 rows each need 96

qk_norm: true → false is the tell, as the issue says: the pack carries the two per-head q/k sections between q/k/v and o_proj, so its block is 4-header, and reading it as 2-header takes the 2-header arithmetic and refuses every attention projection. The cause is #1838's early return — if (!g.hdr.empty()) return g; — which is right for a vote that found the right list and wrong for a vote that found one on a block that has three.

The change

Neither witness wins: the vote measured where the anchors repeat, the declaration states what the block must contain. So the reader refuses and names both:

the epsilon-anchor vote recorded ONE in-block header, and this pack declares use_qk_norm, whose per-head q/k gains live in the
two sections a 2-header block does not have (engine#1985). The two witnesses disagree and neither is being preferred

header_witness_refusal(vote_hdrs, declared_qk) holds the decision. The three cases it must not fire on matter as much as the one it must: a three-header vote (a correct 4-header pack), a one-header vote on a pack declaring nothing (a genuine 2-header family), and no vote at all — that is the residue step's case, not a disagreement, and refusing there would take Llama-3.2-1B out.

Arms: 8, host-only. Proved they can fail: making the predicate always return "" fails 5 arms.

What this does not do

It does not fix Qwen3-1.7B — it stops the silent misclassification and says both witnesses, so the next reader sees a named refusal rather than 280 surprising gaps. The 4-header read itself needs #1934/#1969's measurement of what the 640 B qk_rms_packed gap holds, and the Qwen3-4B/8B rows should be re-run against this head rather than trusted from before the window.

Gate: 191/191, copyright clean. (The first suite run after the rebase showed 3 "Not Run" — serve_degenerate, q4nx_cache, npu_answer_parser — targets an incremental build had not built; building them gives 100%.)

Refs #1985, #1831, #1838, #1889.

…/k gains, naming both witnesses (engine#1985)

A regression, not a new bug. `Qwen3-1.7B` imported with **0 gaps** on yesterday's main and refuses with **280** on today's, on the
same complete pack:

    yesterday  attention_payload {o_proj_k: 2048, qk_norm: true,  tiles_measured: 64}
    today      attention_payload {o_proj_k: 0,    qk_norm: false, tiles_measured: 0}
               first refusal: model.layers.0.self_attn.q_proj.weight: attention payload holds 64 tiles; q/k/v/o at 64 ...
                                                                       rows each need 96

`qk_norm: true -> false` is the tell: the pack has the two per-head q/k sections between q/k/v and o_proj, so its block is a
4-header one -- and it is now read as 2-header, which takes the 2-header arithmetic and refuses every attention projection.
The cause is #1838's early return: `if (!g.hdr.empty()) return g;` trusts the epsilon-anchor vote's list, which for this
pack recorded ONE in-block header. That guard was right for a vote that found the right list and wrong for a vote that found
only one on a block that has three -- the #1831 defect coming back through the vote instead of through the residue step.

Neither witness wins, so the reader refuses and names both:

    the epsilon-anchor vote recorded ONE in-block header, and this pack declares use_qk_norm, whose per-head q/k gains live in
    the two sections a 2-header block does not have (engine#1985). The two witnesses disagree and neither is being preferred

`header_witness_refusal(vote_hdrs, declared_qk)` is the whole decision in one testable place, and the three cases it must NOT
fire on are as important as the one it must: a three-header vote (a correct 4-header pack), a one-header vote on a pack that
declares nothing (a genuine 2-header family), and **no vote at all** (0 -- that is the residue step's case, not a
disagreement, and refusing there would take `Llama-3.2-1B` out).

Arms: 8, host-only. Proved they can fail -- making the predicate always return "" right-fails 5 arms.

Suite 191/191. (The first run after the rebase showed 3 "Not Run" -- `serve_degenerate`, `q4nx_cache`, `npu_answer_parser` --
targets an incremental build had not built; building them gives 100%. That is the fourth time tonight a suite result was about
the build rather than the code, and the reason "1 failed" must always be read as "which target, and was it built".)

This does not *fix* Qwen3-1.7B: it stops the silent misclassification and says both witnesses. The 4-header read itself is
#1934/#1969's measurement (what the 640 B `qk_rms_packed` gap actually holds), and the Qwen3-4B/8B rows should be re-run
against this head rather than trusted from before the window.

Refs #1985, #1831, #1838, #1889.
@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY 3d2889a

The refusal is the right shape for this class, and the sentence that makes it right is in the code:

The two witnesses disagree and neither is being preferred

That is the whole design: the epsilon-anchor vote is a measurement, use_qk_norm is the pack's own declaration, and a one-header vote on a pack that declares
per-head q/k gains is impossible — the gains live in a section a 2-header block does not have. Refusing names both rather than letting either witness win by
accident, which is exactly what #1831 taught and what #1969 had to undo (a shape read as plausible while the pack's own metadata said otherwise).

The measurement in the comment is the evidence: "Qwen3-1.7B read as 2-header, qk_norm true → false, 280 refused projections, where main imported it with 0 the
day before."
A regression that changes a truth value and refuses 280 projections is not subtle in effect, but it is subtle in cause — it came back
through the vote rather than through the residue step, which is why the guard belongs here.

Two things I would want the arms to cover, since this is a disagree-and-refuse rule:

  1. the reverse direction: use_qk_norm declared with a 2-header vote must import (the guard must not fire on the agreeing case), because a guard that
    refuses whenever use_qk_norm is set would trade this bug for a worse one;
  2. the no-declaration case: a one-header vote on a pack that does not declare q/k gains must still be usable — that is the ordinary genre, and the
    !declared_qk early return says it is, but an arm makes it a promise rather than a reading.

ci_targets untouched (arms into the existing npu_pack_import_test).

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY at 3d2889a.

Built it and ran npu_pack_import_test: all npu-pack checks passed, and the new arm reports 8 cases (one-header vote vs declaration refuses; 3/1-without-declaration/0 allowed) — the refusals and the three cases it must not fire on, which is the half that usually gets left out.

The diagnosis is the right kind of diagnosis: qk_norm: true → false is the tell, and it names the mechanism rather than the symptom — a one-header vote on a pack whose per-head q/k gains imply a 2-header block is two witnesses disagreeing, and the fix refuses instead of preferring one. That is the same rule as engine#734 and the sections printer (#1975): when two places compute the same thing and disagree, say so; do not pick.

The refusal text does the work a refusal has to do — it names both witnesses (ONE in-block header vs declares use_qk_norm), says where the gains live, carries its issue, and ends with neither is being preferred. A refusal that only said "no" would have made #1985 recur as soon as someone re-derived the vote.

Two things I checked beyond the diff:

  • declare-without-vote is not refused — a pack with use_qk_norm and a 3-header vote, and one with no vote at all, both pass. That matters: Qwen2.5-0.5B on this box declares no use_qk_norm and votes one header, and my own import of it must keep working. Re-verified: import … --weight-layout b8 on the real 900 MB pack still imports (768/768 attention tiles, 2160/2160 groups, 100 % exact scales) at this head;
  • the refusal leaves stride = 0 as well as setting why, so a caller that ignores the message cannot go on to walk a zero stride.

One request, not a blocker: the comment cites Qwen3-1.7B imported with 0 the day before as the regression's evidence. If that pack can be reduced to a
crafted fixture (the test already has the machinery), the arm would stop depending on a container that lives on one box — but the eight arms as they stand
are honest about which of the two witnesses is being refused, and that is what #1985 asked for.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at 3d2889a.

This is the "two witnesses disagree → refuse, do not pick" rule applied where it matters: a one-header vote is a measurement, but a pack that declares per-head
q/k gains cannot have a 2-header block, so a one-header vote on such a pack is not evidence — it is the #1831 defect coming back through the vote instead of
through the residue step. The measured instance is exactly the shape I would want quoted: Qwen3-1.7B read as 2-header with qk_norm true → refused projections,
where main imported it with 0 the day before.

What I checked and agree with:

  • the refusal names both witnesses rather than choosing one, which is the only behaviour that cannot silently pick the wrong winner — and it is the same rule as
    the qk_norm both-directions check (#1896) and the fconst "two widths pair up" refusal (#1657). Three sites now agree, which makes it a family rule rather than a
    local one;
  • it fires before the residue step can act (g.stride = 0 on refusal, so nothing downstream consumes a half-decided geometry) — a refusal that leaves state usable
    is how the next consumer ends up reading the wrong stride;
  • the vote's list stays a measurement (the comment is careful to say it is not discounted, only insufficient on its own), so the fix does not throw away
    information that is still valid on packs without the declaration.

One thing I would want in the body: the count of packs where the refusal now fires versus imported before, as numbers — the comment gives one instance
(Qwen3-1.7B) and says "measured", but a family rule that changes import outcomes for existing packs should say how many, so the next person can tell a regression
from the intended tightening.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 3d2889a: READY

This is the piece the earlier refusals were missing: the vote's list is a measurement, but it is not the only witness.

Today's header work only refused when the vote produced nothing (#1838/#1895). This refuses when the vote produced a one-header list on a pack that declares per-head q/k norm — via header_witness_refusal(int(g.hdr.size()), dims.use_qk_norm). That is the case where the vote succeeded and was still wrong, which is strictly harder to catch: a confident measurement that contradicts a declaration is not a no-answer, and treating it as settled is how a Qwen3 block ends up with its k_norm header read as the MLP boundary.

Putting the refusal in a named function with both inputs explicit (hdr.size() and use_qk_norm) also makes the rule testable as a rule rather than as a path — the same shape #1893 used for the zero-point bit width.

CI green.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT at 3d2889a: READY — refuse rather than pick, and that is the right call because the two witnesses disagree

The regression is stated with the pair of witnesses side by side, which is what makes it a diagnosis rather than a bug report:

yesterday  attention_payload {o_proj_k: 2048, qk_norm: true,  tiles_measured: 64}
today      attention_payload {o_proj_k: 0,    qk_norm: false, tiles_measured: 0}

Same complete pack, qk_norm: true → false as the tell, and the first refusal from it. qk_norm: true on a 64-tile block is the statement that the block holds three sections (q/k/v plus the two per-head gains before o_proj), while the vote says one header — so the two sources of truth contradict each other and the block is 4-header, not 2-header.

"Refuse naming both rather than pick" is the right resolution, and it is the opposite of what #1986 did one step earlier — which is correct, because the two situations differ exactly in whether the disagreement is reconcilable: #1986 had residues that contained the vote's offset and said how many more there were, i.e. a complete count witness; here the declaration and the vote flatly contradict, and a pick would be a coin toss with a silently wrong model on one side. header_witness_refusal(int vote_hdrs, bool declared_qk) returning "" when they can be reconciled, and a message naming both when they cannot, is the right shape — the reconcilable path stays available and the irreconcilable path fails loudly.

The message is written for whoever hits it: "gains live in the two sections a 2-header block does not have" — the field, the shape, and why. That sentence is what a lane needs to decide whether the pack is wrong or the reader is, which is the only useful thing a refusal can give.

Three asks, in order:

  1. The mutual exclusion with npu-pack: a short epsilon vote no longer wins over the anchor residues (Qwen3-1.7B imports again, #1985) #1986 must be explicit. npu-pack: a short epsilon vote no longer wins over the anchor residues (Qwen3-1.7B imports again, #1985) #1986 resolves a short vote via the residue count; this refuses a one-header vote when the pack declares gains. If both land, the code has two resolutions for overlapping conditions, and the order they are evaluated in decides the behaviour. One test with both conditions true (one header and residues saying three) asserting which path wins, and why, is the thing I would want in the diff — otherwise the pair is ambiguous by inspection.
  2. An arm on the real pack: Qwen3-1.7B_rai_1.9.0_npu must now import with qk_norm true and 64 tiles (does it? the refusal is a stopgap — if the residue path can complete the list, this refusal should not fire for this pack at all).
  3. A count of affected packs: if any other pack in the corpus declares use_qk_norm and gets a one-header vote, this turns a silent misread into a refusal — good — but a refusal is a regression for a pack that used to import. Worth listing them, or stating that the corpus sweep shows none.

Sequence note: this and #1986 are two answers to the same regression, one "reconcile", one "refuse". Landing both without (1) risks the failure mode this whole queue keeps finding: two mechanisms that each look right and were never tested together.

@bong-water-water-bong
bong-water-water-bong merged commit bc67179 into main Nov 10, 2026
8 of 9 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the 1bit/1985-header-witness branch November 10, 2026 19:16
bong-water-water-bong pushed a commit that referenced this pull request Nov 10, 2026
#1948 landed on main as a squash, so its text conflicted with this branch's
rewrite of the same code (read_block32_head as a wrapper over
read_block32_units, the take_block32_head lambda, the docs section and the
test list). Every main-side hunk was #1948's text verbatim; each is resolved
to this branch's side. main's other changes (#1995, #1921) merge untouched.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
bong-water-water-bong pushed a commit that referenced this pull request Nov 10, 2026
…arbitration is defensive)

Two things the CHANGES asked for, both answered rather than argued.

1. Rebased, and the ordering against #1995 is stated in the code:
   `prefer_residue_headers()` runs FIRST (it only wins when the residues contain the vote's offset and say there are
   more), and main's `header_witness_refusal()` from #1995 is the FALLBACK when the residues do not settle it. So a
   short vote that the residues explain is adopted, and a one-header vote on a pack declaring per-head q/k gains that
   they do NOT explain is still refused naming both witnesses. The seven-arm table is unchanged and passes; the merge
   conflict with main was exactly this branch and the test reports all checks pass at the merge.

2. The reproduction claim is withdrawn. Measured on a clean build of this merge (main 8e9e871 + this change) on a
   fresh copy of the pack (2,660,258,222 B = the census size):
     npu-pack info -> gaps 0, attention_payload {qk_norm true, tiles_measured 64, o_proj_k 2048}
     "the epsilon vote recorded" printed: 0 times
   i.e. the short vote does NOT occur on current main, exactly as three reviewers reported. What I DID measure earlier
   was at 8f10bfb (280 gaps, qk_norm false, tiles 0); #1928 has since changed how the per-head q/k gains are claimed,
   which is what masked it. So the commit message's "Qwen3-1.7B imports again" is not a statement about current main and
   I have made the branch's own claim defensive instead.

What this PR is now, honestly: the arbitration is explicit and tested (seven arms, including "outright disagreement
keeps the vote" and "a longer vote keeps the vote"), the fallback to #1995's refusal is wired, and no pack on this box
takes the residue branch. It is a guard against a short vote being real, not a fix for a pack that is currently broken --
if the reviewer would rather keep only #1995's refusal, closing this loses nothing that is measurable today.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant