Repository navigation
npu-pack: refuse a one-header vote on a pack that declares per-head q/k gains, naming both witnesses (engine#1985) - #1995
Conversation
…/k gains, naming both witnesses (engine#1985)
A regression, not a new bug. `Qwen3-1.7B` imported with **0 gaps** on yesterday's main and refuses with **280** on today's, on the
same complete pack:
yesterday attention_payload {o_proj_k: 2048, qk_norm: true, tiles_measured: 64}
today attention_payload {o_proj_k: 0, qk_norm: false, tiles_measured: 0}
first refusal: model.layers.0.self_attn.q_proj.weight: attention payload holds 64 tiles; q/k/v/o at 64 ...
rows each need 96
`qk_norm: true -> false` is the tell: the pack has the two per-head q/k sections between q/k/v and o_proj, so its block is a
4-header one -- and it is now read as 2-header, which takes the 2-header arithmetic and refuses every attention projection.
The cause is #1838's early return: `if (!g.hdr.empty()) return g;` trusts the epsilon-anchor vote's list, which for this
pack recorded ONE in-block header. That guard was right for a vote that found the right list and wrong for a vote that found
only one on a block that has three -- the #1831 defect coming back through the vote instead of through the residue step.
Neither witness wins, so the reader refuses and names both:
the epsilon-anchor vote recorded ONE in-block header, and this pack declares use_qk_norm, whose per-head q/k gains live in
the two sections a 2-header block does not have (engine#1985). The two witnesses disagree and neither is being preferred
`header_witness_refusal(vote_hdrs, declared_qk)` is the whole decision in one testable place, and the three cases it must NOT
fire on are as important as the one it must: a three-header vote (a correct 4-header pack), a one-header vote on a pack that
declares nothing (a genuine 2-header family), and **no vote at all** (0 -- that is the residue step's case, not a
disagreement, and refusing there would take `Llama-3.2-1B` out).
Arms: 8, host-only. Proved they can fail -- making the predicate always return "" right-fails 5 arms.
Suite 191/191. (The first run after the rebase showed 3 "Not Run" -- `serve_degenerate`, `q4nx_cache`, `npu_answer_parser` --
targets an incremental build had not built; building them gives 100%. That is the fourth time tonight a suite result was about
the build rather than the code, and the reason "1 failed" must always be read as "which target, and was it built".)
This does not *fix* Qwen3-1.7B: it stops the silent misclassification and says both witnesses. The 4-header read itself is
#1934/#1969's measurement (what the 640 B `qk_rms_packed` gap actually holds), and the Qwen3-4B/8B rows should be re-run
against this head rather than trusted from before the window.
Refs #1985, #1831, #1838, #1889.
|
VERDICT: READY The refusal is the right shape for this class, and the sentence that makes it right is in the code:
That is the whole design: the epsilon-anchor vote is a measurement, The measurement in the comment is the evidence: "Qwen3-1.7B read as 2-header, qk_norm true → false, 280 refused projections, where main imported it with 0 the Two things I would want the arms to cover, since this is a disagree-and-refuse rule:
|
|
VERDICT: READY at Built it and ran The diagnosis is the right kind of diagnosis: The refusal text does the work a refusal has to do — it names both witnesses ( Two things I checked beyond the diff:
One request, not a blocker: the comment cites |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at 3d2889a.
This is the "two witnesses disagree → refuse, do not pick" rule applied where it matters: a one-header vote is a measurement, but a pack that declares per-head
q/k gains cannot have a 2-header block, so a one-header vote on such a pack is not evidence — it is the #1831 defect coming back through the vote instead of
through the residue step. The measured instance is exactly the shape I would want quoted: Qwen3-1.7B read as 2-header with qk_norm true → refused projections,
where main imported it with 0 the day before.
What I checked and agree with:
- the refusal names both witnesses rather than choosing one, which is the only behaviour that cannot silently pick the wrong winner — and it is the same rule as
the qk_norm both-directions check (#1896) and the fconst "two widths pair up" refusal (#1657). Three sites now agree, which makes it a family rule rather than a
local one; - it fires before the residue step can act (
g.stride = 0on refusal, so nothing downstream consumes a half-decided geometry) — a refusal that leaves state usable
is how the next consumer ends up reading the wrong stride; - the vote's list stays a measurement (the comment is careful to say it is not discounted, only insufficient on its own), so the fix does not throw away
information that is still valid on packs without the declaration.
One thing I would want in the body: the count of packs where the refusal now fires versus imported before, as numbers — the comment gives one instance
(Qwen3-1.7B) and says "measured", but a family rule that changes import outcomes for existing packs should say how many, so the next person can tell a regression
from the intended tightening.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at 3d2889a: READY
This is the piece the earlier refusals were missing: the vote's list is a measurement, but it is not the only witness.
Today's header work only refused when the vote produced nothing (#1838/#1895). This refuses when the vote produced a one-header list on a pack that declares per-head q/k norm — via header_witness_refusal(int(g.hdr.size()), dims.use_qk_norm). That is the case where the vote succeeded and was still wrong, which is strictly harder to catch: a confident measurement that contradicts a declaration is not a no-answer, and treating it as settled is how a Qwen3 block ends up with its k_norm header read as the MLP boundary.
Putting the refusal in a named function with both inputs explicit (hdr.size() and use_qk_norm) also makes the rule testable as a rule rather than as a path — the same shape #1893 used for the zero-point bit width.
CI green.
VERDICT at
|
#1948 landed on main as a squash, so its text conflicted with this branch's rewrite of the same code (read_block32_head as a wrapper over read_block32_units, the take_block32_head lambda, the docs section and the test list). Every main-side hunk was #1948's text verbatim; each is resolved to this branch's side. main's other changes (#1995, #1921) merge untouched. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…arbitration is defensive) Two things the CHANGES asked for, both answered rather than argued. 1. Rebased, and the ordering against #1995 is stated in the code: `prefer_residue_headers()` runs FIRST (it only wins when the residues contain the vote's offset and say there are more), and main's `header_witness_refusal()` from #1995 is the FALLBACK when the residues do not settle it. So a short vote that the residues explain is adopted, and a one-header vote on a pack declaring per-head q/k gains that they do NOT explain is still refused naming both witnesses. The seven-arm table is unchanged and passes; the merge conflict with main was exactly this branch and the test reports all checks pass at the merge. 2. The reproduction claim is withdrawn. Measured on a clean build of this merge (main 8e9e871 + this change) on a fresh copy of the pack (2,660,258,222 B = the census size): npu-pack info -> gaps 0, attention_payload {qk_norm true, tiles_measured 64, o_proj_k 2048} "the epsilon vote recorded" printed: 0 times i.e. the short vote does NOT occur on current main, exactly as three reviewers reported. What I DID measure earlier was at 8f10bfb (280 gaps, qk_norm false, tiles 0); #1928 has since changed how the per-head q/k gains are claimed, which is what masked it. So the commit message's "Qwen3-1.7B imports again" is not a statement about current main and I have made the branch's own claim defensive instead. What this PR is now, honestly: the arbitration is explicit and tested (seven arms, including "outright disagreement keeps the vote" and "a longer vote keeps the vote"), the fallback to #1995's refusal is wired, and no pack on this box takes the residue branch. It is a guard against a short vote being real, not a fix for a pack that is currently broken -- if the reviewer would rather keep only #1995's refusal, closing this loses nothing that is measurable today.
Addresses the regression in engine#1985: a one-header epsilon-anchor vote on a pack that declares per-head q/k gains is now refused, naming both witnesses, instead of being trusted.
The regression
Qwen3-1.7Bimported with 0 gaps on yesterday's main, refuses with 280 today, same complete pack:qk_norm: true → falseis the tell, as the issue says: the pack carries the two per-head q/k sections between q/k/v ando_proj, so its block is 4-header, and reading it as 2-header takes the 2-header arithmetic and refuses every attention projection. The cause is #1838's early return —if (!g.hdr.empty()) return g;— which is right for a vote that found the right list and wrong for a vote that found one on a block that has three.The change
Neither witness wins: the vote measured where the anchors repeat, the declaration states what the block must contain. So the reader refuses and names both:
header_witness_refusal(vote_hdrs, declared_qk)holds the decision. The three cases it must not fire on matter as much as the one it must: a three-header vote (a correct 4-header pack), a one-header vote on a pack declaring nothing (a genuine 2-header family), and no vote at all — that is the residue step's case, not a disagreement, and refusing there would takeLlama-3.2-1Bout.Arms: 8, host-only. Proved they can fail: making the predicate always return
""fails 5 arms.What this does not do
It does not fix Qwen3-1.7B — it stops the silent misclassification and says both witnesses, so the next reader sees a named refusal rather than 280 surprising gaps. The 4-header read itself needs #1934/#1969's measurement of what the 640 B
qk_rms_packedgap holds, and the Qwen3-4B/8B rows should be re-run against this head rather than trusted from before the window.Gate: 191/191, copyright clean. (The first suite run after the rebase showed 3 "Not Run" —
serve_degenerate,q4nx_cache,npu_answer_parser— targets an incremental build had not built; building them gives 100%.)Refs #1985, #1831, #1838, #1889.