npu-pack: the tied block-32 head synthesis (#1633) runs only on the real Coder-1.5B pack — no crafted arm for it or the untied refusal #1850

Activity

1. bong-water-water-bong commented on Nov 10, 2026

   

   bong-water-water-bong

   on Nov 10, 2026

   CollaboratorAuthor

   More actions

   Claiming this, and reporting a blocker before the arms: I cannot make the synthesis branch fire, and the reason looks structural rather than like my fixture.

   What the code requires

   if (onnx.matmuls.empty() && p.head_block != 0 && p.head_block != kKBlock) {   // the synthesis branch

   and p.head_block is set in the plan's block gate only when the pack's MatMulNBits ops are only the head at an off-block size, from declared/off_block, which are built by walking onnx.matmuls.

   Why those two conditions look mutually exclusive

   walk_graph() decides whether each matmul is kept:

   • placeholder with no *.packed sibling → kept.push_back(mm) (kept);
   • placeholder whose *.packed sibling is itself empty → recorded as empty_packed_sibling and continue (dropped).

   So a head at block 32 contributes to the census that sets p.head_block only when it is kept — and a kept matmul means onnx.matmuls is not empty, which fails the first condition. A dropped one leaves matmuls empty (good) and leaves p.head_block == 0 (bad). Both orders give a false.

   What I tried, and how each attempt failed

   Four fixtures, one per way a node can be shaped, each built on the crafted .fconst from test_final_norm_envelope (H 128, L 8, eps anchors at base and base + 262144) with a single lm_head.weight.MatMulNBits node at block_size 32:

   fixture	result
   qweight with real dims + real bytes at N·(K/32)·16	runs the graph-head loop and refuses with needs 4096 B, declares 64 — i.e. it was read as a weight, and matmuls stayed non-empty
   dims {} + real location/length	the same refusal (the placeholder-with-bytes shape reaches bytes_of)
   pb::placeholder (no dims, no location)	no throw, but no synthesis either: head_synthesized_note empty, no lm_head.weight in the container, and the untied twin was not refused
   real dims + real bytes, tie_word_embeddings: false	no refusal naming block 32 or the missing declaration

   The empirical check also cannot settle it here

   Qwen2.5-Coder-1.5B-Instruct_rai_1.9.0_npu is the pack #1633 was measured on, and on today's main it refuses earlier:

   the pack states no intermediate_size: neither config.json nor genai_config.json has none, and neither the initializers nor
   the MatMulNBits/MatMulNBitsBf node attributes of model.onnx (including its If branches) hold a down/gate_up pair whose K and
   N agree …
   

   so the path is currently unexercised and my reproducer for "it works on the real pack" does not run on this box today.

   What I am asking for

   One sentence from whoever merged #1633: which pack shape reaches the synthesis branch, and what does its model.onnx carry for the head node (kept, dropped, or absent)? With that, the fixture is an hour's work and the two arms are exactly as this issue describes — tied imports + note names block 32 + selfcheck_tied + synthesised head, untied refused by name with nothing written, mutation = drop tie_declared.

   If the answer is "no shape reaches it", then the honest outcome for #1633 is different from what this issue asks: not a missing arm but dead code behind a precondition that cannot hold, which is worse than untested and worth its own line on the merge-gate audit.

   Not claiming the behaviour is wrong — claiming the branch is unreachable from import_pack as written, with the four fixtures and the keep/drop rules above as the evidence.
2. bong-water-water-bong commented on Nov 10, 2026

   

   bong-water-water-bong

   on Nov 10, 2026

   CollaboratorAuthor

   More actions

   Solved, and my earlier "unreachable" report was wrong in a useful way — the branch is reachable; I was looking for the block declaration in the wrong place. PR #1882 at 4de4ac169 carries the two arms, and they pass against main unchanged.

   The answer to the question I asked above: p.head_block comes from the pack's cache/*_meta.json (read_declared()), not from the graph. My four fixtures all shaped the graph node — real bytes, dims {} + bytes, pb::placeholder — and every one of them was either read by the graph-head loop (refused for its declared length) or kept in matmuls, which fails the onnx.matmuls.empty() half. The shape that reaches the synthesis is the one the real packs have: the head declared in cache/, and no MatMulNBits node in the graph at all.

   Two more things the fixture had to be told, both of which cost a build and are in the PR body:

   • the embedding must vary — a constant fill quantises to one code with a near-zero scale, so the self-check computes 0/0 and the arm fails for the fixture's sake rather than the code's;
   • the untied arm must run with --allow-incomplete — a strict import bails at the first refusal (this crafted pack has 80 from its empty layer payload) and never reaches the head gate, so asserting there would test the bail-out instead.

   Mutation: inverting the guard makes all four tied arms fail, so the arms bite. And one real (small) finding: dropping the tie_declared term from the guard changes nothing — tied already defaults to false when the key is absent, so tie_declared && tied is exactly tied. I left it alone (it is #1633's code and not this test's subject) but it is worth a one-line cleanup by whoever owns that guard.

   Closing the issue via the PR.
3. bong-water-water-bong

   closed this as completedin #1882on Nov 10, 2026