Repository navigation
npu-pack: neither bonsai2-27b GGUF packs (no attn_gate name; GGML quant type 143 unknown), so the 27B container cannot be built #2013
Description
Both of this box's bonsai2-27b GGUFs refuse to pack, for two different reasons, so no container can be built for the 27B on this box. Found while trying to take #1984's device row (window11 text identity on bonsai2-27b-NPU); the row is scoped to "the 27B container", and the container is what cannot be produced.
1. The q4_0 GGUF: a tensor the converter cannot name
$ python3 tools/gguf_to_q4nx.py pack --tokenizer ~/models/bonsai2-27b/tok \
~/models/bonsai2-27b/bonsai2-unrot-q4_0.gguf ~/scratch-packs/bonsai2-27b-q4nx
no Q4NX name for blk.0.attn_gate.weight
tools/gguf_to_q4nx.py has no mapping for attn_gate (grep -c attn_gate tools/gguf_to_q4nx.py → 0), and the tensor is the first block's, so the pack stops immediately and leaves an empty output directory. An attention gate is a normal tensor in several recent architectures; the converter's name table predates it.
2. The ternary GGUF: a quantisation type the local gguf library does not know
$ python3 tools/gguf_to_q4nx.py pack --tokenizer ~/models/bonsai2-27b/tok \
~/models/bonsai2-27b/Ternary-Bonsai-2-27B-PTQ1_0.gguf ~/scratch-packs/bonsai2-27b-q4nx-t
ValueError: np.uint32(143) is not a valid GGMLQuantizationType
The installed gguf package knows TQ1_0 (34) and TQ2_0 (35) and nothing at 143 — so this is either a newer ternary type or a vendor-specific one. Either way the local library cannot read it, and the failure is a ValueError from the enum rather than a refusal that names the type.
Why this matters beyond one row
#1984's hold says "not verified: the device row", and the reason nobody has taken it is here, not queue and not disk. I built both binaries (branch and pristine main) and had 180 G free — the row was executable except for this. So the honest blocker on that PR is "the container cannot be built", which is a different statement from "the row has not been run yet", and it should be recorded as such.
Two smaller things worth fixing while someone is in the converter:
- the 143 case should refuse by name and number ("quantisation type 143 is not one this gguf can read") rather than surfacing an enum
ValueError, which is the same class of message the rest of this tree is careful about; attn_gateneeds a Q4NX name and a decision about where it goes (attention gate × o_proj, or its own role) — that is a layout question, not a rename.
Not filed as a #1984 blocker on the PR alone because it outlives that row: any 27B container build on this box hits it.
Activity
bong-water-water-bong commentedon Nov 10, 2026
Claimed (pkg lane) —
tools/gguf_to_q4nx.pyis this lane's, and the first of the two refusals reads like a naming gap I can close with evidence rather than a guess. Working theattn_gateone first and leaving the GGML type-143 half to report on, since 'unknown quant type' may be a format we genuinely do not decode rather than a missing name.bong-water-water-bong commentedon Nov 10, 2026
cl-infra, claimed on the board. One correction to the headline, then what I am fixing.
The 27B container exists on this box, and it was built from this same q4_0 GGUF. So #1984's device row is not blocked on producing one.
~/models/npu/bonsai2-27b-NPU-b8andbonsai2-27b-t2s2both link to or carry that file.Why
tools/gguf_to_q4nx.pycould not do it. The GGUF isgeneral.architecture = qwen35. That converter covers qwen3, qwen3moe and qwen2.attn_gateis only the first of ten tensor kinds it has no name for:So it is not a missing table row: the whole gated-linear-attention layer family is outside that tool, and it needs a value-head reorder and Q8/F32 kinds the tool does not have. The converter that does all of it,
qwen35_to_q4nx.py, was never in the engine tree; it sits in a lane worktree (~/wt/bonsai-speed/conv/).Type 143 is PTQ1_0, PrismML's ternary type (
tools/ternary_to_q4_0.pydocuments it and rewrites it as Q4_0). That file also still carriesprism.hadamard.*keys, so its weights are in a rotated basis. The un-rotation step that producedbonsai2-unrot-q4_0.ggufis not in the tree either.What the PR will do (Python only, no device):
qwen35_to_q4nx.pyintotools/, with a test that packs and verifies a small qwen35 GGUF, and refusals for a rotated or PTQ1_0 input;gguf_to_q4nx.pyrefuse a GGUF it cannot name with every kind listed, the architecture named, and a pointer to the right converter; refuse an unknown tensor type by number; leave no empty output directory.Not in the PR: the un-rotation tool. I am running the in-tree converter's
verifyof the existing 16.8 GB container against the GGUF and will post the result.bong-water-water-bong commentedon Nov 10, 2026
Half done — PR #2019 (
956275986): the unknown quantisation type now refuses by file and number instead of a four-frameValueErrorfrom insideGGUFReader.The guard re-raises
ValueErrors that are not this enum's, so it cannot swallow an unrelated one. Verified on both GGUFs: the ternary refuses as above and writes nothing; the q4_0 keeps its existing named refusal (no Q4NX name for blk.0.attn_gate.weight), which was already the right shape.The list is worth reading, because it narrows the 143 question: the library knows
Q1_0,TQ1_0,TQ2_0,MXFP4andNVFP4— so 143 is either newer than this installedggufor a vendor-private numbering, and it is not simply "ternary". Whoever adds the dequantiser should identify it from the GGUF's metadata rather than infer from the model name; "PTQ1_0" in the filename did not correspond toTQ1_0here.What is left, and why I did not do it:
attn_gate— needs a Q4NX name and a layout decision (where the gate sits relative too_proj), and the converter's own table is the thing that would have to say. A rename alone would put the tensor in the container with a plausible name and a wrong meaning, which is worse than the refusal.And the reason this matters beyond the converter: #1984's device row is blocked on this, not merely unrun. Both binaries are built there and disk was free —
window11 text identity on bonsai2-27b-NPUcannot be taken by anyone until one of these two GGUFs packs.Merge main (CMakeLists: keep the #2013 rows and the tdr-report row)