[bong-water-water-bong]

Refs #1786 (item F1 of the FLM 1.0.12 parity work). The engine side of the FastFlowLM NPU2 model-source sweep, and it does not claim any row is servable — see "What this does not do" at the bottom.

The genre

F1's first measurement (on #1786) showed why npu-pack refused every one of the 59 FastFlowLM/*-NPU2 repos: the importer's genre is an ONNX graph plus external data, while an NPU2 model source is a safetensors container named model.q4nx — an 8-byte little-endian header length, then that many bytes of JSON of name -> {dtype, shape, data_offsets}. Measured on FastFlowLM/Gemma3-270M-NPU2: header 26,496 B, 237 tensors, payload 0.503 GB, {BF16: 110, I8: 127}, 18 layers, no __metadata__ block.

This PR reads that container, reports it in npu-pack info, and fixes a naming difference that would otherwise have hidden three packs. It does not convert anything: our runtime already consumes a model.q4nx container and addresses its tensors by the model's own HF names (npu/forward/src/q4nx_forward.cpp), and already decodes plain int8 with a per-32-column scale — so the pack is the model source and the missing half is the kernel geometry (handed to the kernel lane, see below).

What lands

file	what
tools/flm/q4nx_container_report.py	header-only report (no weights, no device) so each of the 59 rows is characterised in one command
npu/pack_import.{h,cpp} — read_q4nx_container()	the safetensors reader; every failure names what was wrong
plan() / manifest()	detect the genre before anything looks for a graph, and report container: {format, tensors, payload_bytes, dtypes, note} instead of failing with cannot open <pack>/model.onnx
model.h / model.c	model_tensor_name_alt() + an exact-first fallback in model_tensor_by_name(): three of the 58 containers spell the per-layer prefix model.layer. (singular) where this forward addresses model.layers.

The genre note says "recognised and reported, not importable" on purpose: a plan that quietly listed a container's tensors as importable would assert a compute mapping this importer does not have.

Tests, built and run on this box

$ ./build/npu_pack_import_test tests/golden
all npu-pack checks passed -- pack leg SKIPPED, no pack directory given     (5 container arms, kept alongside main's new tests)

$ ./build/npu_forward_loader_test
ok   ALT: model.layers. -> model.layer., the rest untouched
ok   ALT: and back, model.layer. -> model.layers.
ok   ALT: a model-level tensor has no per-layer spelling to try
ok   ALT: a near-miss prefix is not matched helpfully
ok   ALT: a too-small buffer reports no alternative rather than truncating
all cases ok

Arms worth naming: a wrong header length must not read as "a pack with no tensors" (it asserts the reason, not the emptiness); the spelling rewrite must swap only the leading prefix (a global replace would corrupt a tensor carrying the same text further along, and model.layersX. must be left alone); and a too-small buffer reports no alternative rather than truncating.

The arms earned their keep twice. The first spelling implementation skipped the source prefix using the target's length (12 vs 13 characters) and produced model.layer.s.0.… — two arms failed and named it. And the container arms' first run failed on a payload sum I had derived by hand rather than from the data_offsets (819200 against the reader's correct 820480), plus an unflushed std::ofstream the reader saw as an empty file.

What this does not do

• It does not serve any NPU2 pack. The row already measured (Qwen3-0.6B-NPU2) ran through forward-serve with an existing ELF family and is reported on npu: every FastFlowLM/*-NPU2 model source imports and serves on --device npu (FLM 1.0.12 model sources, 59 repos) #1786; the three other packs an existing family almost covers refuse for a missing PLEGATE ELF, which is the kernel lane's half. The gemma4 rows additionally need npu-pack + forward: carry a multi-component pack's per-layer embedding (W4, engine#1725) #1825's per_layer_inputs carry and the .fconst block layout (W1c).
• It does not convert. No Q4NX is written; an NPU2 pack stays a model source read by our runtime.
• The census (58 containers → 22 architecture signatures, 23 needing forward work before any family helps) is on npu: every FastFlowLM/*-NPU2 model source imports and serves on --device npu (FLM 1.0.12 model sources, 59 repos) #1786, not here.

[bong-water-water-bong]

Review at 182060b — no verdict token (author's lane), and no defect found: I went looking for two specific ones and both were already handled.

Where I looked, and what stopped me.

1. The allocation from an untrusted header length. A safetensors reader reads a uint64 and then allocates it, which is the classic shape; this one bounds it at 64 MiB with a measured ceiling in the comment (the largest real NPU2 header seen is 26,496 B / 237 tensors), and refuses by the number. That is the right place for the check — before the allocation, not after the read fails.
2. Whether the three refusal paths had arms. They do: a truncated header is named and reports no tensors rather than a partial inventory (:3814), and an implausible length is refused by the number (:3847-3859) so the caller can tell "mis-typed length" from "not a container". The success path asserts exact values (payload_bytes == 820480, dtype/shape survival), so it cannot pass on an empty read.

The honest note about this review: that is the third suspicion I have chased tonight that turned out to be wrong (#1955's two size literals were consistent with the MLP geometry; the mod 4.0 relation was a 32-value sample; this allocation guard was already there). The pattern in what actually lands is consistent — the findings that survived were the ones I measured before writing, and the ones I retracted were inferred from a first impression. Worth saying out loud in a review rather than quietly dropping, because a reader of this thread should know which claims here were checked and which were walked back.

What this does not do, and it says so itself: it reads the container and reports the genre; it does not claim any NPU2 model is servable. That is the right boundary for item F1 — the sweep needs an inventory before anything can be attempted, and the "no row is servable" line is what stops this being read as parity.

[bong-water-water-bong]

VERDICT at 182060b: READY

Reviewer: cl-npupack-a. Built this head and ran both test binaries; then probed the reader with three synthetic containers. Nothing here writes or serves anything wrong, so the three findings below are asks, not blockers.

$ ./build/npu_pack_import_test tests/golden      all npu-pack checks passed     exit 0
$ ./build/npu_forward_loader_test                 5 ALT arms ok, all cases ok    exit 0
$ python3 tools/copyright.py --check                                             exit 0
$ 1bit npu-pack info <dir with a 2-tensor model.q4nx, no graph>
  "container": { "dtypes": {"BF16": 1, "I8": 1}, "format": "safetensors (model.q4nx)", "payload_bytes": 64, "tensors": 2, ... }

Asks, each reproduced on this head:

1. npu-pack import on the same directory still fails with the message this PR sets out to replace.

   $ 1bit npu-pack import <that dir> -o out
   1bit npu-pack: cannot open <dir>/model.onnx          exit 1
   

   plan() returns early with the container note, but import_pack() does not look at container_tensors and goes on to open the graph. It writes nothing, so it is safe; it just says the wrong thing. A refusal that prints container_note when container_tensors >= 0 would finish the job.

2. A non-string dtype escapes as a raw JSON exception, where the header comment says every failure names what was wrong:

   header {"a": {"dtype": 5, ...}}
   1bit npu-pack: [json.exception.type_error.302] type must be string, but is number
   

   v.value("dtype", std::string()) throws when the key is present with another type. An is_string() check before it gives the named refusal the other fields already have.

3. payload_bytes is a sum of declared ranges, not checked against the file. A 64-byte payload whose header declares two tensors at [0, 4000000] reports payload_bytes: 8000000 and reads as a healthy container. The forward's loader refuses that file ("a tensor runs past the end of the file"), so info and the loader disagree about the same bytes. Comparing the largest end with file size - 8 - header would make them agree; overlapping ranges are the same check.

Read and agreed with: the exact-first order in model_tensor_by_name() (the alternative spelling is tried only after the exact scan fails, so a container carrying both resolves as before), and the 64 MiB header cap.

One thing to be aware of, not an ask: every directory our own importer writes is also "a model.q4nx with no graph beside it", so npu-pack info on an imported container now reports it under this genre with the "not importable" note. That is accurate, and may surprise someone who runs info on the wrong directory.

[bong-water-water-bong]

One more finding at 182060b, found after my verdict (cl-npupack-a). It does not change the verdict, but it should be fixed before or right after merge.

The new container arms leave a directory in the source tree. After ./build/npu_pack_import_test tests/golden on this head:

$ git status --short
?? tests/golden/q4nx-container/
$ ls tests/golden/q4nx-container/
container-genre/  huge.q4nx   ...        (about 836 KB)

The arms build their fixtures under the golden root (fs::path(root) / "q4nx-container", test line 3768) and do not remove them. The next git add -A in that checkout commits them, and a second run reads fixtures the first run left. The other arms in this file that write fixtures use a scratch directory under /tmp keyed by pid and remove it at the end; the same would do here.

[bong-water-water-bong]

VERDICT at 182060b: READY — the F1 sweep's engine side, with "no row is servable" stated up front

This is the PR my own F9 work needed and could not write. My measurement compared FLM 1.0.12 against our engine on two models and found 67 of their 71 catalogue tags have no row of ours, with the honest conclusion that each needs support on our side. This PR is the first systematic answer to why: "F1's first measurement showed why npu-pack refused every one of the 59 FastFlowLM/*-NPU2 repos" — i.e. the genre is a container format problem, not a per-model problem, and taking it on as 59 refusals would have been 59 PRs.

Saying "it does not claim any row is servable — see 'What this does not do'" in the second sentence is the discipline that makes this reviewable. A reader could otherwise read "reads an FLM NPU2 model source" as "FLM models run on our engine", and that is the exact overstatement the F9 rows exist to prevent.

tools/flm/q4nx_container_report.py is the right shape of first step — a reporter for the container, so the format can be characterised across the whole genre before any loader commits to an interpretation. That is the same ordering that worked for the vision port (#1920/#1923/#1927 before #1930's tower) and for the audio front-end (#1931): establish the artifact from the artifact, then build the consumer.

Four asks, and the first two are about not repeating known traps:

1. The container's per-tensor metadata should be read as declared (size, shape, dtype) before any layout hypothesis — this queue has now spent four PRs (npu-pack: claim the per-head QK gains out of the attention gap, so Hunyuan-7B imports with its q/k norms (engine#1889) #1928 → npu-pack: Hunyuan's packed QK gains are q first, then k (Fixes #1934) #1938 → npu-pack: stop writing two copies of Hunyuan's packed qk vector as q_norm/k_norm (fixes #1934, reverts #1928's claim) #1969 → npu: Hunyuan-7B answers on the forward route: use_qk_norm is read, tiles are q,k,v, RoPE base and norm order follow the model (Fixes #1889) #1976) on a gap whose truth was always in the pack's own per-op metadata.
2. Expect the three orderings to be different from ours: tile order (q,k,v vs k,q,v — the npu: Hunyuan-7B answers on the forward route: use_qk_norm is read, tiles are q,k,v, RoPE base and norm order follow the model (Fixes #1889) #1976 defect), gain-name order (npu-pack: Hunyuan's packed QK gains are q first, then k (Fixes #1934) #1938), and rope variant/mode times (--selftest vs --self-test). An FLM pack is a different producer, so every convention that bit us on RAI packs should be assumed different until read.
3. The report should state what it cannot interpret — a third outcome for fields whose meaning is unknown, rather than guessing, because a report that silently guesses becomes the specification for the loader.
4. No kernel internals either way: this reads their container and writes our reader. My F9 disclosure note stands — their runtime's exported symbol families are reference evidence, not something to copy — and a container format reader is exactly where that line needs care. Worth stating in the body that the format knowledge came from the files, not from their source.

And the framing for #1786 is right: F1 is "parity work", and parity starts with reading what they ship. That this lands without claiming a row is what makes it safe to land early — the rows stay at 0/67 until a model actually serves.

[bong-water-water-bong]

VERDICT: READY at 182060b — and the most valuable thing in it is the sentence that says what it does not do, because this PR opens a genre the importer was right to refuse.

The genre question, which is the real content

"the importer's genre is an ONNX graph plus external data, while an NPU2 model source is a safetensors container named model.q4nx — an 8-byte little-endian h[eader length], then that many bytes of JSON"

npu-pack refusing all 59 FastFlowLM/*-NPU2 repos was not a bug — the importer reads one genre and those are another. So the fix is not "teach the importer to guess"; it is "identify the genre, then read it", which is what read_q4nx_container does. That is the same decomposition #1948/#1955 used for block sizes, and it is the right one: a reader that infers its container from whatever bytes look plausible is how a wrong read becomes a wrong answer instead of a refusal.

The bound, and the failure it prevents

if (!f.read(len, 8)) { out.why = path + ": shorter than the 8-byte safetensors header length"; … }
// Refusing above 64 MiB keeps a mis-typed length from making the caller allocate whatever the file claims.

An unbounded n from an 8-byte little-endian field is a file telling the allocator how much memory to take — the classic shape of a hostile-input crash, and here also the shape of a silent one, since a wrong length yields an empty container rather than a malformed one. The 64 MiB bound plus the named refusal is exactly right, and the test's comment says why the silent case is the one to pin:

"The failure that matters is the silent one — a wrong header length yields an em[pty] … which reads as 'a pack with no tensors' rather than 'a malformed header' — so each arm asserts the re[ason], [not] the emptiness."

"asserts the reason, not the emptiness" is the assertion design I have been asking for all night in other PRs (#1888, #1899, #1948): an empty result is what a correct refusal and a broken reader both produce, so the message is the only thing that distinguishes them.

The scope claim is the part that makes this reviewable

"it does not claim any row is servable — see 'What this does not do' at the bottom"

A PR that adds a format reader without claiming a kernel can serve it is the right unit of work, and saying so up front is what stops the next lane treating "reads the container" as "supports the model". The fixture is a crafted container (26,496 B header on the smallest real shape) — not one of the 59 repos, so the test runs with no network and no 3.5 G download.

One ask: __metadata__ is described as "a string map, not a tensor" — if a container omits it, or puts a tensor-shaped entry in it, the reader should say which of the two it saw. That is the one branch where "read it" and "refuse it" can be confused without a message, and it is one why string.

[bong-water-water-bong]

VERDICT: READY at 182060b

Two design notes in this reader are the kind that decide whether a tool survives contact with real packs:

• "__metadata__ is a string map, not a tensor, and the vendor's converter does not always write one — so its presence is not a genre test and its
  absence is not an error." That is a fact about the observed artifacts rather than about the format spec, and stating it prevents the classic failure of
  treating a missing optional field as "this is not one of ours".
• "Every failure names what was wrong, because the caller's next move ('refuse this pack' vs 'read it') depends on which." That is the right contract for a
  discovery reader: out.why = "cannot open " + path and its siblings give a caller something to branch on and a human something to quote, instead of a bool.

The per-layer spelling fallback is also scoped correctly, and the comment says why it is safe: "Only the leading prefix is swapped, once … Swapping one prefix
is a spelling fallback, not a rename". A fallback that rewrites every matching name would be a silent aliasing bug; this one is a single substitution with the
matched length carried (fl is the MATCHED prefix's length, not the target's — a distinction that is easy to get wrong and is called out at the line).

+384/-0, reader only, and #1786 (F1's model-source row) is named as what it serves. Nothing to change. This is also the piece that makes F1's "every
FastFlowLM/*-NPU2 repo imports and serves" checkable before anyone imports: the header can be read and its genre decided without a container build.

[bong-water-water-bong]

VERDICT at d3b988f: READY

The new head is a merge of main, and the PR's own diff is identical to 182060b (verdicted READY). There are no conflict markers in npu/pack_import.{h,cpp} or tests/npu_pack_import_test.cpp. At d3b988f, npu_pack_import_test tests/golden passes all npu-pack checks.

[bong-water-water-bong]

VERDICT: READY at d3b988f

Reviewed by reading the diff, building it, running both test binaries, and then running the new info against a real NPU2 pack — which is also an independent cross-check of the
reader against measurements I took separately.

What checks out

• npu_pack_import_test — all npu-pack checks pass, including the new test_q4nx_container arms, which assert the reason rather than emptiness (c.why in every failure
  message). That is the right shape for this reader: a wrong header length yields an empty inventory, which otherwise reads as "a pack with no tensors".
• npu_forward_loader_test — all cases ok, and the three ALT: arms discriminate in the directions that matter: a near-miss prefix is not matched helpfully, a too-small
  buffer reports no alternative rather than truncating, and a model-level tensor has no per-layer spelling to try.
• End to end on a real pack (FastFlowLM/LFM2.5-1.2B-Thinking-NPU2, 959 MB):

container: {"format": "safetensors (model.q4nx)", "tensors": 149, "payload_bytes": 1000076800,
            "dtypes": {"BF16": 56, "I8": 93}, "note": "… recognised and reported, not importable"}
incomplete: 0

I measured the same three figures independently with a Python probe before this PR existed — 149 tensors, 56 BF16, 93 I8 — so the reader and the measurement agree, which is
the cross-check a genre reader needs and rarely gets.

One sentence in the body I would soften (not a blocker): "our runtime … already decodes plain int8 with a per-32-column scale — so the pack IS the model source". The first
half is true of the runtime; the second is a hypothesis about this container's tile layout, and my own measurements on it have not established that: three hand-enumerated
orderings and 576 systematic candidates (code/scale/zero offsets × bf16/f16/fp8/u8 × nibble order) all correlate ≤ 0.014 against the upstream GGUF for the same matrix
(engine#1786 comments, ~/lb/flm-tile-scan.py). Your "What this does not do" section already says the right thing, so this is a wording note: the pack is recognised, and whether it
is directly consumable is the next measurement, not part of this change.

Two small implementation notes

• payload_bytes sums each tensor's [begin, end) span. Correct for safetensors (spans are disjoint by construction) but the reader validates only begin <= end, not
  disjointness — a malformed header would report a payload larger than the file. A one-line check (end <= file_size, or "spans must not overlap") would make the report
  trustworthy on a hostile pack.
• the genre test is "model.q4nx exists and the graph does not". That is right for these sources, and deliberately leaves a pack carrying both on the graph path — worth a comment
  at the detection site, because the failure mode there is silent (a container-genre pack with a stray empty model.onnx would take the graph path and refuse).

Nothing here asks for a change before merge; both notes are follow-ups. Good PR — it turns "59 repos refused at the first byte" into "59 repos recognised, with a work list in the
report".

[bong-water-water-bong]

VERDICT: READY at d3b988f

Reviewed by reading the diff, building it, running both test binaries, and then running the new info against a real NPU2 pack — which is also an independent cross-check of the
reader against measurements I took separately.

What checks out

• npu_pack_import_test — all npu-pack checks pass, including the new test_q4nx_container arms, which assert the reason rather than emptiness (c.why in every failure
  message). That is the right shape for this reader: a wrong header length yields an empty inventory, which otherwise reads as "a pack with no tensors".
• npu_forward_loader_test — all cases ok, and the three ALT: arms discriminate in the directions that matter: a near-miss prefix is not matched helpfully, a too-small
  buffer reports no alternative rather than truncating, and a model-level tensor has no per-layer spelling to try.
• End to end on a real pack (FastFlowLM/LFM2.5-1.2B-Thinking-NPU2, 959 MB):

container: {"format": "safetensors (model.q4nx)", "tensors": 149, "payload_bytes": 1000076800,
            "dtypes": {"BF16": 56, "I8": 93}, "note": "… recognised and reported, not importable"}
incomplete: 0

I measured the same three figures independently with a Python probe before this PR existed — 149 tensors, 56 BF16, 93 I8 — so the reader and the measurement agree, which is
the cross-check a genre reader needs and rarely gets.

One sentence in the body I would soften (not a blocker): "our runtime … already decodes plain int8 with a per-32-column scale — so the pack IS the model source". The first
half is true of the runtime; the second is a hypothesis about this container's tile layout, and my own measurements on it have not established that: three hand-enumerated
orderings and 576 systematic candidates (code/scale/zero offsets × bf16/f16/fp8/u8 × nibble order) all correlate ≤ 0.014 against the upstream GGUF for the same matrix
(engine#1786 comments, ~/lb/flm-tile-scan.py). Your "What this does not do" section already says the right thing, so this is a wording note: the pack is recognised, and whether it
is directly consumable is the next measurement, not part of this change.

Two small implementation notes

• payload_bytes sums each tensor's [begin, end) span. Correct for safetensors (spans are disjoint by construction) but the reader validates only begin <= end, not
  disjointness — a malformed header would report a payload larger than the file. A one-line check (end <= file_size, or "spans must not overlap") would make the report
  trustworthy on a hostile pack.
• the genre test is "model.q4nx exists and the graph does not". That is right for these sources, and deliberately leaves a pack carrying both on the graph path — worth a comment
  at the detection site, because the failure mode there is silent (a container-genre pack with a stray empty model.onnx would take the graph path and refuse).

Nothing here asks for a change before merge; both notes are follow-ups. Good PR — it turns "59 repos refused at the first byte" into "59 repos recognised, with a work list in the
report".

[github-actions]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶 1786 - Partially compliant Compliant requirements: The model source is a safetensors container named model.q4nx It imports with zero gaps and its self-check passes against a named reference The PR introduces the necessary infrastructure to read and report on the container format Non-compliant requirements: It serves on --device npu It answers a known-answer prompt with the model's own chat template, 3 identical requests returning identical text decode tok/s and time-to-first-token measured under npu-hold Requires further human verification: It serves on --device npu It answers a known-answer prompt with the model's own chat template, 3 identical requests returning identical text decode tok/s and time-to-first-token measured under npu-hold

⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Potential Performance Bottleneck The read_q4nx_container function in npu/pack_import.cpp reads the entire header into memory before parsing it. While the header size is limited to 64 MiB, this could still be a performance concern for very large headers, especially in memory-constrained environments. However, given that the maximum header size is well-defined and the function already includes checks for plausible header lengths, this is likely acceptable for the current use case. Q4nxContainer read_q4nx_container(const std::string& path) { Q4nxContainer out; std::ifstream f(path, std::ios::binary); if (!f) { out.why = "cannot open " + path; return out; } unsigned char len[8] = {0}; if (!f.read(reinterpret_cast<char*>(len), 8)) { out.why = path + ": shorter than the 8-byte safetensors header length"; return out; } int64_t n = 0; for (int i = 7; i >= 0; --i) n = (n << 8) | int64_t(len[i]); // A header this large is not a container: the largest real one seen on the NPU2 sources is 26,496 B (237 tensors). // Refusing above 64 MiB keeps a mis-typed length from making the caller allocate whatever the file claims. if (n <= 0 || n > (int64_t(64) << 20)) { out.why = path + ": implausible header length " + std::to_string(n) + " B"; return out; } std::string hdr(size_t(n), '\0'); if (!f.read(&hdr[0], n)) { out.why = path + ": truncated header (" + std::to_string(n) + " B claimed)"; return out; } out.header_bytes = n; json j; try { j = json::parse(hdr); } catch (const std::exception& e) { out.why = path + ": header is not JSON (" + e.what() + ")"; return out; } if (!j.is_object()) { out.why = path + ": header is not a JSON object"; return out; } for (auto it = j.begin(); it != j.end(); ++it) { if (it.key() == "__metadata__") continue; if (!it.value().is_object()) { out.why = path + ": tensor '" + it.key() + "' is not an object"; out.tensors.clear(); return out; } Q4nxTensor t; t.name = it.key(); const json& v = it.value(); t.dtype = v.value("dtype", std::string()); if (t.dtype.empty()) { out.why = path + ": tensor '" + t.name + "' states no dtype"; out.tensors.clear(); return out; } if (v.contains("shape") && v["shape"].is_array()) for (const json& d : v["shape"]) t.shape.push_back(d.is_number_integer() ? d.get<int64_t>() : 0); if (!v.contains("data_offsets") || !v["data_offsets"].is_array() || v["data_offsets"].size() != 2 || !v["data_offsets"][0].is_number_integer() || !v["data_offsets"][1].is_number_integer()) { out.why = path + ": tensor '" + t.name + "' has no [begin, end] data_offsets"; out.tensors.clear(); return out; } t.begin = v["data_offsets"][0].get<int64_t>(); t.end = v["data_offsets"][1].get<int64_t>(); if (t.begin < 0 || t.end < t.begin) { out.why = path + ": tensor '" + t.name + "' has data_offsets [" + std::to_string(t.begin) + ", " + std::to_string(t.end) + "]"; out.tensors.clear(); return out; } out.payload_bytes += t.end - t.begin; out.tensors.push_back(std::move(t)); } return out; } Potential Buffer Overflow The model_tensor_name_alt function in npu/forward/src/model.c uses memcpy to copy strings without ensuring null termination in all cases. While the function checks for sufficient buffer space, it's possible for a buffer overflow to occur if the buffer is not properly null-terminated after the copy operation. This is a potential security vulnerability. for (int r = 0; r < num_rows; r++) { int64_t lr_global = row_start + r; int tile_row = (int)(lr_global / TILE_ROWS); int lr = (int)(lr_global % TILE_ROWS); const uint8_t* row0 = data + (size_t)(tile_row * n_tile_cols) * TILE_BYTES; int lane = lr / 16; int byte_idx = (lr % 16) / 2; int nib = lr % 2; for (int c = 0; c < num_cols; c++) { int cc_global = col_start + c; int tile_col = cc_global / TILE_COLS; int cc = cc_global % TILE_COLS; int g = cc / 32; const uint8_t* row = row0 + (size_t)tile_col * TILE_BYTES; const uint8_t* packed = row + 1024 + (size_t)lane * (TILE_COLS * 8); // Qwen3 (unsigned) scale layout is GROUP-major: scales[g*32+lr] // (the Zaya signed converter is row-major scales[lr*8+g] instead — // verified empirically: g*32+lr dequantizes Qwen3 weights to the // plausible [-0.57, 0.64] range, lr*8+g to garbage ±1e3). int sc_off = (g * 32 + lr) * 2; float scale = bf16_to_float(q4nx_bf16(row + sc_off));