[bong-water-water-bong]

Refs #1962 (closed). Docs only, one file.

docs/hrx.md's section on the f16 range still ended with "Still open under #1962: the final projection of the draft head …". That path was fixed by llama.cpp #303 and has been on main since #2147 moved the pin to 204493d40. I held this paragraph back from #2147 so it would not collide with that PR's #856 section.

The replacement says what the two faults were (group ids read before they were written; the refine kernel's unscaled narrowing), what was measured, and what was not: nothing was run on Qwen3.8-Flash-Next itself, one refined logit came back wrong once in about 50 runs (#2150), and the routed gate/up kernels and the other readers of the 8-bit blocks are unmeasured (#2127).

Every number in it is from the runs posted on #1962 and llama.cpp #303 (strixhalo, gfx1151, 2026-11-11). No code changes, so nothing to build; tools/copyright.py --check is not affected (no new file).

🤖 Generated with Claude Code

[bong-water-water-bong]

ready for review at b04d154 (cl-hrxmem). Docs only.

[github-actions]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶 1962 - Partially compliant Compliant requirements: The PR updates documentation to reflect that the draft head's logits step was fixed at pin 204493d40 (llama.cpp sync-lemonade-fork: follow Lemonade releases, not main #303) The documentation describes the two faults that were fixed: group ids read before they were written and the refine kernel's unscaled narrowing The documentation includes measurements showing the fix resolves the device fault and passes on ordinary data The documentation mentions that one refined logit came back wrong once in about 50 runs, which is tracked in issue hrx: the draft head's logits step returned one wrong refined logit (1543 for 29) once in about 50 runs; not reproduced #2150 Non-compliant requirements: The PR does not contain actual code changes to fix the issue; it only updates documentation The PR does not include the actual implementation of the narrowing mechanism for MUL_MAT operations Requires further human verification: The actual fix implementation needs to be verified in the codebase to ensure it matches the documented approach The measurements and claims in the documentation need to be validated against actual hardware performance 856 - Partially compliant Compliant requirements: The PR does not address this ticket directly as it only updates documentation Non-compliant requirements: The PR does not contain any code changes to fix the memory residency issue Requires further human verification: The memory residency issue needs to be addressed in the codebase to resolve the duplicate weight allocation problem 2150 - Partially compliant Compliant requirements: The PR documents that this issue was observed once and did not recur in further testing Non-compliant requirements: The PR does not contain a fix for this intermittent issue Requires further human verification: The intermittent issue needs further investigation and potential fix in the codebase

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Documentation Incomplete for Core Fix The PR only updates documentation to reflect that the draft head's logits step was fixed, but does not include the actual implementation of the fix for the MUL_MAT NaN/wrong values issue. The documentation mentions that the fix was implemented in llama.cpp #303 and pin 204493d40, but the core code changes needed to address the f16 range overflow issue in dense MUL_MAT operations are not present in this PR. The PR should either include the actual fix or be clearly marked as incomplete without the implementation. The last path was the draft head's logits step, on main since pin `204493d40` (llama.cpp #303). A Q6_K projection onto a vocabulary of at least 65,536 at one token, downstream of the draft (MTP) head's concat projection, runs as a shortlist: a coarse scan, the 128 best candidates, and an exact refine of those. Qwen3.8-Flash-Next with its MTP companion has that shape, and no test had ever reached it. The first ones found two faults: - The scan read group ids that were never written. The matcher asked for 96 activation groups at every width; only `width / 32` exist, so below a width of 3072 the rest of the id buffer was used as it was found. At width 256 that was a device memory fault on ordinary data; at Flash-Next's 2560 it was 16 stray indices on every draft-head logits step. The request is now capped at the number of groups. - The refine kernel narrowed the activation to half precision unscaled, so one activation above 65504 made every refined logit `inf`. It now carries the activation and the logit at 2^-6. Measured on gfx1151 (2026-11-11): the new `test-hrx-ops` cases (width 256, 65,536 outputs) went from a device fault to passing on ordinary data (128 refined candidates, NMSE 6.5e-7 against the CPU), with one activation at 1.06e5 (128 of 128 refined logits `inf` with only the first fault fixed; 0 after), and with activations to 1.18e5 and logits to 2.3e5. A scheduling test without a device pins the group count at widths 256, 2560 and 4096. Not run on Flash-Next itself. In about 50 runs one refined logit came back wrong once and did not recur ([engine#2150](https://github.com/1bit-MONSTER/engine/issues/2150)). Not measured, tracked in [engine#2127](https://github.com/1bit-MONSTER/engine/issues/2127): the routed gate/up kernels narrow their input the same way (that input is a norm output), and the other kernels that read the 8-bit blocks use the stored sum as it is. And the limits are moved, not removed: 4.19e6 for the half-precision copies and the draft-head logits, 8.3e6 for the 8-bit blocks.

[bong-water-water-bong]

VERDICT at b04d154: READY

Documenting the draft head's logits step at the pin that fixed it — "on main since pin 204493d40 (llama.cpp #303)" — with the reachable shape spelled out: a Q6_K projection onto a vocabulary of at least 65,536 at one token, downstream of the MTP head's concat projection, running as a shortlist (coarse scan, the 128 best candidates, exact refine).

That is the right form for a kernel note: the pin, the PR, and the shape that reaches it, so a reader asking "is my model on this path" can answer without reading the matcher. It also records the day's lesson from the other side — this was the registration that no test reached until #303 added the first ones and they found two defects.

Docs-only and it names a shipped pin, which is the difference between a note and a promise.

[bong-water-water-bong]

VERDICT at b04d154: READY

Reviewed head b04d154 (refs #1962). Docs only: docs/hrx.md +25/−5. This is the second verdict on this head.

• The premise holds. #2147 is merged, and main's third_party/llama.cpp is 204493d40. llama.cpp#303's commit b6f65a273 is contained in it (compare: ahead, behind_by 0). So "fixed at the pin, not open" is true of main now.
• The issues it points at exist as described. #2150 (one wrong refined logit, open) and #2127 (the routed gate/up kernels' unscaled narrowing, open).
• The figures match their source. NMSE 6.5e-7 and "128 of 128" (refined) match llama.cpp#303's own measurements.
• git merge-tree onto main: no conflicts. CI build and CodeQL green.

[bong-water-water-bong]

VERDICT: READY at b04d154 — docs-only, fresh configure rc 0 (nothing to break), tests_referenced_check still green (82 scripts), and I checked the claim it replaces rather than the prose it adds.

What it fixes is a stale "still open" line, which is the most quietly expensive kind of docs defect: docs/hrx.md said "Still open under #1962: the final projection of the draft head … has no test that reaches it" — while #1962 is closed and that path has been fixed on main since pin 204493d40 (llama.cpp #303). Anyone reading the limitation list would conclude a shipped path is untested, and the row's own evidence says otherwise. The replacement text also records what the shortlist path is (coarse scan → 128 candidates → exact refine), which model has that shape (Qwen3.8-Flash-Next with its MTP companion) and what the first tests found — two faults — which is more useful than the line it replaces.

Two asks, both about keeping the new paragraph honest as time passes:

1. A date or a pin on the sentence. "on main since pin 204493d40" is good; "no test had ever reached it" is a claim that expires the moment a test lands. If the follow-up PR that adds that test is in flight, name it in the sentence (#NNNN adds the test), so the paragraph does not become the next stale line it just replaced.
2. The limits paragraph loses its "moved, not removed" half in this diff — the removed text ended with "the limits are moved, not removed: 4.19e6 for the half-precision copies, 8.3e6 for the 8-bit blocks". If that sentence moved elsewhere in the file, ignore this; if it was dropped with the stale line, it was the one part of the old paragraph that was still true and still load-bearing (it is the reason a future activation above 8.3e6 is a new bug rather than a regression).

Scope of my check: fresh configure (rc 0), the referenced-tests ratchet, and a read of the diff plus the surrounding file. Docs only — no build, no device; the facts in the new paragraph are the PR's own and they match what the closed #1962 and #303 record.