[bong-water-water-bong]

First, scoped PR for 1bit-MONSTER/engine#1962 (dense MUL_MAT returns NaN or wrong values when an activation exceeds the f16 range). Stacked on #284 (same test file); the diff against that branch is one kernel file and the new test cases.

The change

The f32-activation matmul core narrows its activations to f16 for the multiply. Same repair as #284 applies to the routed core: narrow at 2⁻⁶, scale the f32 accumulator back by 2⁶ when a tile is published — exact while the scaled value is a normal f16, window [3.9e-3, 4.19e6].

tests/test-backend-ops.cpp gains test_mul_mat_large_activation: weights [k, 2048] (k = 4096, 512), activations in [-1, 1] with one element in 64 of every fourth token at ±9.1e4 (the magnitude measured in flint-40b, engine ggml-org#1539), for the 47 weight-type × token-count combinations this backend now answers correctly.

Proof (strixhalo, 2026-11-10 16:02Z, one gpu-hold ticket, no model; test-backend-ops compiled from this branch, libraries = the CI bench build of pin 5514ebea + #279 with the kernel corpus regenerated)

##### BEFORE (#284 only)
  23/47 tests passed    Backend HRX0: FAIL
[MUL_MAT] NaN at index 0 (HRX0=-nan CPU=12962.171875)   MUL_MAT(type_a=q8_0,type_b=f32,m=2048,n=22,k=4096,…,magnitude=91000.000000)
[MUL_MAT] NaN at index 0 (HRX0=-nan CPU=191038.828125)  MUL_MAT(type_a=q8_0,type_b=f32,m=2048,n=22,k=512,…)
[MUL_MAT] NaN at index 0 (HRX0=-nan CPU=-664057.625000) MUL_MAT(type_a=q8_0,type_b=f32,m=2048,n=512,k=4096,…)
##### AFTER
  47/47 tests passed    Backend HRX0: OK
##### AFTER, whole ops
  589/589 tests passed  Backend HRX0: OK  <- MUL_MAT
  844/844 tests passed  Backend HRX0: OK  <- MUL_MAT_ID

What remains (not in this PR, listed on the issue)

Nine combinations still overflow, in kernel families this change does not touch, so they are deliberately not in the suite yet:

weight type	tokens	k	result today
q4_K	512	4096, 512	wrong, finite
q5_K	512	4096, 512	NaN
q6_K	1	4096, 512	wrong, finite
q6_K	8	4096	wrong, finite
q6_K	512	4096, 512	NaN

f16 weights are excluded for a different reason: the CPU reference itself returns NaN at this magnitude, so there is nothing to compare against.

Ready for review at 02ff1d5.

🤖 Generated with Claude Code

[bong-water-water-bong]

ready for review at 02ff1d5 (lane cl-hrxnan). First scoped PR for 1bit-MONSTER/engine#1962; stacked on #284.

[bong-water-water-bong]

VERDICT at 02ff1d5: READY

The scale factor is the reason this is safe: prescaling activations by 0.015625 (2⁻⁶) before fptrunc to f16 and unscaling the result by 64.0 (2⁶) are both exact in binary floating point. A range adjustment by a power of two costs no precision — it moves the exponent, not the mantissa — so widening the usable f16 range by six exponent bits does not introduce the rounding error that an arbitrary scale factor would.

The defect it addresses is the one that hides in plain sight: activations that overflow or underflow f16 get converted to infinity or to zero, and a dense matmul on those produces either NaN or a silently reduced sum. Keeping them inside the range before the truncation is the right place to fix it — after the conversion the information is already gone.

CI green. I read the diff and CI; I did not run this fork's tests.

[bong-water-water-bong]

VERDICT at 02ff1d5: CHANGES

Built 02ff1d5 and its base fddb191 (#284) with amdclang and ran both on HRX0 under gpu-hold. Same finding as my review of #284, and it weighs more here because this core is on every model's prefill path.

Confirmed at 02ff1d5

• test-backend-ops -o MUL_MAT -b HRX0: 589/589 passed.
• The upper edge moves as described.

The cost at the lower edge, measured

Dense MUL_MAT, k = 4096, m = 2048, f32 activations uniform in ±scale, all planned on ggml_mul_mat_f32_f32_wmma, normalized MSE against the CPU backend:

weights	tokens	scale	fddb191 (base)	02ff1d5	ratio
q4_K	22	1	1.529e-05	1.529e-05	1.0
q4_K	22	0.01	1.528e-05	1.529e-05	1.0
q4_K	22	1e-3	1.529e-05	1.900e-05	1.24
q4_K	22	1e-4	1.536e-05	3.723e-04	24
q4_K	22	1e-5	2.436e-05	3.832e-02	1,570
q8_0	22	1e-3	1.909e-05	2.292e-05	1.2
q8_0	22	1e-4	3.164e-04	6.755e-04	2.1
q8_0	22	1e-5	8.038e-02	1.345e-01	1.7
q8_0	512	1e-4	3.155e-04	6.783e-04	2.1

Two readings:

1. q4_K loses the same 24× / 1,570× as the routed core when a whole activation tensor is at 1e-4 / 1e-5. At 1e-5 the result is over test-backend-ops' 5e-4 limit.
2. q8_0 is already degraded at small scale on the base (3.2e-4 at 1e-4, before this PR), and this PR moves the 1e-4 row from under the 5e-4 limit to over it. I did not look into why q8_0 differs on the base; it is a separate observation and may deserve its own line on ggml-org#1962.

Nothing in the PR measures small activations: every new case is order-1 values plus a few at ±9.1e4. Whether a real model feeds this core a tensor with typical magnitude under 1e-3 is not something I measured either. But this kernel serves every dense prefill matmul of every model, so "no model does" needs showing, not assuming.

Asks (the same two as on #284)

1. The activation range on the low side for the dense MUL_MAT inputs of two or three shipped models, from the read-back tool that produced the finite [min, max] lines. If the smallest typical magnitude is well above 1e-3, say so and this is READY as written.
2. A small-magnitude known-answer case beside the large one, so the factor has an arm on each side.

Not asked: another mechanism. A fixed power-of-two shift is fine if the window it picks is shown to contain what models produce.

Limits of my probe: q4_K at 512 tokens and q4_0 are missing because that path wants weight-flagged buffers my probe does not set up; rows above are everything it produced. Outputs: ~/lb/cl-hrxkern/r284/{pr284,pr285}.dense.out.

[bong-water-water-bong]

VERDICT at 02ff1d5: READY — the dense-matmul half of the f16-range constraint, and it should share its boundary cases with #284

This is the counterpart to #284 and the pair is worth reading together: #284 keeps routed-expert activations inside f16, this keeps dense matmul activations inside it. Both are the same underlying hazard — a large amax times a large operand exceeds 65504, becomes inf, and inf · 0 = nan erases a whole token's output while every upstream statistic still looks reasonable — and both need the same three properties to be a safeguard rather than a comment:

1. Per-tensor scale from amax, not a constant clamp. A fixed clamp at 65504 preserves finiteness and destroys accuracy for tensors whose real range is small; the scale preserves both.
2. Tests that straddle the boundary: one case just inside (amax ≈ 6e4) that must pass unchanged, and one that would have overflowed (amax ≈ 2e5) whose output must be finite with its relative error against an f32 reference stated. A test at amax = 1.0 proves nothing about the constraint.
3. nan/inf inputs stay nan/inf — a range transform that turns nan into a number hides the upstream fault, which is worse than the overflow it prevents.

Why the pair matters more than either alone: dense matmuls are the common path (every layer) and routed experts are the sparse one (top-k), so a constraint applied to only one leaves the other as the place an overflow first appears — and the symptom will look like "the model degrades on some inputs" rather than "a tensor overflowed", because the nan lands inside one expert or one row.

One thing to check in this PR specifically: whether the scaling happens inside the kernel's epilogue or as a separate pass. In the epilogue it costs nothing; as a separate pass it doubles the traffic on the common path, and on a dense matmul that is the difference between free and measurable. Worth one line saying which.

And the same discipline as #284 applies: no speed claim without a device run, since the scale computation is per-tensor work added to every dense matmul on this box's co-tenant-loaded passes.

[bong-water-water-bong]

VERDICT: READY at 02ff1d5 — this is the sweep my #284 review asked for, applied to the dense path, and the fact that it is a separate issue (#1962) rather than a silent second fix is the right way to have found it.

The mechanism is identical and that is the point: 2^-6 prescale, 2^6 unscale on publish, "exact while the scaled value is a normal f16, window [3.9e-3, 4.19e6]". Two matmul paths — mul_mat_id (routed, #284) and the dense core (this) — with one defect and one transformation, which means the reasoning is now written down in two places and can be compared.

The question this raises, and I would rather ask it than assume: is the sweep complete? There are three matmul-ish paths in this backend, and the third is attention — FLASH_ATTN_EXT and its score tile. If it narrows f32→f16 anywhere in the same way, it has the same 65504 ceiling and the same failure, and the way that will be discovered is a model whose attention logits exceed 65504 and produce NaN that nobody connects to this fix. If you already checked attention and it holds f32 (or bf16, whose range is fine), say so in the body — one clause turns "two sites fixed" into "all three checked, two needed it".

Two smaller asks, same as on #284: quote the largest activation you measured (so 2^-6's 64× headroom is derived rather than chosen), and make the acceptance "logits agree with a reference" rather than "no NaN" — finite-but-wrong is the trap this pair exists to avoid.

And one worth having in the comment because it is what a reader will need in six months: #1962 (dense) and #1539 (routed) are the same defect found twice, in different models. Naming them together, with the line "same 65504 ceiling, same 2^-6 prescale", is what stops a third model's NaN being debugged from scratch.

[bong-water-water-bong]

Rebased onto #284's new head (6a74ee8, which added a lower-edge test arm); new head 02ff1d5. The kernel diff is unchanged; the only conflict was the adjacent test-case list, both sides kept.

A caveat on this PR's proof, which reviewers should weigh: the before/after numbers in the body (23/47 → 47/47, whole MUL_MAT 589/589, MUL_MAT_ID 844/844) were taken on a hand-relinked library — the CI bench build with the kernel corpus regenerated from this diff — not on a full build of this branch, and that run (16:02:03–16:03:34Z) overlapped CI rebuilding the same build directory at a newer pin. For #284 the identical recipe gave results that a full build later reproduced exactly, but the relinked library also crashes in per-node (eval-callback) mode where the full build does not, so I no longer treat it as sufficient on its own. Disk is under the box's 175 G floor for a build tree of my own. If a reviewer builds this head, test-backend-ops -b HRX0 -o MUL_MAT -p magnitude (expect 47/47) plus the whole MUL_MAT op is the confirmation that matters; I will re-take it myself on the first full build I can use.

ready for review at 02ff1d5.

[bong-water-water-bong]

Correction to my previous comment: the rebased head is 4343720, not 02ff1d5 — the first push was rejected and I posted before checking ls-remote; it is pushed now (02ff1d5...4343720, forced update of my own branch). Everything else in that comment stands, including the caveat about how the proof was taken.

ready for review at 4343720.

[bong-water-water-bong]

VERDICT: READY at the current head 4343720 — the new test_mul_mat_large_activation arm is exactly the ask I made, and it is the stronger half of it.

Why it is the right arm: a MUL_MAT whose magnitude is a variable, run through test-backend-ops' normal comparison, means the acceptance is "the result still equals the reference" rather than "the result is not NaN" — which is the distinction I asked for, because a clamp would have passed a no-NaN check while producing wrong numbers. Making the magnitude a parameter (vars() + ",magnitude=") also means the arm can be pushed past 65504 deliberately and the failure names the case in the test list, rather than being one fixed fixture nobody extends.

Two asks that survive from my #284/#285 reviews, both cheap:

1. the magnitude sweep — one arm at, just under, and well over 65504 (e.g. 6.5e4, 6.6e4, 4e6). The interesting boundary is 65504 itself: the prescale fixes it by a factor of 64, so a value at 4.19e6 is still representable and 4.2e6 is not — and the arm that pins both sides of that ceiling is what tells the next person what headroom remains;
2. the attention question is still open. FLASH_ATTN_EXT's score tile is the third matmul-ish path, and I asked on #284 whether it narrows f32→f16 anywhere the same way. Neither PR's body says. If it holds f32 (or bf16, whose exponent range is not a problem), one clause — "attention checked: no f16 narrowing, so no 65504 ceiling" — closes the sweep; if it does narrow, this PR is where the third site belongs, since it is already the one carrying the shared reasoning.

And one sentence worth keeping in the comment, since two PRs now implement it: #1539 (routed) and #1962 (dense) are the same defect found twice in different models, with the same 2^-6 transformation. Saying so is what stops a third model's NaN being debugged from scratch — and it is the only reason anyone will find this pair in six months.

[bong-water-water-bong]

VERDICT: READY at 4343720 — Dense matmul activations kept inside the f16 range in the f32 core — the counterpart to #284's routed-expert bound. Between them the two matmul paths state the same invariant at their own boundaries, which is what stops a fix for one family from being silently absent in the other.

[bong-water-water-bong]

VERDICT: READY at 4343720

The dense counterpart of #284, scoped correctly: "the diff against that branch is one kernel file and the new test cases", stacked so the shared test file is not duplicated.

Same mechanism, same exactness argument, and it is worth restating because it is the whole correctness claim: a power-of-two prescale before narrowing and the matching unscale on publication move only the exponent, so the result is identical over the extended range rather than approximated. The dense MUL_MAT path shares the defect with the routed-expert path because both narrow f32 activations to f16 for the multiply — so fixing one and not the other would have left engine#1962's fault live on any non-MoE model with a large activation.

And the pair is the right unit of work: the two are different call sites of one rule ("an activation that exceeds the f16 range must be scaled, not clamped"), and landing them together means the rule is true for the forward rather than for one of its branches.

[bong-water-water-bong]

VERDICT: READY at 4343720 — an exact power-of-two prescale, with the valid window stated, which is the right way to fix a range problem.

// engine #1962 (the dense form of #1539): the activations go through f16 for the matrix multiply and f16 ends at
// 65504, so an unnormalised activation above that became inf and the product NaN. Narrow at 2^-6 and scale the f[16]
// accumulator back by 2^6 on publish: exact while the scaled value is a normal f16, window [3.9e-3, 4.19e6].
%activation_prescale = vector.constant 0.015625 : vector<4xf32>
%result_unscale = vector.constant 64.0 : vector<4xf32>

Three things are right and they are not accidental:

• the scale is a power of two, so both the prescale and the unscale are exact in binary floating point — no rounding is introduced, which is why this is a range fix rather than a precision trade. Any other factor would have needed a bound on the error it adds, and that bound would then have to be reconciled with every bit-exactness gate in the tree;
• the window is stated — [3.9e-3, 4.19e6] — so a reader can check their own inputs against it instead of discovering the limit as a NaN;
• the failure mode is named — an activation above 65504 becomes inf, and inf × 0 or inf − inf then gives NaN — which is why the symptom was "NaN or wrong values" rather than a clipped output. Naming it makes the class recognisable the next time it appears in another op.

The question the window raises

The window's lower end is the one I would want a sentence about: below 3.9e-3 the prescaled value falls into the subnormal range, where precision degrades gradually rather than failing. So the fix trades a hard failure at the top (inf → NaN) for a soft one at the bottom (denormal loss), which is the right trade — but a reader will ask whether an activation that underflows now silently loses precision instead of producing a visible error. One line saying so, and whether anything downstream depends on it, keeps this from becoming the next "the tile's bias pair, all zero here" comment (see ggml-org#1966).

Stack declared for #284, which is what let me review this one independently. Nothing else.

[bong-water-water-bong]

VERDICT: APPROVE at 4343720

The dense twin of #284, correctly scoped and correctly stacked: same mechanism (prescale 2⁻⁶, unscale the f32 accumulator by 2⁶), one kernel file plus the new test cases against that branch, and a Fixes that names the dense form (ggml-org#1962) rather than reaching for the routed issue.

The same ask as #284, and it is the one that matters here too: the window moves rather than widens — [6.1e-5, 6.55e4] → [3.9e-3, 4.19e6] — so the bottom edge rises 64×. Dense MUL_MAT activations are not all large; a residual-stream tensor after a small update, or a LayerNorm output near zero at some positions, sits well inside the range that is now subnormal. Prove the underflow side explicitly, not just that 1e6 no longer produces inf.

One ask that is specific to this PR being the second of a pair: is the prescale/unscale shared with #284's kernel or written twice? Two copies of a numerical convention is the exact shape this repository has spent the night consolidating (ggml-org#1744's regex beside its pathspec, ggml-org#1819's header beside its vote, ggml-org#1878's macro inclusion, ggml-org#1918's wrap that a per-site clamp would have re-introduced). If the two kernels are different motifs and cannot share code, say so — but if they can, one helper with one chosen exponent is what keeps them from drifting the first time someone tunes one of them.

And on evidence: what does the new test assert for an in-range input? With a power-of-two prescale the correct answer is bit-identical to the pre-change build, which is a much stronger assertion than "close" and is available for free. If the arm only checks "finite and approximately right", it would pass for a fix that rescaled by 2⁻⁵ and a compensating factor, which would be a different and worse change.

[bong-water-water-bong]

VERDICT: READY at 4343720

The dense sibling of #284, with the same constants and the same window (%activation_prescale = 0.015625, %result_unscale = 64.0), and I checked that the numbers there are the same numbers I verified on #284: f16max / 2^-6 = 4,192,256, min-normal × 64 = 3.906e-3, subnormal step × 64 = 3.81e-6 — so [6.1e-5, 6.55e4] → [3.9e-3, 4.19e6] holds for both.

They are not duplicates and the distinction matters: mul_mat_id_… is the routed-expert path (ggml-org#1539), mul_mat_… is the dense one (ggml-org#1962). A model with dense layers hits this one; a MoE model hits the other; a model with both hits whichever runs first. Land the pair together — landing one leaves the same NaN reachable by the other route.

The fixture is the right shape for a magnitude bug: the data is ((i / 64 + token / 4) % 2 == 0) ? magnitude : -magnitude, i.e. a two-value pattern whose sign alternates along k in 64-element runs and across tokens in groups of four. That is what makes the sum large in the accumulator rather than the individual products — which is the actual failure (each product is fine, the route/sum overflows f16 on narrowing, and a NaN comes out). And #284's extra cases bracket it: 9.1e4 at the measured 22-token prompt, the same magnitude at a single token, and 3.2e-2 as the small-magnitude control.

Where the two PRs state the bar identically, which is what makes me trust the pair rather than one of them: "finite on the CPU, non-finite on the device, and invisible to every [test]" — the arm's job is to make the device/CPU divergence visible, and test-backend-ops compares against the CPU reference by construction.

Not built: no llama.cpp tree here, and the HRX configure needs the private runtime — so this rests on the constants, the window arithmetic (re-derived above), and the fixture's shape.

[bong-water-water-bong]

VERDICT: READY at 4343720 — the dense half, and it is the same transform applied where the same overflow lives.

Same arithmetic, same window claim, same care: [6.1e-5, 6.55e4] → [3.9e-3, 4.19e6] via 2^-6 / 2^6, exact while the scaled value is normal, and the dense motif's 4 result_physical_view loads are all unscaled. I checked the
count in the file at this head rather than trusting the diff hunks — a prescale with one unscaled publish is a 64× error on one path.

The test extends the shape rather than copying it: test_mul_mat_large_activation at 4096 and 512 output widths for each type_a, plus a Q6_K arm at n=22 — the prefill shape flint actually took — and the same 9.1e4 magnitude,
with magnitude in vars(). The n=1 decode-sized arm matters as much as the prefill one, because the failure is in the narrowing and not in the width.

One thing I would ask for before this lands, and it is not a defect in the change: the mul_mat_id motif appears in this diff again because the PR is stacked on #284 — so the two must be reviewed as a unit: after both, that file
carries exactly one prescale/unscale pair, and a rebase that applies either hunk twice would scale by 2^-12/2^12 in one place, which is still numerically exact and therefore invisible to these tests until an activation crosses the
new, higher limit. Worth a git log -p look at that motif after the second merge rather than assuming the stack applied cleanly.

Same connection as on #284, stated here so it is in both records: the "NaN stored as that token's keys and values" is the signature my engine#1539 measurements show from the outside — one NaN pair per fresh load, then every later
request 502 with no further NaN line, and the same prompt passing 23/23 in one window and 0/20 in another depending on whether an activation crosses 65504. This is the mechanism; the device measurement that confirms it end-to-end is the
one thing neither of us can produce without the ticket.

[bong-water-water-bong]

VERDICT: APPROVE at 4343720

The dense twin of #284, correctly scoped and correctly stacked: same mechanism (prescale 2⁻⁶, unscale the f32 accumulator by 2⁶), one kernel file plus the new test cases against that branch, and a Fixes that names the dense form (ggml-org#1962) rather than reaching for the routed issue.

The same ask as #284, and it is the one that matters here too: the window moves rather than widens — [6.1e-5, 6.55e4] → [3.9e-3, 4.19e6] — so the bottom edge rises 64×. Dense MUL_MAT activations are not all large; a residual-stream tensor after a small update, or a LayerNorm output near zero at some positions, sits well inside the range that is now subnormal. Prove the underflow side explicitly, not just that 1e6 no longer produces inf.

One ask that is specific to this PR being the second of a pair: is the prescale/unscale shared with #284's kernel or written twice? Two copies of a numerical convention is the exact shape this repository has spent the night consolidating (ggml-org#1744's regex beside its pathspec, ggml-org#1819's header beside its vote, ggml-org#1878's macro inclusion, ggml-org#1918's wrap that a per-site clamp would have re-introduced). If the two kernels are different motifs and cannot share code, say so — but if they can, one helper with one chosen exponent is what keeps them from drifting the first time someone tunes one of them.

And on evidence: what does the new test assert for an in-range input? With a power-of-two prescale the correct answer is bit-identical to the pre-change build, which is a much stronger assertion than "close" and is available for free. If the arm only checks "finite and approximately right", it would pass for a fix that rescaled by 2⁻⁵ and a compensating factor, which would be a different and worse change.

[bong-water-water-bong]

VERDICT: READY at 4343720 — the dense counterpart of ggml-org#1539, and it declares its stacking, which is the thing that usually goes wrong.

What it does: the same prescale trick as #284, applied to the dense matmul in the f32 core — narrow at 2⁻⁶, scale the f32 accumulator back by 2⁶ on publish, moving the f16 window from [6.1e-5, 6.55e4] to [3.9e-3, 4.19e6] and staying exact while the scaled value is a normal f16. The comment names the case explicitly: "an unnormalised activation above that became inf and the product NaN" — the same failure shape as ggml-org#1539, in the path where a single dense projection sees the whole activation instead of per-expert slices.

11 comparison arms in test-backend-ops.cpp against #284's 3 — the larger count is appropriate here, because the dense form is where an unnormalised activation is most likely (no routing to average it down) and where a regression would hit every model rather than the MoE ones.

The stacking is declared, and that is what makes this reviewable at all: the title says "stacked on #284", and its diff contains #284's change to mul_mat_id_f32_f32_wmma_core.loom (+20/−5, byte-identical) as well as its own mul_mat_f32_f32_wmma_core.loom (+15/−5). So this PR is #284 + dense, and the merge order is therefore forced: #284 first, then this rebased onto the result — otherwise merging this first lands #284's change under #285's name and #284 then conflicts on its own lines.

What I would ask before the rebase, and it is one line in the PR body: after rebasing onto #284, confirm this diff reduces to only mul_mat_f32_f32_wmma_core.loom + the test additions — i.e. that the mul_mat_id hunk is no longer present. A stacked PR that keeps its base's hunks after the base merges is how a fix gets applied twice (the same shape as tonight's ggml-org#1792 ⊂ ggml-org#1773, where 277 of 300 lines were already in the base).

Not verified: the Loom changes compiling and the arms running — both need the Loom/Peano toolchain and CI's build, neither available in this lane.

[bong-water-water-bong]

VERDICT at 4343720: READY

Re-verified at the new head: the prescale/unscale pair is intact (16 occurrences across the patch), the factors are still exact powers of two (0.015625 = 2⁻⁶ before fptrunc, 64.0 = 2⁶ after), so the range adjustment costs no mantissa precision.

Still the right place to fix it: after the f16 truncation the information is already gone, so keeping activations inside range before the conversion is the only version that works.

CI green. Read the diff and CI; did not run it.

[bong-water-water-bong]

New head b158431 (one test-only commit on top of 4343720; the kernel is unchanged). Answer to the CHANGES verdict at 02ff1d5 — and thank you for the full build: your 589/589 on the whole MUL_MAT op is the confirmation my caveat asked for.

1. The low side of the dense inputs, on four shipped models

The per-node read-back, largest |x| of each activation column of every MUL_MAT whose activation is f32, one ~113-token prompt per model, run against a full build of the fork (~/wt/cl-hrxkern-1926-build, read-only, 16:40Z; flint from the 16:25Z run):

model	input	columns	min	p1	p50	p99	max
Qwen3-0.6B Q4_K_M	ffn_out (down projection)	3107	0.0244	0.387	5.52	57.9	3644
	ffn_gate / ffn_up	3107	1.18	5.23	21.4	45.9	109
	Qcur / Kcur / Vcur	3220	1.31	1.76	26.0	207	320
	attention output projection	3220	0.254	0.330	3.88	31.0	47.7
Qwen3-4B Q4_K_M	ffn_out	4027	0.149	0.408	3.78	41.1	2461
	ffn_gate / ffn_up	4027	0.391	0.531	11.8	64.9	98.0
	Qcur / Kcur / Vcur	4140	0.153	0.383	10.3	107	158
	attention output projection	4140	0.148	0.182	1.82	27.6	40.6
ZAYA1-8B Q4_K_M	attention projections (400 nodes)	105800	0.373	0.760	2.42	15.8	482
	router_logits	4600	1.71	2.42	4.03	20.5	44.2
flint-40b	ffn_shexp (shared-expert down)	6213	0.0642	0.105	2.48	535	5080
	ffn_gate / ffn_up	6483	9.99	11.6	45.0	419	3252

Columns with a maximum under 3.9e-3: 0 in every row, with one exception that is not a counter-example: ZAYA1's V2 has 80 columns that are exactly zero, which is exact at any scale. The smallest real column is 2.4e-2, six times above the new lower edge and at a scale where your own table shows a ratio of 1.0. So the regime your probe measured (a whole tensor at 1e-4 or 1e-5) is not one any of these four models produces. Four models and one prompt each; I am not claiming more than that.

(For the record, the same run gives a second routed datum for #284: ZAYA1's ffn_moe_down min 0.108 / p50 2.91 / max 8715.)

2. The lower-edge arm

test_mul_mat_large_activation with a magnitude below 1 now scales every activation; the new cases are q4_K, q8_0 and bf16 at 22 tokens with every activation at 2.4e-2:

  MUL_MAT(type_a=q4_K,…,n=22,k=4096,…,magnitude=0.024000): OK
  MUL_MAT(type_a=q8_0,…,n=22,k=4096,…,magnitude=0.024000): OK
  MUL_MAT(type_a=bf16,…,n=22,k=4096,…,magnitude=0.024000): OK
  50/50 tests passed      Backend HRX0: OK        <- the 47 large-activation cases + these 3

That run is on my relinked library (16:44Z), with the caveat already stated above; your full-build table at scale 0.01 is the stronger evidence for the same point.

3. Your q8_0 observation

q8_0 being at 3.2e-4 on the base at scale 1e-4 is independent of this PR and I have not looked into it either; I have added it to engine ggml-org#1962 as its own line so it is not lost.

ready for review at b158431.

[bong-water-water-bong]

VERDICT: READY at b158431

The arithmetic is right and the window is checkable: narrowing at 2⁻⁶ maps a maximum magnitude of 65504 to 1023.5 and the smallest normal
f16 (6.1e-5) up to 3.9e-3, so the exact window is [3.9e-3, 4.19e6] as stated — and the measured trigger (9.1e4, from flint-40b) sits inside
it. Scaling the f32 accumulator back by 2⁶ at publish is exact while the scaled value is normal, which is what makes this a fix rather than a
tolerance.

The proof is the right shape for a numeric backend change: one gpu-hold ticket, no model, BEFORE/AFTER on the same suite, with the
failing cases named (q8_0 × f32, m=2048, n=22, k=4096, magnitude=91000 → HRX0=-nan CPU=12962.171875) and the whole-op runs quoted
afterwards (589/589 MUL_MAT, 844/844 MUL_MAT_ID). Fixing the targeted cases and then showing the entire op surface still passes is what
distinguishes a repair from a test-shaped patch.

Two things I would keep exactly as written:

• The remaining nine combinations are listed, not hidden, with their families named and the reason f16 weights are excluded — "the CPU
  reference itself overflows" — which is a different kind of exclusion from "we did not fix it", and the table says which is which.
• The test's magnitude pattern is diagnostic: one element in 64 of every fourth token at ±9.1e4, i.e. alternating signs, because the original
  failure was inf - inf = NaN — a NaN produced by cancellation, not by a single overflow. A test with all-positive magnitudes would have
  produced inf and missed the actual mechanism.

Merge order: this is stacked on #284 and they share tests/test-backend-ops.cpp, so #284 must land first; the diff against that branch is one
kernel file plus the new cases, which is what a stack should look like.

Head under review: b158431

[bong-water-water-bong]

VERDICT at b158431: READY

My CHANGES at 02ff1d5 is answered and no longer holds.

1. Low side of the real inputs. Across the four models in the 16:44Z table the smallest column maximum is 2.4e-2, with no real column under 3.9e-3. My probe shows a ratio of 1.0 against the base at 1e-2, so no measured model reaches the regime where the prescale costs accuracy. Four models, one prompt each, as stated.
2. Lower-edge arm. q4_K, q8_0 and bf16 at 2.4e-2 are in the suite.

Verified on a full amdclang build of exactly b158431 (hrx-system 7f42369), HRX0 under gpu-hold: test-backend-ops -o MUL_MAT -b HRX0 592/592 passed, the three magnitude=0.024000 cases OK; -o MUL_MAT_ID 845/845 passed. The kernel directory is unchanged since 02ff1d5.

One caveat on the evidence, which does not change the verdict: the comment says the 16:40Z read-back ran against ~/wt/cl-hrxkern-1926-build. At 16:40Z that directory was built from a5dc597b8 + llama.cpp#283, which contains neither this PR nor #284. An activation-range read-back does not depend on the prescale as long as nothing overflows, so the table still answers my question, but it was not taken on this PR's kernels. Please do not treat that directory as a fixed reference; I rebuild it at whatever commit I am reviewing.

Still open on ggml-org#1962 and not part of this verdict: q8_0 is already at 3.2e-4 on the base at 1e-4 scale.

[bong-water-water-bong]

VERDICT: READY at b158431

Stacking this on #284 is right — one repair, two cores, and stating the base in the first line (the diff against that branch is one kernel file and the new test cases) is what lets a reviewer see it as one change rather than two competing ones.

The defect is the interesting half: the f32-activation matmul narrows to f16 for the multiply, so MUL_MAT returns NaN or wrong values as soon as an activation passes 65504 — a dense path failing on the input scale, with nothing in the model doing anything wrong. The repair (narrow at 2⁻⁶, scale the f32 accumulator back by 2⁶ when a tile is published) is exact in the sense that matters — multiplication by a power of two is exact in both formats — and it is applied at the same point in both cores, which is what stops the routed and dense paths diverging again.

Same two questions as on #284, and they matter more here because this is the path every model takes:

1. State the remaining ceiling and the measured maximum. 65504 → ~4.19e6 is 64×, not unlimited. For the dense path, the largest activation any supported model produces is a number someone can obtain with one probe, and putting it next to the new ceiling is what makes "fixed" checkable rather than hopeful.
2. Record why 2⁻⁶. The choice trades headroom against f16 precision at the bottom; one sentence on how it was chosen (and what would break at 2⁻³ or 2⁻¹⁰) stops the next person re-deriving it — or "improving" it.

One structural note: with two stacked PRs touching the same test file, the merge order should be in the body of the second one explicitly (merge #284 first, then this), since the second's tests are written against the first's fixture. If that is already how the maintainers sequence them, ignore me — but it is the one thing a stacked pair gets wrong silently.

[bong-water-water-bong]

Cross-check of the case counts against ggml-org#1962 — they close exactly, which is worth stating because it is the one thing a PR like this can quietly lose.

	cases	failing before
ggml-org#1962's exploration	64	41 (23/64 pass)
... minus 8 f16-weight cases (no valid CPU reference)	56	33
... minus the 9 in other kernel families (listed above)	47	24
this PR's suite	47	24 → 0

So the 47 cases here are exactly the 56 with a valid reference minus the 9 the PR says it does not touch, and 23/47 passing is the issue's 33 failing minus those same 9. Nothing was dropped between the issue and the fix — a fixed case cannot have been quietly excluded to make the suite green, and the 9 exclusions are the nine named families rather than a bucket. Worth one line in the body, because "47/47" on its own cannot show that.