diff --git a/docs/hrx.md b/docs/hrx.md index e4d661a58..f6c7c3313 100644 --- a/docs/hrx.md +++ b/docs/hrx.md @@ -252,6 +252,23 @@ decode-path logits (`llama-perplexity -c 2049 -b 1 --save-all-logits`) are byte- to the previous pin, the buried code word is exact at 4700 tokens, and no GPU faults were observed. +### Fixed: one-token decode at a head size that is not a power of two ([engine#1926](https://github.com/1bit-MONSTER/engine/issues/1926)) + +A model whose attention head size is not a power of two (stories15M: 48; Sarashina2.2-3B: 160) and whose +Q weights are not Q4_K or Q6_K loaded and prefilled on `HRX0`, then failed at the first decode step: +`HRX Loom JIT Loom compilation failed`, `llama_decode: failed to decode, ret = -3`. The one-token Q +projection kernel wrote each rotated pair at `channel rem head_size`, and the kernel compiler can only +prove that two-wide store in bounds when the remainder is a mask. The kernel now indexes by rotation +pair, which is in bounds at every head size (llama.cpp #282). Q4_K and Q6_K models took a different +kernel at one token and were not affected. + +Measured at pin `a5dc597b8` on gfx1151 (2026-11-10, power mode `performance`): +`test-hrx-attention-q-decode-head-size` compares the fused projection with the CPU backend at head +sizes 48, 96 and 160 (controls 64 and 128) on BF16, F16 and Q8_0 weights: 9 of 15 cases failed before +the fix, 0 of 15 after. `llama-bench -p 512 -n 128 -r 3`, two runs each: Sarashina2.2-3B bf16 decodes +at 26.49 and 25.92 tok/s, stories15M Q8_0 at 317.77 and 338.54 tok/s (the box was shared; the same +binary gave 545.39 tok/s on a quieter run). + ### MoE decode and the kernel cache (llama.cpp #32, #33) - **MoE decode.** With one to four tokens, a `MUL_MAT_ID` (a MoE expert matmul) runs as a GEMV over diff --git a/registry/architectures.json b/registry/architectures.json index b3bf4f23a..b2fb3f4e9 100644 --- a/registry/architectures.json +++ b/registry/architectures.json @@ -1,7 +1,7 @@ { "about": "HF architecture -> GGUF architecture and the backends whose code accepts it. Generated by tools/registry_build.py from the pinned sources; do not edit.", "sources": { - "llama.cpp (hrx)": "2d62cc7afdae961bce83abcdb54dd95a17db7b50" + "llama.cpp (hrx)": "a5dc597b81f10c1fd29356018e56da074380ce60" }, "counts": { "hrx": 298, diff --git a/third_party/llama.cpp b/third_party/llama.cpp index 2d62cc7af..a5dc597b8 160000 --- a/third_party/llama.cpp +++ b/third_party/llama.cpp @@ -1 +1 @@ -Subproject commit 2d62cc7afdae961bce83abcdb54dd95a17db7b50 +Subproject commit a5dc597b81f10c1fd29356018e56da074380ce60