From 24391a62577ec7465f6820db9980a9f6a769a638 Mon Sep 17 00:00:00 2001 From: bong-water-water-bong Date: Tue, 10 Nov 2026 11:49:49 -0400 Subject: [PATCH] bump llama.cpp to a5dc597b8: one-token decode at any head size, and a clang build has its HRX0 again (fixes #1926, #1942) Four fork commits past 2d62cc7af: - 7fb31383b (#282) the fused Q decode kernel compiles at every head size, not only a power of two (#1926) - c47eae4a7 (#279) the host term is read before it is classified, so a clang build registers HRX0 (#1942) - 69ee6f50e (#281) IQ3_XXS routed experts take the workgroup-table decode by default (#572) - a5dc597b8 (#278, carries #277) a pool refusal at compute time is ALLOC_FAILED, with device-free tally arms (#1618, #1823) Measured at a5dc597b8 on gfx1151 (amdclang build, gpu-hold): --list-devices prints HRX0; test-hrx-attention-q-decode-head-size 15 cases, 0 failures; Sarashina2.2-3B bf16 tg128 26.49 / 25.92 tok/s, stories15M Q8_0 tg128 317.77 / 338.54 tok/s. registry/architectures.json: the recorded source pin only. docs/hrx.md: the #1926 fix. Co-Authored-By: Claude Fable 5.2 --- docs/hrx.md | 17 +++++++++++++++++ registry/architectures.json | 2 +- third_party/llama.cpp | 2 +- 3 files changed, 19 insertions(+), 2 deletions(-) diff --git a/docs/hrx.md b/docs/hrx.md index adddded4e..9879aed47 100644 --- a/docs/hrx.md +++ b/docs/hrx.md @@ -246,6 +246,23 @@ decode-path logits (`llama-perplexity -c 2049 -b 1 --save-all-logits`) are byte- to the previous pin, the buried code word is exact at 4700 tokens, and no GPU faults were observed. +### Fixed: one-token decode at a head size that is not a power of two ([engine#1926](https://github.com/1bit-MONSTER/engine/issues/1926)) + +A model whose attention head size is not a power of two (stories15M: 48; Sarashina2.2-3B: 160) and whose +Q weights are not Q4_K or Q6_K loaded and prefilled on `HRX0`, then failed at the first decode step: +`HRX Loom JIT Loom compilation failed`, `llama_decode: failed to decode, ret = -3`. The one-token Q +projection kernel wrote each rotated pair at `channel rem head_size`, and the kernel compiler can only +prove that two-wide store in bounds when the remainder is a mask. The kernel now indexes by rotation +pair, which is in bounds at every head size (llama.cpp #282). Q4_K and Q6_K models took a different +kernel at one token and were not affected. + +Measured at pin `a5dc597b8` on gfx1151 (2026-11-10, power mode `performance`): +`test-hrx-attention-q-decode-head-size` compares the fused projection with the CPU backend at head +sizes 48, 96 and 160 (controls 64 and 128) on BF16, F16 and Q8_0 weights: 9 of 15 cases failed before +the fix, 0 of 15 after. `llama-bench -p 512 -n 128 -r 3`, two runs each: Sarashina2.2-3B bf16 decodes +at 26.49 and 25.92 tok/s, stories15M Q8_0 at 317.77 and 338.54 tok/s (the box was shared; the same +binary gave 545.39 tok/s on a quieter run). + ### MoE decode and the kernel cache (llama.cpp #32, #33) - **MoE decode.** With one to four tokens, a `MUL_MAT_ID` (a MoE expert matmul) runs as a GEMV over diff --git a/registry/architectures.json b/registry/architectures.json index b3bf4f23a..b2fb3f4e9 100644 --- a/registry/architectures.json +++ b/registry/architectures.json @@ -1,7 +1,7 @@ { "about": "HF architecture -> GGUF architecture and the backends whose code accepts it. Generated by tools/registry_build.py from the pinned sources; do not edit.", "sources": { - "llama.cpp (hrx)": "2d62cc7afdae961bce83abcdb54dd95a17db7b50" + "llama.cpp (hrx)": "a5dc597b81f10c1fd29356018e56da074380ce60" }, "counts": { "hrx": 298, diff --git a/third_party/llama.cpp b/third_party/llama.cpp index 2d62cc7af..a5dc597b8 160000 --- a/third_party/llama.cpp +++ b/third_party/llama.cpp @@ -1 +1 @@ -Subproject commit 2d62cc7afdae961bce83abcdb54dd95a17db7b50 +Subproject commit a5dc597b81f10c1fd29356018e56da074380ce60