Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions docs/hrx.md
Original file line number Diff line number Diff line change
Expand Up @@ -252,6 +252,23 @@ decode-path logits (`llama-perplexity -c 2049 -b 1 --save-all-logits`) are byte-
to the previous pin, the buried code word is exact at 4700 tokens, and no GPU faults
were observed.

### Fixed: one-token decode at a head size that is not a power of two ([engine#1926](https://github.com/1bit-MONSTER/engine/issues/1926))

A model whose attention head size is not a power of two (stories15M: 48; Sarashina2.2-3B: 160) and whose
Q weights are not Q4_K or Q6_K loaded and prefilled on `HRX0`, then failed at the first decode step:
`HRX Loom JIT Loom compilation failed`, `llama_decode: failed to decode, ret = -3`. The one-token Q
projection kernel wrote each rotated pair at `channel rem head_size`, and the kernel compiler can only
prove that two-wide store in bounds when the remainder is a mask. The kernel now indexes by rotation
pair, which is in bounds at every head size (llama.cpp #282). Q4_K and Q6_K models took a different
kernel at one token and were not affected.

Measured at pin `a5dc597b8` on gfx1151 (2026-11-10, power mode `performance`):
`test-hrx-attention-q-decode-head-size` compares the fused projection with the CPU backend at head
sizes 48, 96 and 160 (controls 64 and 128) on BF16, F16 and Q8_0 weights: 9 of 15 cases failed before
the fix, 0 of 15 after. `llama-bench -p 512 -n 128 -r 3`, two runs each: Sarashina2.2-3B bf16 decodes
at 26.49 and 25.92 tok/s, stories15M Q8_0 at 317.77 and 338.54 tok/s (the box was shared; the same
binary gave 545.39 tok/s on a quieter run).

### MoE decode and the kernel cache (llama.cpp #32, #33)

- **MoE decode.** With one to four tokens, a `MUL_MAT_ID` (a MoE expert matmul) runs as a GEMV over
Expand Down
2 changes: 1 addition & 1 deletion registry/architectures.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"about": "HF architecture -> GGUF architecture and the backends whose code accepts it. Generated by tools/registry_build.py from the pinned sources; do not edit.",
"sources": {
"llama.cpp (hrx)": "2d62cc7afdae961bce83abcdb54dd95a17db7b50"
"llama.cpp (hrx)": "a5dc597b81f10c1fd29356018e56da074380ce60"
},
"counts": {
"hrx": 298,
Expand Down
Loading