From 7b419e2d83a38879f46d98aa50e08a97d32f9e57 Mon Sep 17 00:00:00 2001 From: bong-water-water-bong Date: Wed, 11 Nov 2026 02:45:36 -0400 Subject: [PATCH] bump llama.cpp to 1703bfeaa: a gated-delta-net layer behind a type cast no longer fails the graph (fixes #1925) One fork commit past 7a1f2b81c: - 1703bfeaa (#302): the scheduler's dependency walk no longer counts an in-place CPY's own output as a pending input (a ggml_cast stalled the walk and everything after it was taken in index order, where the L2_NORM root of the fused gated delta net comes before the projections it reads); the fused matcher keeps a match whose layout alias was already passed. Measured on gfx1151 with the two changed objects replaced in an add-on build: Flash-Next UD-IQ3_XXS 18,782-token request, unsupported HRX node 1 -> 0, 390 of 390 fused roots accepted, the whole prefill runs. Qwen3.5-4B old vs new library: 128 of 128 tokens identical in three pairs, the same 33 dispatch registrations, decode 57.4/58.1 vs 58.3/58.2 tok/s. Host: test-hrx-gdn-passed-alias 7 cases, hrx-backend-test 66/66. Not shown: the 18.8k answer with the mask chain on the device against the CPU; those requests end in #1536's all-NaN logits on the old and the new library alike. registry/architectures.json: the recorded source pin only. docs/hrx.md: the #1925 section. Co-Authored-By: Claude Fable 5.2 --- docs/hrx.md | 32 ++++++++++++++++++++++++++++++++ registry/architectures.json | 2 +- third_party/llama.cpp | 2 +- 3 files changed, 34 insertions(+), 2 deletions(-) diff --git a/docs/hrx.md b/docs/hrx.md index 2faf2db4d..5f1a2232f 100644 --- a/docs/hrx.md +++ b/docs/hrx.md @@ -389,6 +389,38 @@ layout (8.80 s and 8.85 s) and gives the same text. `test-hrx-fa-transposed-valu backend (8 cases). Why 256-token launches are faster than one 512-token launch at long key counts is measured, not explained. +### Fixed: a gated-delta-net layer behind a type cast failed the whole graph ([engine#1925](https://github.com/1bit-MONSTER/engine/issues/1925)) + +On Qwen3.8-Flash-Next, a long prompt failed at the first 512-token batch past 2,048 tokens of context when the +sparse attention path's mask chain ran on the device: + +``` +graph_compute: unsupported HRX node 160: L2_NORM output=f32[128,16,512,1]<-L2_NORM inputs=[f32[128,16,512,1]<-VIEW] consumers=[174:GATED_DELTA_NET] +``` + +The fused gated-delta-net dispatch did match that node. The fault was the order in which the graph was walked. +The scheduler walks a graph in dependency order and takes the matmuls first, which is what puts a layer's gate +and beta projections ahead of the `L2_NORM` its fused pattern starts from. A type cast is an in-place copy that +names its own output as an input. The walk counted that as a dependency, so the copy waited for itself, the walk +stopped there, and every node after it was taken in index order. In index order the `L2_NORM` comes before the +projections, and the pattern was refused because it would have read a projection that had not run. The sparse +path's mask chain contains such casts, so the failure needed both a long context and that chain on the device. + +The walk no longer counts a node's own output as a dependency (llama.cpp #302). The fused matcher also keeps a +match when a node of its pattern that only renames storage was already passed. + +Measured on gfx1151 (2026-11-11, Flash-Next UD-IQ3_XXS, `-c 19456 -np 1`, one 18,782-token request, a build of +the fork with only the two changed objects replaced): `unsupported HRX node` 1 before, 0 after; all 390 fused +gated-delta-net roots accepted; the whole prefill runs. `tests/test-hrx-gdn-passed-alias.cpp` plans the layer +behind a cast on the host and requires the same dispatches as without it. Qwen3.5-4B, which has the same layer +type and no sparse path, gives the same tokens on the old and the new library (128 of 128 in three alternated pairs), uses the same 33 +dispatch registrations in the same order, and runs at the same rate (`llama-bench -p 512 -n 64 -r 5`, old, new, +old, new: decode 57.4, 58.3, 58.1, 58.2 tok/s; prefill 3,516, 3,042, 3,109, 3,213). + +Not shown by this: that the answer at 18.8k is the same with the mask chain on the device as with it on the +CPU. Three of the four long requests in these runs, on the old library and the new, ended in all-NaN logits on +the first sampled token, which is [engine#1536](https://github.com/1bit-MONSTER/engine/issues/1536). + ### MoE decode and the kernel cache (llama.cpp #32, #33) - **MoE decode.** With one to four tokens, a `MUL_MAT_ID` (a MoE expert matmul) runs as a GEMV over diff --git a/registry/architectures.json b/registry/architectures.json index cbe8229e0..1d43ec4ea 100644 --- a/registry/architectures.json +++ b/registry/architectures.json @@ -1,7 +1,7 @@ { "about": "HF architecture -> GGUF architecture and the backends whose code accepts it. Generated by tools/registry_build.py from the pinned sources; do not edit.", "sources": { - "llama.cpp (hrx)": "7a1f2b81ce17aaa84a40b766d1b06e5fd4601196" + "llama.cpp (hrx)": "1703bfeaaaae16686e0ea5fd22b18b6fa2163fa1" }, "counts": { "hrx": 298, diff --git a/third_party/llama.cpp b/third_party/llama.cpp index 7a1f2b81c..1703bfeaa 160000 --- a/third_party/llama.cpp +++ b/third_party/llama.cpp @@ -1 +1 @@ -Subproject commit 7a1f2b81ce17aaa84a40b766d1b06e5fd4601196 +Subproject commit 1703bfeaaaae16686e0ea5fd22b18b6fa2163fa1