bump llama.cpp to 4107043c1: a weight in the file mapping keeps one device copy after a long or a short prompt, and a dense 27B shares most of its copies (engine #2137)

docs/hrx.md

@@ -945,8 +945,8 @@ one line per device copy, since pin `2b7960bae`), the extra copies have two caus
 A prompt made only of 512-token batches, a short prompt, or decode alone holds one copy per weight.
 
 `GGML_HRX_Q4K_PACKED_TAIL=1` (llama.cpp #308) makes the generic kernels read Q4_K in the packed layout, and
-`GGML_HRX_Q6K_PACKED_ROWS=1` (llama.cpp #310) makes Q6_K prefill read the packed rows decode reads, at every batch size. Both are
-opt-in; the stored path is the default. Measured on Qwen3-4B Q4_K_M with both on, against both off (one binary, fork `ccf14f9c9`):
+`GGML_HRX_Q6K_PACKED_ROWS=1` (llama.cpp #310) makes Q6_K prefill read the packed rows decode reads, at every batch size. Measured on
+Qwen3-4B Q4_K_M with both on, against both off (one binary, fork `ccf14f9c9`):
 
 | | off | on |
 |---|---:|---:|
@@ -958,9 +958,70 @@ opt-in; the stored path is the default. Measured on Qwen3-4B Q4_K_M with both on
 | `llama-bench -p 1700` | 2,211.7, 1,901.1 tok/s | 2,319.6, 2,307.0 tok/s |
 | `test-backend-ops -o MUL_MAT` on `HRX0` | 641 of 641 | 641 of 641 |
 
-They are not the default yet because that is one model: there is no dense 27B row and no routed-expert row, the 6-to-16-token Q6_K
-kernel reads a third layout that neither switch covers, and the switches only save memory with the file mapping. With
-`GGML_HRX_MODEL_BUFFER_COPY=1` the stored layout is the loader's buffer and costs nothing extra, so they add packed copies there.
+**Since pin `caa2d4dde` the packed layout is the default for a weight in the file mapping** (llama.cpp #313). The switches only save
+memory where the stored layout costs a copy, and that is a property of each weight's buffer. So with neither switch set, a Q4_K or
+Q6_K weight in the file mapping is read in the packed rows, and a weight that was uploaded (`GGML_HRX_MODEL_BUFFER_COPY=1`, a
+host-weight limit, `--no-mmap`) keeps the stored layout, which is already on the device there. The switches now force the choice for
+every weight: `1` packed, `0` stored. Measured at `caa2d4dde`'s change on Qwen3-4B Q4_K_M, default against both switches at `0`,
+counting distinct (weight, layout) copies:
+
+| after | stored (`=0`) | default |
+|---|---:|---:|
+| a 1,700-token prompt and a 16-token reply | 647 copies, 4,400.6 MiB (249 weights in two layouts) | **398 copies, 2,375.9 MiB (none)** |
+| a 100-token prompt | 398 copies, 2,375.9 MiB | 398 copies, 2,375.9 MiB |
+
+Perplexity at a 300-token batch is 19.2890 in both arms. Prefill is not slower: 1,229 against 1,098 tok/s at 300 tokens and 2,040
+against 1,860 at 1,700 (back to back, not interleaved).
+
+**Since pin `0494dfa28` a short prompt and its reply share one copy as well** (llama.cpp #316). The Q6_K kernel for batches of 6 to
+16 tokens read the weight in an i8 layout of its own, so after a 12-token prompt and a reply 35 Q6_K weights were held twice:
+433 device copies, 2,856.2 MiB. For a weight whose layout is the packed rows, that batch size now goes to the tiled kernel, which
+reads the packed rows: 398 copies, 2,375.9 MiB, the model's own size. With that, Qwen3-4B Q4_K_M holds one device copy per weight
+at every prompt shape measured (3, 8, 12, 40, 100 and 1,700 tokens, with and without a reply). It costs prefill speed at those
+sizes: 8.9 % at 8 tokens, 5.3 % at 12 and 6.9 % at 16 (interleaved, quiet box), which is 2 to 3 ms on a short prompt. Perplexity
+at a 12-token batch is 59.0414 before and 59.0417 after. An uploaded weight keeps the i8 kernel.
+
+**Speed against the previous behaviour.** At pin `4107043c1` (clean build, `build_commit 4107043`), the defaults against all
+three switches off (`GGML_HRX_Q4K_PACKED_TAIL=0 GGML_HRX_Q6K_PACKED_ROWS=0 GGML_HRX_PREFILL_FOLLOWS_DECODE=0`, the previous
+behaviour), order default / off / off / default in one ticket, box load 4 to 7:
+
+| tok/s | all three off | defaults |
+|---|---|---|
+| Qwen3.8-27B, `llama-bench -p 1700 -n 48 -r 2`, prefill | 251.1, 311.9 | 321.3, 308.4 |
+| Qwen3.8-27B, the same, decode | 10.66, 7.37 | 9.29 ± 3.24, 10.84 |
+| Qwen3.8-27B, `llama-completion -c 4096 -n 48`, 1,282-word prompt: prompt | 248.6, 296.5 | 263.2, 254.6 |
+| Qwen3.8-27B, the same: decode | 10.54, 6.20 | 10.13, 7.57 |
+| Qwen3-4B Q4_K_M, `llama-bench`, prefill | 1,540.5, 1,725.0 | 1,711.1, 2,299.9 |
+| Qwen3-4B Q4_K_M, `llama-bench`, decode | 43.96, 72.58 | 45.10, 73.31 |
+| Qwen3-4B Q4_K_M, `llama-completion`: prompt | 1,473.2, 1,692.1 | 1,664.9, 2,139.5 |
+| Qwen3-4B Q4_K_M, `llama-completion`: decode | 49.32, 49.62 | 40.41, 49.99 |
+
+The generated text is identical between the arms on each model. The two runs of one arm differ by up to 40 % here (27B decode
+7.4 against 10.7 with the switches off), more than any difference between the arms, so this table shows no regression and does
+not measure a gain. An earlier decode-only window on the 27B (`-p 0 -n 48 -r 3`, eight alternated arms, before #320) was steadier:
+11.21 to 11.91 tok/s with both packed-layout switches at `0` and 11.91 to 11.92 with the default. `llama-bench` runs its prompt
+test before any decode in the process, so its prefill row does not include #320's effect, which needs a stored copy made by
+decode first; the `llama-completion` rows do (they decode a warm-up token first).
+`GGML_HRX_Q4K_PACKED_TAIL=0 GGML_HRX_Q6K_PACKED_ROWS=0 GGML_HRX_PREFILL_FOLLOWS_DECODE=0` restores the previous behaviour.
+
+**A dense mixed-quant 27B shares most of its copies too since pin `4107043c1`** (llama.cpp #320). On Qwen3.8-27B UD-Q4_K_XL the
+one-token decode goes through the kquant kernels, which read the stored (GGUF) layout, while some prefill kernels read the packed
+rows, so after a long prompt 70 weights were held in both layouts (2,486.7 MiB of duplicate). HRX now records each stored copy
+it makes, and prefill keeps the stored layout for a weight that already has one: the Q6_K prefill kernel through its stored form,
+and the 256-aligned Q4_K prefill through the 8-bit-block kernel, which reads Q4_K stored as well. Measured with
+`llama-completion -c 4096 -n 32` and a 1,282-word prompt, `GGML_HRX_PREFILL_FOLLOWS_DECODE=0` against unset:
+
+| Qwen3.8-27B UD-Q4_K_XL | `=0` | default |
+|---|---:|---:|
+| device copies | 920, 18,205.2 MiB | 866, 16,181.4 MiB |
+| weights in two layouts | 70 (2,486.7 MiB duplicate) | 16 (462.9 MiB) |
+
+The generated text is identical, and prefill was not slower in the one interleaved window measured (prompt eval 263.5 and 317.1
+tok/s against 298.7 and 317.1). Qwen3-4B is unchanged (398 copies). What is left on the 27B, 15 Q4_K weights read by a prefill
+kernel that has only a packed form and one Q6_K weight, is tracked in
+[engine#2137](https://github.com/1bit-MONSTER/engine/issues/2137). Making decode step aside instead was measured and does not help:
+the same duplicates, and decode 8.5 % slower. flint UD-Q4_K_XL (routed experts) reads every weight stored and holds one copy per
+weight with or without the default.
 
 `GGML_HRX_Q6K_PACKED_PREFILL=1`, the older switch, never saved a copy: its dispatch bound the stored layout. Off its kernel's wide
 tile (tokens not a multiple of 256, or outputs not a multiple of 128) it also read past the weight: a Q6_K matmul at 128 tokens with

registry/architectures.json

@@ -1,7 +1,7 @@
 {
  "about": "HF architecture -> GGUF architecture and the backends whose code accepts it. Generated by tools/registry_build.py from the pinned sources; do not edit.",
  "sources": {
-  "llama.cpp (hrx)": "2b7960bae8d949ed77e3b6c6f1907932e80c198f"
+  "llama.cpp (hrx)": "4107043c16d18a9d7fb890cd1681850f9f8255d4"
  },
  "counts": {
   "hrx": 298,