[bong-water-water-bong]

Refs 1bit-MONSTER/engine#2137 (the step after #308 and #310 on its "done when" list: the default). Reaches engine main with a later pin bump, not in this PR.

What changes

#308 and #310 let prefill read a Q4_K / Q6_K weight in the packed rows decode reads, behind GGML_HRX_Q4K_PACKED_TAIL=1 and GGML_HRX_Q6K_PACKED_ROWS=1. Both say why they stayed opt-in: the packed layout saves a device copy only when the stored one costs a copy, i.e. when the weight is in the loader's file mapping (#304); for an uploaded weight the stored layout is already on the device and the packed one is the extra copy.

That condition is a property of each weight's buffer, so the default now reads it there:

• a weight whose buffer is the file mapping: packed, at every batch size;
• any other weight (GGML_HRX_MODEL_BUFFER_COPY=1, a host-weight limit, a load without mmap, a test tensor): stored, as before.

=1 still forces packed for every weight and =0 forces stored for every weight. The once-per-process log line says which of the three is in effect.

buffer_is_file_mapped() (new, in model-buffer-from-mmap) identifies the mapped buffer type by its interface, not by name.

This also composes with #311 (open, engine ggml-org#2163), which would upload models above a size limit: those weights are not file-mapped, so they stay stored without either PR knowing about the other. I have not run the two together.

Test (device-free)

hrx-backend-test packed_layout_follows_weight.{q4_k,q6_k}.{file_mapped,uploaded}: one MUL_MAT at 100 tokens (off both aligned grids), the weight allocated either in a buffer made by model_buffer_from_mmap() or in a plain weights buffer; the scheduled ggml.mul_mat.weight_format must be 44 / 46 for the file-mapped weight and 4 / 6 for the uploaded one.

[PASS] packed_layout_follows_weight.q4_k.file_mapped
[PASS] packed_layout_follows_weight.q4_k.uploaded
[PASS] packed_layout_follows_weight.q6_k.file_mapped
[PASS] packed_layout_follows_weight.q6_k.uploaded
host: 4 passed, 0 failed

(strixhalo, amdclang Release, fork 2b7960b + this change, run host-only.) Before this change the two file_mapped rows cannot pass: the default was stored for every weight. I did not run them against the old code, because the rows use the new buffer_is_file_mapped().

Not measured yet

The device before/after is queued, not done: Qwen3-4B Q4_K_M, device copies after a 1,700-token prompt with a reply, default against =0, with perplexity at a 300-token batch and prefill tok/s in both arms. I will post it here when the ticket has run; please do not treat this as verified on a device until then. cl-hrxkern's numbers on #308/#310 are for the switches forced on, which is the same kernel path this default selects for a file-mapped model.

Also not done: a dense 27B row and a routed-expert row (the box has not had the memory), and the whole hrx-backend-test / test-backend-ops suites at this change (in test-backend-ops no weight is file-mapped, so the default there is unchanged by construction).

🤖 Generated with Claude Code

[bong-water-water-bong]

VERDICT at e02f3ba: READY

This is the last step on ggml-org#2137's "done when" list — the default — and the ordering of the series is what makes it safe: #308 (Q4_K opt-in), #310 (Q6_K opt-in, which also fixed a fault in the first opt-in), and now the default path reading a Q4_K/Q6_K weight from the file mapping.

Two things are right about finishing here rather than earlier:

• the opt-in path ran twice with its own flag before anything became default, so the second opt-in (#310) had a chance to find the first one's fault before it was on by default;
• each step was validated on a real model's shapes (Qwen3-4B's 216 q4_K / 37 q6_K) rather than on a synthetic pair.

Reaches engine main with a later pin bump, which is the chain ggml-org#2141's submodule-pin checker guards — so the default cannot land here and sit unreached in the fork.

[bong-water-water-bong]

The device before/after the body said was queued (cl-hrxmem). Head e02f3ba, full build of exactly that commit, Qwen3-4B Q4_K_M, weights in the file mapping, one gpu-hold ticket, 12:5xZ. Default = both switches unset; stored = both =0.

	stored (=0)	default (this PR)
distinct device copies after -p 1700 -n 16	647, 4,400.6 MiB	398, 2,375.9 MiB
weights held in two layouts	249 (214 q4_K + 35 q6_K)	0
distinct device copies after -p 100	398, 2,375.9 MiB	398, 2,375.9 MiB
perplexity, -c 300 -b 300, 12 chunks	19.2890 ± 1.66255	19.2890 ± 1.66255
llama-bench -p 300 -r 3	1097.62 ± 21.45 tok/s	1229.17 ± 56.43 tok/s
llama-bench -p 1700 -r 3	1859.66 ± 300.72 tok/s	2040.45 ± 79.98 tok/s

The model is 2,375.9 MiB, so the default holds exactly one device copy per weight after a long prompt with a reply; the stored arm reproduces the issue's own figure (647 copies, 4,400.6 MiB). Copies are counted from GGML_HRX_HOST_WEIGHT_TRACE=2 as distinct (weight, layout) pairs. The raw trace has more lines than that (796 and 1,010) because llama-bench makes the copies again for its second test; I counted pairs, not lines. The log line confirms the arm in each run ("PACKED layout for a weight in the file mapping, stored otherwise" / "stored layout for every weight").

Prefill is not slower with the default. The two arms ran back to back in one ticket, not interleaved, and the -p 1700 stored figure has a wide spread, so read the tok/s rows as "no regression", not as a measured gain.

Still not done, as in the body: a dense 27B row, a routed-expert row, and the whole suites at this head.

ready for review at e02f3ba.