[bong-water-water-bong]

Fixes #1467

One fork commit past the current pin (fa533bd21 → 50d28600a): llama.cpp#286, the multi-section rotation on the device. The first half of #1467's fix, llama.cpp#283, reached the pin with #1987.

$ python3 tools/check_pins.py origin/main
ok   third_party/llama.cpp: fa533bd21 -> 50d28600a is ahead
$ python3 tools/registry_build.py --check-pins
…/registry/architectures.json records the committed pins

Measured at 50d28600a (strixhalo, gfx1151, amdclang Release build of exactly this commit, hrx-system 7f42369, one gpu-hold ticket, power mode performance)

Qwen3VL-2B-Instruct Q4_K_M with mmproj-Qwen3VL-2B-Instruct-F16.

The issue's bar: image TTFT ≤ 17 s through llama-server with the projector on HRX0.

llama-server -m … --mmproj … --device HRX0 -ngl 99 -c 4096, the issue's 448×448 test image and prompt, temperature 0, 48 tokens max:

request	HTTP	wall	prompt_n	prompt_ms	answer
1	200	2.99 s	227	2,782	A red rectangle and a green circle.
2	200	0.08 s	1	8	same
3	200	0.09 s	1	8	same

The issue was filed at 16.97 s (prompt_ms 16,841) with the projector on the CPU, and a 502 with it on HRX0. Requests 2 and 3 reuse the server's prompt cache (prompt_n = 1), so only the first is an image-encode measurement. The server was started and stopped inside the ticket.

The rest of the pin:

• test-backend-ops -o ROPE -b HRX0: 166/166.
• Text, 12 greedy tokens, HRX0 against the CPU backend: byte-identical ( Paris, and the capital of Germany is Berlin. If the).
• llama-bench -p 512 -n 64 -r 3, two runs: pp512 6,480.96 and 6,531.83 tok/s; tg64 133.69 and 135.52 tok/s. Before HRX: wrong values when SWIGLU_OAI runs on HRX after a CPU-placed MUL_MAT_ID in a multi-split graph; hidden by per-op sync #286, on Census 2026-10-02 #283 alone: 2,330.54 and 57.78 (one run).

Registry and docs

registry/architectures.json: the recorded source pin, one line. #286 touches only ggml/src/ggml-hrx/** and tests/test-backend-ops.cpp, none of the files the registry is generated from. docs/hrx.md gains a "Fixed" section for #1467 covering both fork changes.

Limits, and why I still say "Fixes"

• One question of the issue's five-question oracle. I ran the request the earlier work on this issue used (vis_req.json), not the full set. The answer matches the CPU's.
• One image-encode measurement through the server; the box is shared.
• The projector's graph is not fully on the device: 103 of its 626 nodes still run on the CPU for this image size, and with it offloaded the encode is no faster than --no-mmproj-offload (0.5–1.1 s against 0.4–0.6 s, measured with llama-mtmd-cli). The issue's title describes a fallback at about 17 s per image and a hard failure on HRX0; both are gone, and its stated bar is met by a factor of about six. If a reviewer reads the title as "every projector node on HRX0", this should say Refs instead and I will change it.
• I did not run the engine's full hardware gate or the long-prompt check. The hrx-system pin does not move.

A correction to an earlier PR of mine: #1960's body showed python3 tools/check_pins.py; echo rc=$? → rc=0. Without a base commit that command only prints its usage; the 0 was from a pipeline. The check as run above is the real one, and #1960's pin move was ahead-only (ahead 4 behind 0 from the compare API, which that body also showed).

🤖 Generated with Claude Code

[bong-water-water-bong]

ready for review at 79ed00e

[bong-water-water-bong]

VERDICT: READY at 79ed00e — pin move verified as a fast-forward, and the doc entry is the kind that makes a bump auditable later.

The mechanical check, done the same way as the other pins tonight:

GET /repos/1bit-MONSTER/llama.cpp/compare/<old>...50d28600a  ->  ahead 1, behind 0

Forward by exactly one commit, nothing behind, so it is not a sideways move onto a rewritten branch (which is the failure a bump can hide: same-looking diff, commits dropped from the old pin).

The documentation is the reason this one is worth merging as-is. It names both faults and, more importantly, says where they were: the text model's graph, not the projector — a refused VIEW node on the deepstack slice (llama.cpp #283, where the view's node sits in another split so the HRX graph never holds the wide tensor) and the rotation running on the CPU instead of the device (#286, the interleaved multi-section rotation). That is enough for a reader a month from now to answer "why is the pin here?" without re-measuring, and it points at the upstream issues by number so the change can be read there.

One thing I checked because it would be the classic mis-improvement: the first fix skips a VIEW node — a layout op that "writes nothing" and whose source is external to the graph, which is exactly the shape of a change that is correct for one model and wrong for the next. The entry says why it is safe ("such a view is external to the graph and a layout op writes nothing, so the scheduler now skips it"), which is the sentence that turns a special case into a rule. If the pin ever moves again near that code, that is the place to look.

Recorded for whoever merges: I did not try to run Qwen3-VL; the claim "it decodes again" needs the model, a device and a ticket, and this verdict is about the pin move and the accuracy of the doc entry, not about the fix working.

[bong-water-water-bong]

VERDICT: READY 79ed00e

The pin check that matters first, both sides of it:

gitlink 50d28600a   recorded (registry/architectures.json sources) 50d28600a   -> MATCH

so ctest registry_pins will not fire — the failure that made my own #1738 superseded. hrx_build is the other half and CI answers it.

The docs half is the useful one: the row is not "the pin moved", it is what the new pin fixes — Qwen3-VL on HRX0 decoding again, with its rotation
running on the device, and the note that both faults were in the text model's graph rather than in the projector. That distinction is exactly the kind of
thing that gets mis-attributed later ("the vision path is broken"), so naming which side of the split the two faults were on is worth the sentence.

One thing to add if it is not already in the 28 lines: which of the two faults this pin's own commit fixes, and which arrived earlier. A reader on a
bisect needs to know whether 50d28600a is the fix for both or the last of a pair — the same reason the fconst19 rows name the twin they were measured
against.

[bong-water-water-bong]

VERDICT: READY at 79ed00e

Pin bump, checked the three ways that make one safe — and this one completes a two-step bump rather than jumping:

pin:            fa533bd21  ->  50d28600a
fast-forward:   git merge-base --is-ancestor fa533bd21 50d28600a  ->  yes
commits added:  50d28600a "hrx: the multi-section rotation of Qwen3-VL runs on the device, no…"
#1987 (the bump TO fa533bd21) is MERGED

So: the cited commit exists locally, the move is a fast-forward (nothing reviewed was rewritten), and it lands on top of a pin the tree already carries — which is
the least risky shape a bump takes.

The release note is the part worth reading, and it names two faults rather than one:

1. a refused view — "a deepstack model adds a slice of the wide input embedding to each of its first layers, taken as a view of a view. With the first view's
   node in another split, the HRX graph never holds the wide tensor, the slice could not alias its input, and the graph was refused at unsupported HRX node 20: VIEW. So Qwen3-VL-2B could not decode even text, and every image request failed" — that is a precise chain from a graph-shape fact to a user-visible
   symptom, and it matches the not_aliasing_input work I reviewed in #267/#282;
2. the rotation moving on-device, with the measured 2.99 s image request in the title.

The note also names what the fix is not: "both in the text model's graph rather than in the projector" — which saves anyone chasing the vision tower first.

Not built: no llama.cpp tree here; the bump's own CI (hrx-build) is the right instrument for the compile, and the submodule-side checks above are the ones
CI cannot do (it resolves the tree without verifying the old pin is an ancestor of the new one).

[bong-water-water-bong]

VERDICT: READY at 79ed00e — a one-commit bump whose three mechanical facts I verified with the repo's own tools, plus a measurement that is scoped honestly.

The bump, checked the way a bump should be

gitlink   fa533bd21053 (main) → 50d28600a5f7
ancestry  git merge-base --is-ancestor fa533bd21 50d28600a  →  TRUE   (moves FORWARD, drops nothing)
registry  registry/architectures.json names 50d28600a5f7a38df390709b94c463df02785a54  ✓ matches the gitlink
tools     python3 tools/check_pins.py origin/main      → "ok third_party/llama.cpp: fa533bd21 -> 50d28600a is ahead"
          python3 tools/registry_build.py --check-pins → "…/registry/architectures.json records the committed pins"
gates     ci_targets_check rc 0 (9 baseline, none new);  python_test_order_check rc 0
duplicate none — no other open PR bumps to 50d28600a (checked every open head's gitlink)

Also worth recording: #1987 merged eleven minutes of work ago and this is the next pin, not a rival to it — fa533bd21 is on main, and the body says so ("the first half of #1467's fix, llama.cpp#283, reached the pin with #1987"). Two lanes bumping the same file in one hour is the shape that produced tonight's #1960/#1961 duplicate; this is the correct version of it, sequenced and stated.

The measurement is scoped the way I want measurements scoped

"Requests 2 and 3 reuse the server's prompt cache (prompt_n = 1), so only the first is an image-encode measurement. The server was started and stopped inside a gpu-hold ticket" — a bump body that distinguishes the number that means something (2.99 s, prompt_ms 2,782, 227 prompt tokens) from the two that do not (0.08 s and 0.09 s, one-token prompt). Publishing the cache hits and labelling them costs nothing and stops a reader from averaging 3 requests into a 1.05 s figure that no one measured.

The doc entry names both faults and the remaining cost

docs/hrx.md gains a "Fixed" section for #1467 with the two causes (the refused view of a view — llama.cpp#283, and the multi-section rotation on the CPU — #286), the 2.99 s against "the 16.97 s this issue was filed on", and — the part most entries skip — what still runs on the CPU: "103 of 626 nodes … its 2-D rotation, the attention score matmul and softmax, IM2COL, UPSCALE". A "Fixed" heading that says which half is fixed is worth more than one that says the issue is closed.

One thing I would add, not a change request

The doc gives 2.99 s and the 16.97 s filing number but not the bar the issue set (≤ 17 s image TTFT through llama-server with the projector on HRX0), which is the sentence that makes 2.99 s a pass rather than a number. One clause — "against the issue's 17 s bar" — and a reader can grade it without opening #1467.

[bong-water-water-bong]

VERDICT: READY at 79ed00e — the only open pin bump, so no subset collision this time (I checked: #2011 is the sole open PR touching third_party/llama.cpp), and the documentation here is the strongest of the three bumps I have reviewed tonight.

Two faults, each attributed to the fork commit that fixes it, which is the property a bump PR needs:

• A refused view (llama.cpp #283). A deepstack model slices the wide input embedding as a view of a view; when the first view's node lands in another split the HRX graph never holds the wide tensor, the slice cannot alias its input, and the graph is refused at unsupported HRX node 20: VIEW. Consequence stated plainly: Qwen3-VL-2B could not decode even text, so every image request failed with the text path's error — the kind of failure where the symptom points at the multimodal part and the cause is in the text graph. And the fix's justification is a reason, not a workaround: "such a view is external to the graph and a layout op writes nothing, so the scheduler now skips it."
• The rotation on the CPU (llama.cpp #286). The interleaved multi-section rotation does not collapse to the plain one, so 56 ROPE nodes per text graph ran on the CPU and cut the graph into 114 splits. That number pair (56 nodes → 114 splits) is the useful form of a performance claim: it says the cost is graph fragmentation, not arithmetic, and it predicts what to look for if the fix regresses.

The measured line follows the standard set by #1960: pin named (50d28600a5f7a38df390709b94c463df02785a54 in both architectures.json and the submodule), gfx1151, performance power mode, and the model named (Qwen3VL-2B-Instruct Q4_K_M). Every bump tonight that named these was re-derivable; the ones that did not were not.

Sibling check and scope: 3 files (+28/−2) — docs/hrx.md, registry/architectures.json, the submodule — with no second open PR touching the pin, so merging this cannot conflict on the architectures.json row the way #1960/#1961 would have.

The one thing I would add, carried over from my #280 review: these are fork commits (#283, #286), so the bump cannot be reconstructed from upstream by SHA alone. The docs section names the PR numbers, which is better than naming nothing — but a line saying whether either is upstream-cherry-picked or fork-only would tell the next bump-hrx run whether to expect them to disappear when AMD rebases. #280 carried a (cherry picked from …) trailer for exactly this reason.

Not verified: the measurements (the author's, on gfx1151) and that the pin compiles — this lane does not build.

[bong-water-water-bong]

VERDICT: READY at 79ed00e — a pin bump one commit past the current pin, carrying the rotation fix I reviewed in llama.cpp #286.

Fixes #1467. One fork commit past the current pin (`fa533bd21` → `50d28600a`): llama.cpp#286, the multi-section rotation
on the device. The first half of #1467's …

Why a one-commit bump is the right shape here

• the fork PR and the pin bump are separate, so the mechanism was reviewable on its own (llama.cpp HRX: wrong values when SWIGLU_OAI runs on HRX after a CPU-placed MUL_MAT_ID in a multi-split graph; hidden by per-op sync #286's per-pair (t,h,w) section selection) and the bump is a record rather than a second review. That is the convention this lane has been reading all day: serve: Hadamard-rotated Q4_0 files run on --device hrx too #277 descriptions in the fork, closure in the pin.
• "one fork commit past the current pin" is the sentence that makes the diff auditable — a bump that moves 40 commits hides its content, one that moves one names it;
• "the first half of hrx: the CLIP graph runs on no HRX backend, so a vision model's projector falls back to the CPU at ~17 s/image #1467's" says the issue stays open, which keeps a green merge from being read as "vision runs now".

The ask from that review, still open on this side

llama.cpp #286 defaulted section_t3/h3/w3 to 0, so a mode-3 call that never sets them rotates every pair by the section at (0,0,0) — plausible and wrong everywhere. Whatever the fork does, the engine should refuse a mode-3 launch whose sections were never set; if the answer is "the importer always sets them", say so where the launch is made.

[bong-water-water-bong]

VERDICT: READY at 79ed00e — the bump is sound and, unlike most, it carries the reason a reader can check.

Bump verified: third_party/llama.cpp → 50d28600a5f7, fast-forward from fa533bd21053 (ahead=1, behind=0, so one commit and nothing ours dropped),
registry lockstep — registry/architectures.json sources["llama.cpp (hrx)"] moves to the same sha byte-for-byte. And the pin is one commit behind the fork
head (69aac7f), which is the right way round: a pin should be a tested commit, not "whatever is newest", and the relationship is checkable
(compare 50d28600a5f7...69aac7fa25a1 → ahead=1, behind=0, i.e. the pin is on the branch and the branch has moved once past it).

The docs entry is what a one-commit bump should look like: it names the two faults the commit fixes in terms of symptoms — a refused view at
unsupported HRX node 20: VIEW for a deepstack model whose slice could not alias, and the rotation issue — rather than saying "bump to 50d2860". A reader hitting
Qwen3-VL-2B before and after can tell which of the two they are looking at.

One note for the queue, since this is the third bump PR I have reviewed today: three of the four were duplicated by another lane (two exact copies earlier), and
the pin updates are now the most frequent conflict source on this repo. If the bump-hrx workflow can open its own PR, it should be the only thing that writes
third_party/llama.cpp + registry/architectures.json, and hand bumps should be reserved for a pin that needs to be held — which is what the engine's
hrx-system hold file already does for the other half of the pair.

Not verified: that the one incoming commit builds (that is hrx_build's job) and the Qwen3-VL end-to-end result (needs the pack and an NPU run).

[bong-water-water-bong]

VERDICT: APPROVE at 79ed00e

This carries the commit I reviewed as llama.cpp#286 — Qwen3-VL's multi-section rotation running on the device instead of silently falling back to the host because the collapse test is false — and Fixes #1467 with the second half named (llama.cpp#283 reached the pin with #1987). Citing tools/check_pins.py origin/main and registry_build.py --check-pins in the body is the right evidence for a bump: the pin moves forward and the registry agrees, stated rather than assumed.

Two asks, both about churn rather than correctness:

1. Does this pin carry the fix's test? That is the check #1960 taught me tonight and it is one API call: is there a fork test at 50d28600a covering the multi-section rotation (and is it absent at fa533bd21)? If yes, say so — it turns "the pin moves forward" into "the pin moves forward and brings the arm". If it is a device-only comparison, say that instead, because then the arm is the model-level logits check I asked for in #286's review.
2. This is the third bump today (2d62cc7af → a5dc597b8 → fa533bd21 → 50d28600a, with #1960/#1961, #1987 and this one). Each is individually right and each costs a registry regenerate plus a CI cycle. If the fork's graph is going to keep producing one-commit fixes at this rate, it may be worth batching to a daily bump — or, if the pins are cheap and the CI is not the bottleneck, say that so the pattern is a choice rather than an accident.

One detail worth stating: fa533bd21 → 50d28600a is one commit, so the risk of the bump is that single fork change — and since #286 is a perf change (moving work off the host), the thing a bump reviewer actually wants is the evidence that the Qwen3-VL output is unchanged. If that comparison exists anywhere (the engine side of #1467), point at it.

[bong-water-water-bong]

VERDICT: READY at this head — the pin names the right sha, the delta is exactly the one commit it describes, and both halves of #1467's fix are where the body says they are.

Checked against the fork, not the body:

origin/1bit/hrx-vulkan-patched:
  69aac7fa  tests: known-answer MUL_MAT cases … (#287)        <- one ahead of the pin target; NOT included, correctly
  50d28600  hrx: the multi-section rotation of Qwen3-VL runs on the device (#286)   <- the pin target
  fa533bd2  hrx: a view into storage the graph does not hold is skipped (#283)      <- already in the current pin
git log fa533bd2..50d28600 -> exactly 50d28600 (one commit, as claimed)
git merge-base --is-ancestor fa533bd2 50d28600 -> yes

So: fa533bd21 → 50d28600a is one fork commit, it is #286, and #283 is already inside the current pin (it arrived with #1987) — which is what the body says in one sentence. The pin does not drag in #287, which is merged on the branch but is not part of this issue; that is the difference between a scoped bump and a "move to branch head" bump, and this one is scoped.

The docs section is the right size for the change: two faults, both in the text model's graph — a VIEW of a view that the graph never holds (skipped, because a layout op writes nothing) and 56 ROPE nodes per graph running on the CPU and cutting the graph into 114 splits. Those are the two numbers a reader can re-measure, and the headline (2.99 s against 16.97 s) has both arms in it.

One thing worth keeping in view for the next bump: #287 is now ahead of the pin on the same branch, so tomorrow's automated bump will pick it up — that is fine, but a pin that is deliberately one commit behind the branch head should say so in the body (this one does, by naming the sha and quoting check_pins.py's is ahead line), because the natural reading of "bump to a sha" is "bump to head".

Unverified by me: the timing measurement (2.99 vs 16.97 s on an image request) and the graph-split counts. Those need the device and the model; what I verified is the pin's identity, its delta, and that the commit inside it is the one the body describes.

[bong-water-water-bong]

VERDICT: READY at 79ed00e

The measurement is the kind that justifies a pin bump on its own: "image request 2.99 s against 16.97 s" — a 5.7× improvement on a user-visible path, from moving the rotation to the device. A pin bump that says "for the Qwen3-VL rotation" and gives both numbers is doing what the last week of pin PRs learned to do: state the effect of the bump, not only the commit.

And the pin arithmetic is stated rather than assumed: "One fork commit past the current pin (fa533bd21 → 50d28600a): llama.cpp#286, the multi-section rotation on the device. The first half of #1467's fix, llama.cpp#283, reached the pin with #1987." — so a reader can tell that this bump completes a fix whose first half is already in, which is exactly the question a later bisect would ask.

The check_pins.py output in the body is the right shape too (ok third_party/llama.cpp: fa533bd21 -> 50d28600a is ahead), because "is ahead" is the property that matters and "matches" would not be. Two small asks:

1. name what in registry/architectures.json moves with it — the body says it "records the committed pins", which is true; a reader rebuilding from the registry needs to know the value changed, not just that the check passed;
2. record the 2.99/16.97 pair next to the bump in the registry or docs/hrx.md, so the number is attached to the pin that produced it and not only to the PR — the same reason docs/hrx.md exists.

Neither blocks: the pin is one fork commit ahead of a tip I have just verified carries the #1539 prescale, so this bump keeps that fix and adds the rotation.

[bong-water-water-bong]

VERDICT: READY at 79ed00e — the pin bump is the one commit it says it is, and the bar the issue set is cleared by ~6× with the measurement to show it.

The row is the right row. The issue's bar is image TTFT ≤ 17 s through llama-server with the projector on HRX0; this reports 2.99 s wall / 2,782 ms prompt_ms for the first request on the issue's own 448×448 image and prompt, where it was filed at 16,841 ms with the projector on the CPU and a 502 with it on HRX0. Requests 2 and 3 are labelled as cache hits (prompt_n 1, 8 ms) rather than counted as three wins — which is the honest way to present a three-request table.

The pin is exactly one fork commit (fa533bd21 → 50d28600a) and that commit is llama.cpp#286, which I reviewed and verdicted — including the gate question I raised there, answered by narrowing sect_dims to >= n_dims/2 with the wrap case left on the CPU. So the behaviour arriving here is the behaviour I read, not a superset.

The contract half is checked, not assumed: tools/check_pins.py (fa533bd21 -> 50d28600a is ahead) and registry_build.py --check-pins (architectures.json records the committed pins). That is the pair that matters after a bump — the generated registry follows the pin, and a bump that leaves it stale makes the census lie about what we carry.

The rotation itself is covered at two levels: test-backend-ops -o ROPE -b HRX0 166/166, and HRX0 text against the CPU backend byte-identical on 12 greedy tokens.

One suggestion, from a mistake I made today: 12 greedy tokens is a short text identity, and this PR's whole risk is a rotation that is right near the start and wrong further out — I spent an hour this evening chasing a "9× regression" that turned out to be a cold cache, and the lesson I wrote down was that short inputs do not separate a real difference from a state difference. If the same byte-identity check can run at 200+ tokens cheaply, it would separate a rotation bug from noise; the op test is what makes me comfortable shipping without it.

Ticket discipline noted and appreciated: "one gpu-hold ticket" for the measurement, and the build is named (50d28600a, gfx1151, amdclang Release, hrx-system 7f42369) so the row can be reproduced rather than trusted.

[bong-water-water-bong]

VERDICT: READY at 79ed00e — A submodule bump whose title states the behavioural half — Qwen3-VL's rotation now runs on the device — with docs/hrx.md +26 carrying the explanation. That is the right shape for a bump that changes where work happens: the pin alone would be opaque, and the docs line is what tells the next reader whether a decode-rate change is expected.

[bong-water-water-bong]

VERDICT: READY at 79ed00e

The evidence set is what makes this a pin bump rather than a hope, and it is measured in the order that matters:

• test-backend-ops -o ROPE -b HRX0 166/166 — the op is right before any model is involved;
• 12 greedy text tokens byte-identical to the CPU backend — the route is right, not merely fast;
• llama-bench -p 512 -n 64: 6,481 / 6,532 tok/s prefill, 133.7 / 135.5 tok/s decode against 2,331 and 57.8 before the rotation moved — 2.3× decode and 2.8× prefill from one kernel mode, which is what the 114-split diagnosis in llama.cpp#286 predicted;
• and the projector run (448x448 first request 2.99 s, prompt_ms 2,782 for 227 tokens).

Naming two faults and which PR fixed which (#283 the refused view, #286 the rotation) is the part a future bisect will thank you for: a pin bump that carries two unrelated fixes hides which one is being blamed, and this body does not.

Two asks:

1. docs/hrx.md says "2 or 3 splits ... PAD (and GET_ROWS on token batches) left on the CPU" — give the exact count for the run you measured. "2 or 3" is a range from two different token-batch shapes; the doc should say which shape produced which, because the next person comparing split counts needs a number to match, not a set.
2. One line on what remains on the CPU and why it is acceptable. PAD and GET_ROWS being host ops is an honest state; saying why (no registration, or a registration that does not pay for itself at these shapes) is what stops it being read as "unfinished" — and if the answer is "no registration exists", that is a follow-up issue rather than a doc sentence.

Both the check_pins.py and registry_build.py --check-pins outputs being in the body is the habit every pin bump should have.