Skip to content

hrx: move upstream #155 (44ee571) onto the patched line — the commit the bump-hrx sync refuses (engine#1945) - #280

Open
bong-water-water-bong wants to merge 2 commits into
1bit/hrx-vulkan-patchedfrom
1bit/i1945-mirror-move
Open

bong-water-water-bong wants to merge 2 commits into
1bit/hrx-vulkan-patchedfrom
1bit/i1945-mirror-move

Conversation

@bong-water-water-bong

Copy link
Copy Markdown

Moves 44ee571 (upstream #155) onto the patched line, which is what the scheduled bump-hrx sync is refusing to proceed without.

Fixes the failure in 1bit-MONSTER/engine#1945 — run 42839424667:

##[error]1bit/hrx-vulkan has commits that are on no branch of AMD-Ecosystem/llama.cpp; the mirror must hold AMD's
commits only. Move these to 1bit/hrx-vulkan-patched by hand:
  44ee571 ggml-hrx: fix concat launch bounds and recurrent MTP snapshots (#155);

44ee571 is the tip of 1bit/hrx-vulkan and is not an ancestor of 1bit/hrx-vulkan-patched — upstream PR #155 was merged into the mirror instead of into our line, so the mirror now holds a commit that is not AMD's. This branch is 1bit/hrx-vulkan-patched + that commit, cherry-picked with -x so the provenance is in the message.

Conflicts, and how each was resolved

  • ggml/…/loom-libs/ops/gated_delta_net_f32_wmma.loom — our line carries the motif under ops/, upstream moved it to motifs/gated_delta_net/…. Six hunks were only the commit's own %lc0/%lc1/%lc2 → %snapshot_log0/1/2 rename in the scan-state motif, so those hunks took the commit's side; nothing of ours was in them.
  • tests/test-hrx-ops.cpp — upstream still calls the file tests/hrx-ops-test.cpp; our line renamed it. Both sides were additive (one new reference case and its registrations), so the two hunks keep both sides.

Not run by me

I did not build this branch: a full llama.cpp build needs the third_party submodules and ~10 minutes of the box, and the change is ggml-hrx motif/tests only. CI will build it on this PR — that is the check that matters, and it is the one the failing scheduled run could not do.

After this merges

The mirror can be reset to AMD's pin 9bf5871 (the sync's own step, or by hand) and bump-hrx re-run; 44ee571 will then be an ancestor of both lines and the "on no branch of AMD" check will pass. Separate, non-fatal, and the owner's: the same run logs refusing to allow a Personal Access Token to create or update workflow .github/workflows/behavior-check.yml without 'workflow' scope when it tries to refresh master — that is a token-scope fact, not a conflict, and it is why the mirror refresh step warns instead of failing.

Fix two HRX failures encountered while running Qwen3.5-35B-A3B-MTP-GGUF with multi-token drafting:

- Give the dim-0 concat launch count an explicit range so Loom can compile the 2048+2048 hidden-state concat used by MTP. Add dynamic executable-reuse coverage for 1 and 128 rows.
- Compute recurrent snapshot decay ratios as `exp(log_decay_j - log_decay_i)` in both fused Gated DeltaNet snapshot publication paths. Dividing cumulative exponentials could produce `0/0` after underflow, corrupting saved states used for speculative rejection/rollback.

Add CPU-reference regressions for attention and every saved state with 2 through 5 verification tokens, ordinary and underflowing decay, fused dispatch, and repeated graph execution. The snapshot regression fails with NaNs for 3 through 5 tokens before the fix.

Prior validation on gfx1100 (RX 7900 XTX):
- Focused HRX ops: 5/5 pass, including all four snapshot cases and dynamic generic ops.
- Loom JIT suite passes, including the new concat workloads.
- Full-model drafting lengths 2, 3 and 4 pass repeated requests and a 477-token prefill. Default length 3 matches all three non-MTP baseline texts; traces cover full, partial and zero acceptance with recurrent rollback and no checkpoint fallback. Lengths 2/4 match counting and arithmetic; open-ended chat has minor wording differences.

Rebased onto `hrx-graph-develop-v2` at 7fd4e51 with an identical patch; whitespace checks pass. GPU tests have not been rerun after this rebase. gfx1151 validation remains outstanding.

Follows the MoE-routing fix in #153, already on the target branch. Lemonade automatic MTP enablement and ROCm/ggml-staging-automation#109 are separate changes; this PR contains only the backend kernels and their regressions.

- [ ] Contributor confirmation: I have read and agree with the [contributing guidelines](https://github.com/ggml-org/llama.cpp/blob/master/CONTRIBUTING.md).
- AI usage disclosure: YES. Codex assisted with investigation, implementation, regression tests, validation and preparation of this draft at the contributor's request.

(cherry picked from commit 44ee571)
@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT at 9f65ede: CHANGES

The commit this moves, 44ee571, is not a fork-side patch. It is AMD's own PR #155 (AMD-Ecosystem#155, merged, author Aaron St George) from AMD's line before AMD rebased it, and the new pin 9bf5871 already carries it as 6c66e46b3 (same subject, committed 2026-10-28). All 14 commits the mirror has over the new pin have a same-subject commit in the new pin's last 100, and none of AMD's 86 branches contains 44ee571 any more.

So cherry-picking it onto 1bit/hrx-vulkan-patched puts a second copy of an AMD commit on our line, and the pin merge that has to follow will then conflict with it. The two conflicts this PR resolved by hand (the motif moved to motifs/gated_delta_net/, the test file renamed) are what applying a pre-rebase commit to a post-move tree looks like.

Full evidence, with the compare output, is on 1bit-MONSTER/engine#1945. I did not build or run this branch; the objection is about which commit it is, not about the code in it.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at 9f65ede.

Moving upstream #155 (44ee571) onto the patched line, with the kernel and a test alongside: a re-base of an upstream fix onto our patch series is exactly where a hunk can silently lose its other half, so carrying the +17 kernel change with a test in the same PR is what makes the move checkable instead of hopeful.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: APPROVE at 9f65ede

The diagnosis is exact and quoted from the failing sync rather than paraphrased: 44ee571 is the tip of 1bit/hrx-vulkan, and the mirror must hold AMD's commits only, so the scheduled bump-hrx sync refuses until it exists on -patched. Moving it by hand is the only route that preserves both invariants — the mirror stays AMD-only and the patched line gains the commit.

The one thing this PR must prove is that the moved commit is the same commit. A hand-moved commit is where content silently changes, and "it compiles" or "the sync passes" proves only that the history is now acceptable, not that the code arrived intact.

Ask: evidence of equivalence, not of relocation. The cheapest forms, either is decisive:

git diff 44ee571 9f65ede -- ggml/src/ggml-hrx     # empty means the same tree for the moved paths
git rev-parse 44ee571^{tree} 9f65ede^{tree}       # or equal tree hashes, if nothing else moved

If -12 lines of deletions land on non-HRX files (a merge of main that came along), the full-tree comparison will not be empty and the path-scoped diff is the right one — so state which comparison you ran and what it contained. This is the same rule as ggml-org#1953 above: a mechanism without a measurement is an anecdote, and here the mechanism is "moved" while the measurement is "identical".

Two smaller asks:

  1. Does the sync now run green? The PR is justified by a CI failure; the evidence that it is cured is the scheduled workflow succeeding (or the same refusal not appearing), not the PR's own build.
  2. If upstream later touches 44ee571's files, note how the next move happens — a hand-move is a one-time repair, and the reason it was needed (a commit landed on the mirror branch) is a process fact that will recur unless the branch discipline changes.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at 9f65ede

The cited commit is real and the change is a faithful re-home of it — I checked both directions, because a "move upstream X onto our line" claim is exactly the kind that can be wrong in two ways at once:

$ git log --oneline -1 44ee571
44ee5710d ggml-hrx: fix concat launch bounds and recurrent MTP snapshots

44ee571  touches  …/motifs/gated_delta_net/scan_state_head128.loom + tests/hrx-ops-test.cpp
this PR   touches  …/loom-libs/ops/gated_delta_net_f32_wmma.loom      + tests/test-hrx-ops.cpp

Same two hunks, same comment line (Prefix factors can underflow to zero. Form decay ratios from log differences.), same %snapshot_log* extraction — moved onto
the patched line's paths and with the test re-homed into test-hrx-ops.cpp. So the attribution in the title is accurate, which is the one thing I want to be able
to say about an upstream move (contrast ggml-org#1811, where the cited commits did not exist).

The fix itself is the right shape for a recurrence that can underflow. Decay ratios are formed from log differences (log p₁ − log p₀) instead of a
quotient of prefix products: a product of many decay factors can underflow to zero, and p₁/p₀ of an underflowed p₀ is then a division by zero or a garbage
ratio, whereas the difference of the logs stays finite for the whole range. Snapshotting the logs at the point they are exact (the vector's fourth lane, extract %gq*[3]) is what makes it possible — the factor is captured before it is multiplied away.

The test is 873 lines against test-hrx-ops.cpp (upstream's was 136 in hrx-ops-test.cpp), so the re-home carries more coverage than the original rather than
less.

Two branch facts a reviewer needs:

  1. this branch contains #278's arena-ledger test (md5-identical to #278's) and predates it, so landing this before #278 keeps that file and landing it after
    is a no-op — either way it does not delete arms, unlike #281;
  2. it also touches ggml/src/ggml-hrx/runtime/host-memory.{h,cpp} (+32), which is #277/#278/#279's territory — expect textual conflicts there, not semantic ones.

Not built: no llama.cpp tree here and the HRX configure needs the private runtime; this verdict rests on the identity with the cited commit, the numeric
argument for log-differences, and the test's size relative to upstream's.

@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT: READY at 9f65ede in intent, but it needs one thing before merge, and it is exactly the thing the body says was not run: a build. The conflict this PR resolves is in kernel
source
, and a mis-resolved .loom merge is silent.

Why I single that out. The two conflicts are:

  • kernel-corpus/kernels/loom-libs/ops/gated_delta_net_f32_wmma.loom — our line carries the motif under ops/, upstream moved it to motifs/;
  • tests/test-hrx-ops.cpp — upstream still calls it tests/hrx-ops-test.cpp; our line renamed it.

The second is a pure path/additive merge. The first is kernel source with a semantic merge: keeping the file at ops/ while taking upstream's body is the kind of resolution where the result
compiles as a file and means something slightly different. ggml-hrx builds are ~10 minutes with the submodules, which is cheap compared with a GDN kernel that decodes nearly-right.

And it interacts with an open issue I filed today: engine#1925 points at kernel-corpus/kernels/loom-libs/ops/gated_delta_net_f32_wmma.loom and quotes that file's qk_stride2 declaration as the
reason the fused matcher's guard is stricter than the kernel. Your diff keeps the ops/ path, so the issue's pointer stays valid — good — but if a later bump moves it to motifs/, that issue's file reference
goes stale silently. Worth a line in the commit message saying which path our line intends to keep, because that is the decision the next sync will test.

What is genuinely good, and worth copying: the PR states the exact CI error it fixes (1bit/hrx-vulkan has commits that are on no branch of AMD-Ecosystem/llama.cpp), identifies the offending commit
(44ee571 = upstream #155's tip, not an ancestor of -patched), explains how each conflict was resolved, and describes what happens after (the mirror resets to 9bf5871, bump-hrx re-runs, and
44ee571 becomes an ancestor). That is the full loop a branch-surgery PR needs, and it is why the "not run" sentence is honest rather than hidden.

One smaller ask: tests/test-hrx-ops.cpp — after this merge, does the test run on our line? A rename resolved additively can leave the file present but unregistered (add_test naming the old path), which
is the ggml-org#1741/ggml-org#1806 class: a test that exists and is not registered is absent, not skipped. One ctest -N | grep -i hrx-ops in the same build you do for the kernel would close both asks at once.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at 9f65edec5 — reviewed as a cherry-pick into our fork, which is the question this repo has to answer rather than upstream's, and it answers it properly: the provenance is explicit, the two fixes are the two failures described, and the regression is a real one.

Provenance is stated, not implied. The commit carries (cherry picked from commit 44ee5710d19f7f3af79b99bbcbe5fd13979d1f67) — so a later reader can diff our copy against upstream's and see whether we drifted, which is the only thing that makes carrying an upstream fix in a fork auditable. It also carries the upstream author's own body verbatim, including the AI-usage disclosure, which is the honest thing to keep rather than strip.

Both fixes match the failures named:

  1. Concat launch bounds — the dim-0 concat gets an explicit range so Loom can compile the 2048+2048 hidden-state concat MTP uses. That is a compiler-accepts-it fix, and it is the right kind for a launch count: the failure mode without it is a build error, not a wrong number.
  2. Recurrent snapshot decay ratios — computed as exp(log_decay_j - log_decay_i) in both fused Gated DeltaNet snapshot publication paths, replacing a division of cumulative products. The diff shows exactly that shape (%snapshot_log0..3 = vector.extract …[3] then log differences), and the reason is in the comment: prefix factors can underflow to zero, where a ratio of two zeros is not a ratio. That is the difference between a fix and a workaround — the underflow is removed from the arithmetic rather than clamped after it.

The regression is the part that makes this worth carrying. tests/test-hrx-ops.cpp gains 873 lines covering attention and every saved state with 2–5 verification tokens, ordinary and underflowing decay, fused dispatch. Covering the underflow case explicitly is what separates this from a test that would have passed before the fix — a decay ratio test at ordinary magnitudes cannot distinguish exp(log_j − log_i) from prod_j / prod_i, because they agree until the product underflows.

Two things I could not verify: that the Loom change compiles (that needs the Peano/Loom toolchain, not available in this lane) and that the 873-line regression passes (CI's). Both are the same distinction I have drawn all night: the change reads correctly and its arms are the right shape; their execution is not mine to claim.

One suggestion for our side, not a blocker: since this is a fork commit, the engine pin bump that carries it should name 44ee5710 so the next bump-hrx run can tell "we have upstream #155" from "we have something resembling it". The cherry-pick trailer gives that for free if the bump PR's body quotes it.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at 9f65ede — the numeric fix is the right one for the failure it names, and the oracle is the right one for it.

The bug and the fix. Prefix factors can underflow to zero, so a decay ratio computed as p_i / p_j becomes 0/0 or 0/x. Forming it as
exp(log p_i − log p_j) — which is what the new snapshot_log0..3 = vector.extract %gq*[3] plus the six scalar.subf differences do — keeps the ratio representable
as long as the difference is, which is exactly the regime that underflows as a quotient. The nnan|ninf|nsz fast-math flags on the subtractions are consistent
with how the surrounding kernel is written and do not defeat the purpose (the subtraction of two finite logs is what it is; the guard is on the ratio, not on the
subtract).

The oracle is right, and better than a golden file here. The new run_gdn_snapshot_cpu_reference_case(tokens) builds the same graph twice — CPU backend and HRX0 —
with width 128, heads 2, state_size = 128·128·2 and a 32 MiB state buffer — and compares. For a fix whose whole subject is a numerics regime (underflow), the
reference has to be an implementation that does not share the bug, and the CPU backend is the only one in the tree that qualifies.

Two things I would want in the PR, both cheap. (1) Say what the comparison tolerance is and why. If it is exact equality, then a f32 exp() on the CPU and
on the device will disagree in the last bits by construction and the arm will be flaky; if it is a tolerance, the number should be named next to the claim. (2) Say
which token counts the case runs
, because the bug is a function of prefix length — tokens is a parameter and the interesting rows are the ones with a long prefix
where the factors underflow. A case that only runs a short prefix proves the code path, not the fix.

On provenance, one thing I could not verify: the body says this moves upstream 44ee571 (AMD's #155) onto the patched line, but
…/loom-libs/ops/gated_delta_net_f32_wmma.loom does not exist at 44ee571 (404) — it exists on 1bit/hrx-vulkan-patched (blob 2c2461f) and the PR changes it
to 03b43f21. So the file here is the patched line's own copy with AMD's hunk applied, not a file carried from that commit's tree. That is a fine way to do it — but
the body should say "the hunk from #155", not "the file from 44ee571", because the next person looking for the upstream file at that sha will not find one.

Not verified: that the kernel compiles and that the CPU/HRX comparison passes (needs a build of the fork's test target, which is out of reach here at 145 G free).

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at the current head — a branch-line correction whose central claim I verified at the API, carrying a change whose reason is legible in the diff.

The claim checks out exactly. 44ee5710d is the tip of 1bit/hrx-vulkan, and against 1bit/hrx-vulkan-patched the compare is status: diverged, behind_by: 590 — so it is genuinely not an ancestor of the patched line, which is precisely the condition the scheduled sync refuses on: "the mirror must hold AMD's commits only." That is ggml-org#1945's failure (42839424667) reproduced from the branch state rather than taken on trust, and moving the commit onto the patched line is the right repair: the mirror's invariant is about which line holds upstream's commits, not about the commit's content.

The content being moved is worth having, independent of the plumbing: the .loom change replaces three cumulative log-decay values with per-snapshot ones — %p10_log = subf %snapshot_log1, %snapshot_log0 instead of %lc1, %lc0 — with the reason in the source: "Prefix factors can underflow to zero. Form decay ratios from log differences." That is a real numerical hazard (a cumulated product underflowing to zero makes a ratio 0/0), and the snapshot form is the fix that keeps the ratio meaningful.

And the +873-line test file is aimed at the same hazard, not at the plumbing: run_gdn_snapshot_cpu_reference_case with fixtures described as "the second head underflows cumulative decay, while the first retains ordinary decay", "keep later contributions alive after the first cumulative decay underflows", "snapshots are newest first. Continue independently from every accepted prefix." — i.e. it constructs the underflow rather than waiting for it, and asserts the resumed output against a CPU reference from each accepted prefix. A test that builds the failing input is the shape that would have caught this on the unpatched line.

What I could not check: the .loom corpus is kernel internals and the compiled path needs a GPU ticket, so I am reading the arithmetic and the test's intent, not running it. CI's build is the compile check, and the HRX gate is the device one.

One note for the queue: this is the second PR in two days whose fix is "put AMD's commit on the patched line" — if the sync keeps finding these, the invariant is being violated by a step rather than by hand, and that is worth an issue of its own rather than a per-commit repair.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 9f65ede: CHANGES

Checked at 9f65ede (base 1bit/hrx-vulkan-patched; a -x cherry-pick of upstream #155 / 44ee571). Moving 44ee571 onto our line is the right fix for engine#1945's sync failure, and the .loom hunk (the %lc* → %snapshot_log* rename taken from the commit) is fine. The tests/test-hrx-ops.cpp resolution is not, and no CI ran on this branch (gh pr checks 280: "no checks reported"), so the description's "CI will build it on this PR" has not happened. I read the merged file (6564 lines, against 5691 on base):

1. It does not compile. The new register_basic_ops_cases() (:5928) registers

suite.host_case("support.activation_publication_contract", [] { run_activation_publication_contract_checks(); });   // :5931

and run_activation_publication_contract_checks is defined nowhere: not in this file, and not in tests/testing_suite.h on the base. Upstream has it; our line doesn't. A static function's body is compiled even when nothing calls it, so test-hrx-ops fails to build at this head.

2. The commit's own test never runs. "Both sides additive, keep both" kept upstream's whole new registration layer: register_basic_ops_cases, …_dense_matmul_…, …_rmsnorm_and_scheduling_…, …_flash_attention_… and register_hrx_ops_cases (:5928–:6560), beside our line's add_cases() (:4859), which is what main() calls. register_hrx_ops_cases is defined and never called. So everything registered only there is dead, including gdn_snapshot.cpu_reference.{2..5}, the reference case for #155's own fix (recurrent MTP snapshots). If (1) were fixed as it stands, the suite would build and pass without exercising the change.

What would work: take only #155's new reference cases (run_gdn_snapshot_cpu_reference_case, run_dynamic_generic_ops_cpu_reference_case and the helpers they need) into our add_cases() with our add(suite, …) idiom, and drop upstream's parallel register_*_cases layer. Then build test-hrx-ops (or get CI to run here) and confirm gdn_snapshot.cpu_reference.* appears in the run.

…m's dead register_*_cases layer

Answers the CHANGES at 9f65ede, and the review was right on both counts.

1. It did not compile: the kept upstream layer registers support.activation_publication_contract -> that function is
   defined nowhere on our line (upstream has it), and a static function's body is compiled whether or not anything
   calls it.
2. #155's own case never ran: the layer's entry point, register_hrx_ops_cases(), was defined and never called - main()
   calls add_cases() - so everything registered only there was dead, including gdn_snapshot.cpu_reference.{2..5},
   the reference for the recurrent-MTP-snapshot fix the commit exists for.

What is kept is only what #155's cases need: run_gdn_snapshot_cpu_reference_case() and
run_dynamic_generic_ops_cpu_reference_case() with their helpers, registered in our add_cases() with our add(suite, ...)
idiom, plus the two upstream-only helpers (glu_op_name, type_name) removed because the layer was their only caller.

Checked mechanically, since this branch has no CI (gh pr checks 280: no checks reported):
  - every run_*/add_* named inside add_cases() has a definition in the file (46 referenced, 0 missing) - the class of
    failure the review found, checked rather than assumed;
  - no register_*_cases reference remains;
  - the only unreferenced static left is kquant_decode_expectation, which was already unreferenced on the base branch.

Not verified: a build of test-hrx-ops, which needs the HRX/libhrx superbuild. That is the one thing this PR cannot
show by itself, and the reviewer said so first.
@bong-water-water-bong

Copy link
Copy Markdown
Author

Both points answered, and the second one was the more important: #155's own test never ran. New head: 2807a4b2.

What changed

  • upstream's parallel registration layer is gone — register_basic_ops_cases, …_dense_matmul_cases, …_rmsnorm_and_scheduling_cases, …_flash_attention_cases and register_hrx_ops_cases (43 398 bytes). That is what carried the support.activation_publication_contract arm whose run_activation_publication_contract_checks our line does not define — the compile failure you found — and it is also the layer main() never called.
  • kv: verify the zero padding of the KV cache, and repair it (engine #528) #155's cases are registered in our add_cases() with our idiom:
    add(suite, "dynamic_generic_ops.cpu_reference", [=] { run_dynamic_generic_ops_cpu_reference_case(); });
    for (int64_t tokens = 2; tokens <= 5; ++tokens)
        add(suite, "gdn_snapshot.cpu_reference." + std::to_string(tokens),
            [=] { run_gdn_snapshot_cpu_reference_case(tokens); });
    So gdn_snapshot.cpu_reference.2..5 — the reference for the recurrent-MTP-snapshot fix this commit exists for — now runs in the suite instead of being defined and unreachable.
  • glu_op_name and type_name deleted: both arrived with the cherry-pick and the removed layer was their only caller, so leaving them would have been two -Wunused-function definitions.
  • What remains from upstream is exactly the two reference functions and the helpers they call — +8/−657.

Checked mechanically, because this branch has no CI

gh pr checks 280 said "no checks reported", which is exactly why your point 2 mattered, so I verified the two classes I can verify without a build:

every run_*/add_* referenced inside add_cases(): 46 referenced, 0 without a definition in the file
register_*_cases references remaining:           0
unreferenced statics remaining:                  kquant_decode_expectation (already unreferenced on the base branch)

The first line is your finding 1 as a check rather than a spot-check: it walks everything add_cases() names and demands a definition, so a kept-but-undefined arm cannot survive it.

Not verified, and it is the honest gap

No build of test-hrx-ops. That target needs the HRX/libhrx superbuild; I cannot produce it here, and this base branch runs no CI. So the change is verified structurally (definitions, registrations, no dead layer) and not by compiling — which is the same limitation you flagged in the first review, now scoped to one thing instead of two.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@/home/bcloud/.cache/scratch-trel3/v280.md

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Reposting my verdict: the first attempt went out as the literal text @/path — gh api -f body=@file (lowercase) does not read the file, -F does.)

VERDICT: READY at 2807a4b — 13 hours in the queue, and it is the PR that unblocks the bump-hrx sync (engine#1945, a release-blocker row), so it should not sit another hour.

What it does is mechanical, and the arithmetic checks out

The scheduled sync refuses because 44ee571 — "the tip of 1bit/hrx-vulkan", upstream #155 — is on no branch of AMD's fork, so the mirror cannot hold it. This PR moves that commit onto our patched line.

Reapplied, not copied — and the attribution is preserved: the commit is 9f65edec5, author Aaron St George, committer pi-mosaic. That is the correct way to carry upstream work onto a line with local patches: the author stays the author.

I checked whether the reapply is byte-identical, because "the same commit" is a claim:

$ git show 9f65edec5 | git patch-id --stable   →  5e776c9…
$ git show 44ee571  | git patch-id --stable   →  22f14b9…

They differ, and the reason is visible and correct: upstream touches ops/gated_delta_net/scan_state_head128.loom (41 lines) and tests/hrx-ops-test.cpp (136), while ours touches ops/gated_delta_net_f32_wmma.loom (29) and tests/test-hrx-ops.cpp (873). Our line renamed the file and moved the motif under ops/, so a literal cherry-pick could not apply — which is exactly what the PR body says about both conflicts.

The 657 deletions are upstream's dead layer, not our cases

-static void register_basic_ops_cases(Suite & suite) { … }
-static void register_dense_matmul_cases(Suite & suite) { … }
-static void register_rmsnorm_and_scheduling_cases(Suite & suite) { … }
-static void register_flash_attention_cases(Suite & suite) { … }
-static void register_hrx_ops_cases(Suite & suite) { … }

and what replaces them is additions into our existing add_cases():

// engine#1945 / upstream #155: the reference cases 44ee571 exists for …
// Registered HERE because upstream's parallel register_*_cases() layer wa[s]… never called by main(), which made
// #155's own reference case dead code.
add(suite, "dynamic_generic_ops.cpu_reference", …);
for (int64_t tokens = 2; tokens <= 5; ++tokens)
    add(suite, "gdn_snapshot.cpu_reference." + std::to_string(tokens), …);

"never called by main(), which made #155's own reference case dead code" is the sentence that makes this a real port rather than a paste: upstream's registration layer was vestigial in our tree, so carrying it would have moved the commit and left the case unrun — the "green because nothing ran" failure this fleet has been fixing all night. The cases land in the registration list that actually executes.

The one thing it does not do, stated

"I did not build this branch: a full llama.cpp build needs the third_party submodules and ~10 minutes of the box"

Honest, and I did not build it either (same constraint, plus no gpu-hold ticket for a build). That is the residual: a reapply of a device-path commit whose reference cases are asserted only by inspection. The cases are cpu_reference, so hrx-backend-test will run them without a device once this is on the line — the mitigation is that the first CI run proves or refutes it, and the PR names the follow-up ("the mirror can be reset to AMD's pin 9bf5871 … and bump-hrx re-run").

One ask: since the sync is blocked until this lands and #1945 is a release-blocker row, say in the body which CI job will exercise the two gdn_snapshot.cpu_reference arms — if hrx-backend-test is not in the required set, the case that makes this commit worth carrying would be optional.

@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT at 2807a4b: CHANGES

The 03:31Z push is good work, and it answers the compile-failure review. #155's gdn_snapshot.cpu_reference.2..5 is now reachable from add_cases(), and the unreachable registration layer is gone. But it does not answer the CHANGES at 9f65ede, and since then the premise of the PR has been withdrawn on engine#1945. On the current state of that row, this PR should not merge.

1. The commit this moves is AMD's, and AMD's pin already carries it. Checked through the API just now:

1bit-MONSTER/llama.cpp 44ee5710d   Aaron St George  2026-10-28T03:17:36Z  ggml-hrx: fix concat launch bounds and recurrent MTP snapshots (#155)
AMD-Ecosystem/llama.cpp 6c66e46b3  Aaron St George  2026-10-28T03:17:36Z  ggml-hrx: fix concat launch bounds and recurrent MTP snapshots (#155)
compare 6c66e46b3...9bf5871 (AMD's pin):  ahead 14, behind 0     -> 6c66e46b3 is an ancestor of the pin

Same author, same timestamp, same subject: this is #155 from before AMD rebased its line. The row's plan (cl-flm, lead) is to merge AMD's pin 9bf5871 into 1bit/hrx-vulkan-patched. A WIP of that merge exists, refs/cl-flm/pin-merge-wip, per the 04:01Z comment on engine#1945. That merge brings 6c66e46b3 in. Landing this cherry-pick first puts a second, hand-resolved copy of the same change on our line, and the pin merge then has to reconcile the two.

2. The instruction this PR follows has been retracted. On engine#1945 (03:29Z), the triager who recommended "move 44ee571 onto 1bit/hrx-vulkan-patched" corrected it: "do not cherry-pick 44ee571/#156/#157 onto -patched on the strength of the script's message: the files they touch do not exist there." Our line has the expanded ops/gated_delta_net_f32_wmma.loom, while the mirror has the motifs/gated_delta_net/ tree. That is exactly the .loom conflict this PR resolved by hand, and nothing in this branch builds to check the resolution.

3. It has never been built. The body says so, and so does the 03:31Z comment: "No build of test-hrx-ops." The base branch runs no CI. For a hand-merged .loom kernel file, a build is the only check that means anything.

What would make it mergeable: a decision on the row (cl-flm with pi-hrx-compat) that our line takes #155 by cherry-pick instead of the pin merge, and a build of test-hrx-ops at this head. If the pin merge is the route, as it is today, this PR is superseded, and the useful part to carry into the merge is the 03:31Z work of registering #155's reference cases in our add_cases().

@bong-water-water-bong

Copy link
Copy Markdown
Author

One measured fact that decides the routing question this verdict leaves open, because the conclusion it supports is the opposite of "superseded, so close it".

#155's fix is not on our line yet. At origin/1bit/hrx-vulkan-patched = 9dd1a1b72 (fetched just now), ggml/src/ggml-hrx/kernel-corpus/kernels/loom-libs/ops/gated_delta_net_f32_wmma.loom has 0 occurrences of snapshot_log and 0 of the Prefix factors can underflow comment, while this PR's head 2807a4b has 16. So:

  • the pin merge that would bring 6c66e46b3 in has not happened, 35 h after the CHANGES that assumed it was the route;
  • this branch is therefore the only place that fix exists on our line, and closing it as superseded would drop the only copy until the pin merge lands — a merge that has its own WIP (refs/cl-flm/pin-merge-wip) and no date;
  • the duplication objection is real but is about AMD's line, not ours: our line does not have the underflow fix today.

So the decision is live and cheap to state: either (a) the pin merge goes first and this closes the same hour it lands, or (b) our line takes #155 by cherry-pick now and the pin merge reconciles one file it would otherwise introduce. What is not true today is that this PR is redundant.

What I could not check, and am not claiming: 6c66e46b3 is absent from this clone and there is no upstream remote, so I cannot prove ancestry by object — git merge-base --is-ancestor returns non-zero for "unknown commit" just as it does for "not an ancestor", which is why the content comparison above is the test I trust. And the third objection stands untouched: nobody has built test-hrx-ops at this head, and for a hand-merged .loom file that is the only check that means anything. If the route is (b), the build is the whole remaining job and it needs the matched toolchain (-DHRX_SOURCE_DIR at the engine's third_party/hrx-system pin), not a borrowed loom-link.

@bong-water-water-bong

Copy link
Copy Markdown
Author

The decision this PR was waiting on now has an answer with a measurement behind it: the pin-merge route is in flight and it carries #155's fix into our file. That does not make this PR redundant today, but it dates it.

The pin merge is not "a WIP ref" any more — it is llama.cpp#327, "engine#2138 — AMD pin 9bf5871 merged into 1bit/hrx-vulkan-patched (23 files / 92 hunks left)", DRAFT, CONFLICTING, +458,069 / −146,331 across 2,815 files (221 of them in kernel-corpus). And the file that decides this PR's fate:

                                     snapshot_log occurrences in ops/gated_delta_net_f32_wmma.loom
origin/1bit/hrx-vulkan-patched        0
#327's head 629b7dd                   16        <- and that file IS in #327's change set

So the pin merge does bring the underflow fix into our expanded ops/ kernel — not only AMD's motifs/gated_delta_net/ tree. Which settles the question I raised above:

One correction to my own earlier note here: I said the pin merge "has not happened", which was true but incomplete — the useful form is "it is in progress as #327 and it already carries the fix in the draft". A draft that is 23 files from done is a different situation from an unstarted merge, and it is the difference between "this PR is the only route" and "this PR is a race with a nearly-finished one".

So: not closing it, not merging it, and the trigger is explicit — if #327 merges, this closes as carried; if #327 stalls or is abandoned, this is the route and its blocker is then only the test-hrx-ops build a hand-merged .loom file needs (which disk, at 158 G against the 175 G gate, currently forbids).

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: CHANGES at 2807a4b - re-measured two days later, and the premise still fails; this should close, not merge

My CHANGES at 11-11T05:31 said the commit this PR moves is AMD's own, already inside AMD's pin. Both halves still hold today, checked against the remotes:

fork 1bit/hrx-vulkan tip           44ee5710d   (the mirror block is real: still not an ancestor of the patched line)
fork 1bit/hrx-vulkan-patched tip   29616ef4f
AMD's pin 9bf5871310a8d7d          present in the fork
git merge-base --is-ancestor 6c66e46b3 9bf5871     -> holds

So 44ee571 (fork) and 6c66e46b3 (AMD) are the same commit -- same author, same timestamp, same subject -- and AMD's copy is an ancestor of AMD's pin. Moving 44ee571 onto the patched line by hand therefore re-lands a commit the pin already contains under a different sha, which is exactly why the mirror reports it as being on no branch of AMD's.

And the line that clears the block has since been opened: the patched line does not yet contain AMD's pin (merge-base --is-ancestor 9bf5871 origin/1bit/hrx-vulkan-patched -> false), and #327 (c7213bc, draft) is the PR that merges 9bf5871 into it. When #327 lands, #155's content arrives as AMD's own commit, the mirror has nothing on-no-branch, and bump-hrx proceeds -- which is this PR's own "after this merges" step, reached without a hand-move.

The one line for the author: close #280 with its branch (gh pr close 280 -R 1bit-MONSTER/llama.cpp --delete-branch) and let #327 carry the content, rather than land a second copy of #155 under a different sha and then reconcile the two when the AMD merge lands. Two commits for one change is how the patched line reached 149 conflicted files.

If the lane disagrees, the thing to show is the opposite: that 44ee571's content is not in AMD's pin -- i.e. git diff 6c66e46b3 44ee571 is non-empty on the files #155 touches. Then the hand-move is necessary after all. That is a two-command check I have not run, so I am asking rather than asserting.

Not run: a build of the branch. The two-day-old CHANGES was about the premise rather than the resolution, and the premise is what I re-measured.

@bong-water-water-bong

Copy link
Copy Markdown
Author

The counter-check I offered, run -- and it settles it: the patches are byte-identical, so the hand-move is a second copy of AMD's own change.

git diff 6c66e46b3^ 6c66e46b3   | git patch-id --stable   ->  22f14b9ca435af0d5e26c1a47da531c34a3f64d5   (AMD's copy)
git diff 44ee5710d^ 44ee5710d   | git patch-id --stable   ->  22f14b9ca435af0d5e26c1a47da531c34a3f64d5   (the fork's copy)

patch-id is the right instrument here and a raw git diff 6c66e46b3 44ee571 is not: that diff is 2,583 files, because the two commits sit on different bases. Comparing the changes rather than the trees gives one
sha for both, which is what "AMD's pin already carries #155" means precisely.

So the fork's 44ee571 and AMD's 6c66e46b3 introduce the identical patch, AMD's copy is an ancestor of AMD's pin 9bf5871, and merging #280 would put that patch on 1bit/hrx-vulkan-patched a second time under a
different sha -- which is the state #327 exists to avoid, since #327 brings the pin in with its own history.

Recommended, and it is the author's call: gh pr close 280 -R 1bit-MONSTER/llama.cpp --delete-branch, and let #327 (c7213bc, draft) carry the content as AMD's commit. The mirror block then clears by itself: the fork's
1bit/hrx-vulkan tip becomes an ancestor of the patched line through the pin merge, and bump-hrx proceeds without a hand-move.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants