Skip to content

hrx: a host-bound refusal names the host term, not the GTT ceiling (engine #2069) - #296

Closed
bong-water-water-bong wants to merge 1 commit into
1bit/hrx-vulkan-patchedfrom
hrx/2069-name-the-binding-term
Closed

bong-water-water-bong wants to merge 1 commit into
1bit/hrx-vulkan-patchedfrom
hrx/2069-name-the-binding-term

Conversation

@bong-water-water-bong

Copy link
Copy Markdown

Refs engine ggml-org#2069 — a message defect found by hitting it, not by reading it.

The defect

pool_budget_terms() (hrx-1bit-safety.cpp:96-130) takes the smaller of two terms — GTT headroom and host MemAvailable — and sets host_bound when the host one wins. The open-time guardrail line is worded from that flag. The refusal path was not. It printed:

HRX0: refusing to use the device: GTT total 120.0 GiB with 78.0 GiB free leaves nothing under the
104.0 GiB ceiling (reserve GGML_HRX_GTT_RESERVE_MB, override GGML_HRX_POOL_LIMIT_MB); free GTT or lower the reserve

78 GiB free against a 104 GiB ceiling is ~62 GiB of headroom, so that sentence cannot be the explanation — and the advice ("free GTT") names the resource that was not binding.

How it was found

Reproducing engine ggml-org#1926 on a self-built HRX configuration: llama-bench --list-devices registered no device and printed exactly that line. The real cause was the old pin's argument-evaluation-order bug (engine ggml-org#1942, fixed by ggml-org#1960), which makes the host figure read as 0 — a measured zero, which #259/ggml-org#1942 deliberately made refuse rather than read as "unreadable". So the path was correct and the sentence was wrong.

The change

Word the refusal from the terms:

  • host_bound → name host memory, its figure, and the GTT figures it was compared against; advise free RAM (or lower the reserve / set GGML_HRX_POOL_LIMIT_MB);
  • otherwise → the existing GTT/ceiling/reserve wording, unchanged.

Message only. No threshold, no condition, no admission decision changes — budget == 0 && memory_total != 0 keeps its engine-ggml-org#680 fault-line rationale exactly as it was.

Verification

-fsyntax-only against the HRX compile command for this TU, taken from the nested llama.cpp build's compile_commands.json (the HRX backend is built there, not in the top-level project), rc=0. I did not run a functional test: the path needs a host-bound budget to fire, which means a box whose host memory is exhausted — deliberately reproducing that on a shared machine is worse than the message it would prove.

…ngine ggml-org#2069)

pool_budget_terms() takes the smaller of the GTT headroom and host MemAvailable, but the refusal path printed the
GTT figures and advised 'free GTT or lower the reserve' regardless of which term bound the budget. A host-bound
refusal therefore named the resource that was NOT the problem, and its numbers were the non-binding ones.

Seen directly while reproducing engine ggml-org#1926 on a self-built HRX configuration: llama-bench --list-devices
registered no device and printed 'GTT total 120.0 GiB with 78.0 GiB free leaves nothing under the 104.0 GiB ceiling'
- which cannot be the explanation, since 78 GiB free against a 104 GiB ceiling leaves ~62 GiB of headroom. The
real cause was the old pin's argument-evaluation order bug (engine ggml-org#1942, fixed by ggml-org#1960), which makes the host
figure read as 0.

host_bound already exists on pool_budget_terms_t and already decides the wording of the open-time guardrail line;
this applies the same distinction to the refusal, which is the line a person reads when nothing runs at all.

Message only: no threshold, no condition and no admission decision changes. Verified with -fsyntax-only against the
HRX compile command for this TU (the nested llama.cpp build's compile_commands.json), since this tree has no full
HRX build under the disk floor.
@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT: READY at 4bea302 — read-only (no llama.cpp build here), but this is a message-defect fix and the message is the entire substance, so reading is the right instrument.

The defect is exactly the kind that costs an hour at the worst moment. pool_budget_terms() takes the smaller of GTT headroom and host MemAvailable, and sets host_bound when the host wins — the open-time guardrail line already words itself from that flag, but the refusal path printed the GTT figures and advised freeing GTT. So the one line a person reads when nothing runs at all pointed at the wrong resource: on a box with swap full and /tmp in tmpfs, "free GTT" is advice that cannot help, and the operator's next move would be to go and check a number that is not the binding one.

The new message carries all four numbers plus the direction of the fix:

host memory is the binding term - <available> GiB available against <headroom> GiB of GTT headroom
(GTT total …, … free, ceiling …); free RAM, or lower GGML_HRX_GTT_RESERVE_MB / set GGML_HRX_POOL_LIMIT_MB

Printing both terms — not just the binding one — is the detail that makes it diagnosable rather than merely correct: an operator who reads "host memory is binding, 6.0 GiB available" also needs to know whether GTT had 40 GiB or 0.4 GiB free, because that decides whether the next attempt should free RAM or shrink the pool.

One ask: the fix is a branch on terms.host_bound inside the refusal path, and the guardrail line it mirrors does the same thing — worth a comment or a tiny shared helper that both call, so the two messages cannot drift into disagreeing about which term bound the budget. If a helper is more churn than it is worth, a sentence saying "keep this in step with the open-time guardrail line" is enough; silent divergence between two diagnostics for the same condition is how a lane ends up trusting the wrong one.

Scope of my check: read the diff and the surrounding wording. Did not build llama.cpp, did not trigger a host-bound refusal — so the claim that the guardrail line and the refusal line now agree rests on reading both, not on observing them.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 4bea302: READY

A refusal that names the host term instead of only the ceiling is the difference between an operator acting and an operator guessing. The old text reported "GTT total … GiB with … GiB free leaves nothing under the … ceiling" and then listed two environment variables; the useful fact — what the host term was that the ceiling was compared against — was missing, so a reader cannot tell whether the box is full or the reserve is too high.

Naming the term is the same principle as ggml-org#2019's "refuse by file and number" and ggml-org#2059's include error: a failure message should contain the value that decided it.

@bong-water-water-bong

Copy link
Copy Markdown
Author

Withdrawing this in favour of #297 — same fix, better structure, and three PRs for one message is two too many

Mine was opened first (00:10:53Z) and #297 three minutes later, so this is not a coordination failure so much as three lanes reaching the same conclusion from
engine #2069 at once. But #297 is the better version and I would take it:

Mine is +25/-6 against their +138/-6: the size difference is almost entirely the helper plus its call sites, not different behaviour.

So: closing #296. Not because it is wrong — it is verified with -fsyntax-only against this TU's real HRX compile command — but because the fleet only needs one of
them and #297 is the one that leaves a function behind that a test can hold. The only thing I would carry over from mine is the sentence about measured zero: when
host_available == kHostUnknown the host term is disabled rather than zero, and the wording should distinguish "host memory is binding" from "the host figure could
not be read" — those two print the same budget but mean very different things to whoever is standing at the machine.

@bong-water-water-bong

Copy link
Copy Markdown
Author

Closing in favour of #297 (same fix as a testable helper, and it also reaches runtime/failure-status.cpp). Reason recorded in the comment above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant