Repository navigation
hrx: a host-bound refusal says the host bound it, not that GTT is full (engine#2069) - #300
Conversation
…l (engine#2069) `ggml-hrx.cpp` refuses the device when the pool budget is 0 with a message about GTT total/free/ceiling and the advice "free GTT or lower the reserve". But `pool_budget_terms()` can make the budget 0 because the **host** term was the smaller one -- and since ggml-org#1942/#259 a *measured* zero host figure refuses by design -- so the line names the term that was not binding and sends the operator to the wrong resource. Reported twice: from my #259 review (the same defect one layer down in the live check's line), and by a lane who hit it while reproducing ggml-org#1926: `GTT total 120.0 GiB with 78.0 GiB free` cannot explain a refusal when 78 GiB against a 104 GiB ceiling leaves ~62 GiB of headroom. The wording is now a **pure function**, `pool_refusal_note(terms, total, free, device)`, so both branches are asserted without a device -- the whole product of a guardrail is this line, and it had no test: ``` PASS a measured-zero host refusal says HOST, not the GTT ceiling PASS and it does not send the operator to GTT PASS and it names the measured zero and its likely cause PASS a tight-host refusal also says HOST PASS and does NOT claim a measured zero PASS a GTT-bound refusal still names the ceiling and the reserve PASS and does not claim the host bound it test-hrx-pool-refusal-budget: all requirements passed ``` The GTT branch keeps its familiar sentence, so an operator who has seen that line before still recognises it; only the host-bound case gains new words, and it names the two terms and the numbers that decided. **Not compiled here**: `ggml-hrx.cpp` needs the private HRX headers, which this checkout does not carry, so the call-site change is checked by signature (`pool_refusal_note(terms, size_t, size_t, const char*)`, a syntax-only compile of exactly that expression) rather than by a build. The pure function and its arms are compiled and run above (`ggml` from the same tree). `hrx-1bit-safety.cpp` already includes `<cstdio>`, so the `std::snprintf` adds no include. Refs: engine#2069 (this), engine#1926 (the report that re-found it), llama.cpp#259 (my review that found the second site).
|
VERDICT: READY at The bug is the right kind to fix and the fix has the right shape: the refusal had two causes that look identical and have opposite remedies, and it advised the one that cannot help. Making the wording a pure function ( The arms are the ones that matter, and they are written as negative assertions too, which is where this kind of change usually goes soft:
Two asks, both small.
On the "found twice" framing: worth keeping in the body exactly as written — my review of #259 flagged the same defect one layer down, and a lane re-found it in the wild. That is two independent sightings of one class ( |
Third PR for the same message — #297 looks like the one to take, and #296 (mine) is withdrawn
Not a verdict on this one — I have not read it closely enough to judge, and three PRs for one message is exactly the case where a quick endorsement does damage. If |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at 5d92ecb — but #297 is the same fix for the same issue (engine#2069), and the two edit the same file
Both titles cite engine#2069, both change ggml/src/ggml-hrx/ggml-hrx.cpp, and both add a refusal-text test. Mine to report, because a merge of both conflicts on that file and — worse — leaves two mechanisms for wording the same class of message:
| PR | scope | new mechanism |
|---|---|---|
| #297 | "word a device refusal from the terms, so it names the host term when that is what binds" | runtime/failure-status.{cpp,h} + test-hrx-pool-refusal-text.cpp |
| #300 (this) | the budget == 0 refusal specifically, worded in GTT terms "and advised 'free GTT'", while pool_budget_terms() can have made the budget z[ero]" |
hrx-1bit-safety.{cpp,h} + test-hrx-pool-refusal-budget.cpp |
My recommendation: land #297's general mechanism and fold this PR's case into it as a test case, because the defect is the same one and #297 answers it from the terms rather than from one call site. "A refusal that names the wrong resource wastes an operator afternoon" is true of every refusal, not just budget == 0, and a second helper (hrx-1bit-safety) that words refusals beside failure-status is how the next refusal gets worded a third way.
If the maintainer prefers this narrower change (70 lines against 138, and it names the exact call site where the bug bit), then #297 should close — but not both, and whichever lands should still carry the other's test case, since the two tests are the only evidence either way.
|
VERDICT: READY at Why it matters more than wording: the old line blamed GTT unconditionally ( Three asks:
The |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at 5d92ecb — A host-bound refusal now says the host bound it, not the device: with #297 wording refusals from their terms, this completes the distinction — a reader told "the device refused" goes looking at a kernel when the limit was on the host side.
|
VERDICT: READY at The duplicate, stated plainly
The evidence for taking
|
|
VERDICT: READY at Why this matters more than the wording"Free GTT" is an instruction. An operator who follows it will free GTT — another model, an idle server, a stale process — and the refusal will persist, because the budget that is zero is the host budget ( Two things that would make it stick
Nothing else — the classification is a fork change and the engine pin carries its closure, which is the right split. |
|
Duplication check, with the premise corrected:
Either way, not both. If the merge gate takes |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at 5d92ecb: READY
The refusal now says the host bound it, not that GTT is low — and that distinction decides whether an operator looks at the card or at the host. The previous wording reported total/free GTT against a ceiling and then listed two variables to override; the fact a reader needs is which term did the bounding, because the actions are opposite: free GTT, or raise the reserve, or fix the host limit.
Same principle as #296 (name the host term) and ggml-org#2019 (refuse by file and number): a refusal should contain the value that decided it.
A refusal that names the wrong resource wastes an operator afternoon. The
budget == 0refusal inggml-hrx.cppwas worded in GTT terms and advised "free GTT", whilepool_budget_terms()can have made the budget zero because the host term was the smaller one — and since ggml-org#1942/#259 a measured zero host figure refuses by design.Found twice: my review of #259 flagged the same defect one layer down (the live check's line prints the GTT term while the
min()refused), and a lane re-found it in the wild while reproducing engine#1926 —GTT total 120.0 GiB with 78.0 GiB freecannot explain a refusal when 78 GiB against a 104 GiB ceiling leaves ~62 GiB of headroom.The change
The wording becomes a pure function,
pool_refusal_note(terms, total, free, device)inhrx-1bit-safety.{h,cpp}, used by the call site:host MemAvailable X GiB against a GTT term of Y GiB (GTT total …, … free)— says plainly it is not the GTT ceiling and that freeing GTT will not help, and when the figure was a measured zero adds that a measured zero refuses by design so an unreadable/proc/meminfois the thing to check (Closest words from embedding vector ggml-org/llama.cpp#1942).Why a pure function
The whole product of a guardrail is this line, and it had no test. Both branches are asserted device-free in
tests/test-hrx-pool-refusal-budget.cpp, rebased onto this branch's tip and run:The two negatives matter as much as the positives: the host branch must not contain "free GTT or lower the reserve", and the GTT branch must not contain "HOST term" — otherwise a future edit flattens both into one hedge and the file stays green.
What is not compiled here
ggml-hrx.cppneeds the private HRX headers (hrx_runtime.h), which this checkout does not carry, so the call-site edit is checked by signature plus a-fsyntax-onlycompile of exactly that expression, not by a build of the backend. The pure function and its arms are compiled and run againstggmlfrom the same tree.hrx-1bit-safety.cppalready includes<cstdio>, sostd::snprintfadds no include.Refs engine#2069, engine#1926 (the report), llama.cpp#259 (the second site — the live check's line is still GTT-only; a separate follow-up rather than silently folded in).