Repository navigation
hrx: the alias fallback of the alternate-value lookup compares the scale (a miss in #298) - #312
Conversation
…ale (a miss in #298; engine ggml-org#2159) #298 gave an f16 activation copy a scale (stored = value * 2^-6) and made the lookup ask for a scale, so that a reader which does not know about scales only finds unscaled copies. The last step of the graph-aware lookup, which matches an alternate of another value with the same storage range, did not compare the scale. A reader asking for the unscaled copy through an alias could be handed the scaled one, and the other way round, and would then scale its result wrongly. Test: hrx-backend-test alias_value_import. Value 1 aliases value 0, value 0 has an f16 copy at 2^-6: the scale-6 lookup through the alias finds it, the default lookup does not. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
|
ready for review at e5406fe |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at e5406fe: READY
"The bug (mine, from #298)" is the most useful sentence in this PR. A defect introduced by an earlier change, found by bisecting a different issue, and labelled as the author's own rather than left for someone else to attribute — that is what makes a bisection result trustworthy rather than a hunt.
And the framing of the relationship to ggml-org#2159 is exactly right:
Refs engine#2159 (found while bisecting it; not shown to be its cause, see below)
Saying "found while bisecting X" without claiming causation is the discipline that keeps an issue's history clean — the opposite of the pattern where a plausible-looking bug near a symptom gets promoted to "the cause" and closes the wrong ticket. If this later turns out to be part of ggml-org#2159, the link is there; if not, nobody has been misled.
The fix itself — the alias fallback of the alternate-value lookup comparing the wrong thing — is the class of defect that only shows in an opt-in path with an alternate spelling, which is why #298's own tests passed.
Refs 1bit-MONSTER/engine#2159 (found while bisecting it; not shown to be its cause, see below).
The bug (mine, from #298)
#298 gave an f16 activation copy a scale (stored = value x 2^-6) and made the lookup take the scale, so a reader that does not ask for one only finds unscaled copies. The graph-aware lookup has a last step that returns an alternate of another value with the same storage range. That step compared type and size and not the scale. A reader asking for the unscaled copy through an alias could be handed the 2^-6 copy, or the reverse, and would then scale its result wrongly.
Change
One condition in
find_alternate_value(graph, plan, ...)(dispatch/command-plan.h), and a host case.Measured (strixhalo, full amdclang Release build, host cases only, no device used)
hrx-backend-test alias_value_import: value 1 aliases value 0, and value 0 has an f16 copy stored at 2^-6.Device cases were not run for this PR (the change is in the plan-time lookup; the 12 device cases of the binary need a ticket and I did not take one).
What this does not claim
engine#2159 (identical long prefills on flint giving different logits) reproduces without #298: 12 of 30 prefills deviate at
6ec7c26, the commit before it. So this bug is not that issue's cause, and I have not measured whether any model's plan takes the fallback step with a scaled copy today. It is fixed because the invariant #298 promised ("a reader that does not ask for a scale only finds unscaled values") did not hold on this path.🤖 Generated with Claude Code