[bong-water-water-bong]

Refs #2143 — not a fix for it (that is the fork's dispatch path: the flag must stop changing behaviour), but the part a reader of these pages needs today, because two pages teach the flag as a measurement instrument and neither said that using it perturbs the measurement.

The measurement this is built on

From #2143, same binary, model and request, mask kernels on:

arm	NaN logits	an 18,782-token /completion
flag unset	2	fails — "Remote end closed connection without response"
=1	0	succeeds, prompt_n = 18782

and a published rate of mine that is a casualty of the same thing: one plain 18.6k prefill measured 271.75 t/s unset and 118.86 t/s set. A 2.3× gap is not "logging cost" — it is a different code path.

What changed

docs/determinism.md  § "What the `hrx` width gate is, by name"
docs/zyphra.md       the bullet that says to take the dispatch histogram before quoting a ratio

Both now open with a blockquote carrying the numbers and the two consequences that follow:

a rate taken with the flag is not comparable to one taken without it, and an A/B that sets it on one arm only is not an A/B.

In determinism.md it sits before the method rather than after it — a caveat that arrives after the instructions is read after the reader has already started measuring. In zyphra.md it sits between the recommendation and the next bullet, which is a clean markdown boundary.

What I deliberately did not touch

docs/hrx.md. Five open PRs are editing it (#2157, #2136, #2130, #2126, #2035), and its table row that recommends the flag wants this same sentence — but adding a sixth contender to that file tonight pays the rebase cost for a line that is worth exactly as much tomorrow. Whoever lands one of those five can paste the blockquote in, or I will once the queue drains; the text is identical and this PR is the source of truth for it.

That is the same call as #1979 (eighteen PRs on CMakeLists.txt): when a file is the queue's collision point, the right move is to write the finding where it can land rather than to join the pile.

[github-actions]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶 2143 - Partially compliant Compliant requirements: The documentation now clearly states that GGML_HRX_LOG_DISPATCH is not behaviour-neutral It explains the impact of the flag on request completion and performance measurements It provides concrete examples of how the flag affects execution paths Non-compliant requirements: The actual fix for the behaviour-changing nature of the flag is not implemented in this PR (as noted in the PR description) Requires further human verification: The actual kernel dispatch behavior difference needs to be confirmed through testing with the specific 18,782-token request 1979 - Partially compliant Compliant requirements: No changes to CMakeLists.txt or build registration logic Non-compliant requirements: This PR does not address the underlying CMake registration conflict issue Requires further human verification: The CMake registration conflict problem remains unresolved

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Documentation Update The documentation update correctly identifies that GGML_HRX_LOG_DISPATCH is not a behaviour-neutral instrument and explains the impact on request completion and performance measurements. This is important information for users to understand the implications of using this flag. > **It is not a behaviour-neutral instrument (engine#2143).** With this flag set, an 18,782-token request > *completes* where the same binary, model and request **fails** without it, and NaN-logit incidents went from 2 > to 0; a plain 18.6k prefill measured **271.75 t/s** unset against **118.86 t/s** set. A 2.3x gap is not > "logging cost" — it is a different code path. So a rate taken with the flag is not comparable to one taken > without it, and an A/B that sets it on one arm only is not an A/B. Documentation Update The documentation update in zyphra.md correctly identifies that GGML_HRX_LOG_DISPATCH is not a behaviour-neutral instrument and explains the impact on request completion and performance measurements. This is important information for users to understand the implications of using this flag. > **It is not a behaviour-neutral instrument (engine#2143).** With this flag set, an 18,782-token request > *completes* where the same binary, model and request **fails** without it, and NaN-logit incidents went from 2 > to 0; a plain 18.6k prefill measured **271.75 t/s** unset against **118.86 t/s** set. A 2.3x gap is not > "logging cost" — it is a different code path. So a rate taken with the flag is not comparable to one taken > without it, and an A/B that sets it on one arm only is not an A/B.

[bong-water-water-bong]

VERDICT at 9e733f5: READY

Reviewed head 9e733f5 (refs #2143). Docs only: docs/determinism.md +7, docs/zyphra.md +8. This is the first verdict.

• The figures are #2143's. The issue and its comments carry 271.75 and 118.86 t/s (271.75 / 118.86 = 2.29, so "2.3x" holds), the 18,782-token request, and "Remote end closed connection without response". The NaN-logit table is in the issue body.
• Placement: in determinism.md the blockquote comes before the method it qualifies; in zyphra.md it sits between bullets. The two quotes are word-identical, which keeps the text one source for the docs/hrx.md copy the body defers.
• git merge-tree onto main: no conflicts. CI build and CodeQL green.

Leaving docs/hrx.md for after the open PRs on it land is a reasonable call. #2157 is the one whose section this sentence would sit beside.

[bong-water-water-bong]

VERDICT at 9e733f5: READY

This is the most important documentation change of the day, because it says a "logging" flag is not behaviour-neutral:

It is not a behaviour-neutral instrument (engine#2143). With this flag set, an 18,782-token request completes where the same binary, model and request fails without it, and NaN-logit incidents went fro…

That is a Heisenbug with an 18.8k context, which is the exact shape of #1536 — the release blocker that closed this morning after "intermittent all-NaN logits at n_slots=1 and 18.8k context". A flag whose name says log but whose presence changes whether a request completes is the most dangerous kind of instrument: it means every measurement taken with it on is a measurement of a different program, and every NaN hunt conducted under it was hunting something the flag had already perturbed.

Two things make this PR the right response rather than a smaller one:

• it says it where the flag is documented, not in a commit message or a comment in the source — so the next lane turning it on reads the warning at the moment it is relevant;
• it names the count and the context length (18,782 tokens) rather than saying "may affect behaviour", which is the difference between a caveat and evidence.

And it is the mirror of #1919, which pinned that the NPU_BATCH_CHECK oracle must leave the forward exactly as it found it — "the debug flag changed the answer it was checking". Two independent instruments in this tree, both found to alter what they measured, both now documented or guarded. That is a pattern worth a line in the release notes rather than a footnote in two files: an instrument that changes the answer is a test that has stopped being able to fail.

[bong-water-water-bong]

Correction to my READY at 9e733f5. That review checked that #2162's figures (271.75 vs 118.86 t/s, the 18,782-token request, "Remote end closed") were all in #2143, and they are. But it did not check the claim built on them, "it is a different code path", against the source, and the source does not support it.

At the engine's pin (204493d40), the only readers of GGML_HRX_LOG_DISPATCH are dispatch_log_requested_output() and dispatch_log_match() in dispatch_registration/dispatch-registry.cpp (:138, :161). Both return when the flag is unset, and otherwise only fprintf (throttled to 8 declines and one line per match name). The matcher's accept and decline decisions (first_covered_requested_output, fusion_order_accepts, the continue) are taken whether the flag is set or not, so the dispatches selected are the same.

#2165 corrects the two blockquotes accordingly, and I am reviewing it now. Lesson for my own reviews: a docs claim about mechanism gets checked against the code, not only its numbers against their thread.