[bong-water-water-bong]

Fixes #2143

What the issue says, and what is measured

#2143 reports that an 18.8k request failed with GGML_HRX_LOG_DISPATCH unset and succeeded with it set, one run each, and concludes that the flag changes behaviour. It does not, on the source and on the device. The one-against-one difference is the intermittent long-prompt fault that is tracked on its own (#1536 on the build without the f16 fixes, and #2159): a fault that shows in 3 of 3 or 12 of 30 runs will split a single pair by chance.

Source (the pinned fork): the flag is read in two places, dispatch_log_requested_output() and dispatch_log_match() in dispatch-registry.cpp. Both are static void, take their arguments by const reference, and only fprintf. Nothing that selects a registration or binds a value reads it.

Device (strixhalo, one gpu-hold ticket, 12:40Z, the CI HRX build at 204493d4, Qwen3-4B Q4_K_M, llama-server -c 16384 -np 1 --cache-reuse 256): 18 greedy requests per arm (6 x a 219-token prompt, 6 x a 9,300-token prompt, 6 x the same with cache_prompt true, 128 tokens each), the replies concatenated in order and hashed.

arm	ggml_hrx dispatch: lines in the server log	distinct replies per group of 6	sha256 of the 18 replies
flag unset	0	1, 1, 1	f7516b2f63deec75
GGML_HRX_LOG_DISPATCH=1	23	1, 1, 1	f7516b2f63deec75

The flag was live in the second arm (23 log lines) and the output is byte-identical.

What this PR adds

tests/hrx_log_dispatch_neutral_test.py, registered as ctest hrx_log_dispatch_neutral: it fails when the flag is read anywhere but those two functions, when a reader is not static void, or when a reader takes an argument it could write through. So the flag cannot quietly become a selection input later, which matters because measurements are taken with it set (the dispatch profile of #2084, the #1536 close test).

$ python3 tests/hrx_log_dispatch_neutral_test.py
OK: GGML_HRX_LOG_DISPATCH is read 2 times in 1157 source files, both in static void log functions of const arguments

Its self-test (always run, also when the submodule is not checked out) covers: the two readers as they are; a comment naming the flag; a reader in another file; a reader that returns a value; a reader with a writable argument; a reader in another function. It uses a check() helper, not assert.

Not covered

Timing. Two fprintf calls (at most 8 + one per registration name per process) could shift a race by microseconds; a race that depends on that is a defect of the racing code (#2159 is looking for one), not of the flag.

🤖 Generated with Claude Code

[bong-water-water-bong]

ready for review at fb5dc73

[github-actions]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶 2143 - Partially compliant Compliant requirements: The GGML_HRX_LOG_DISPATCH flag is verified to be read only in the two allowed static void functions The test ensures that the flag is not read anywhere else in the codebase The test verifies that the functions are properly declared as static void Non-compliant requirements: None Requires further human verification: None 1536 - Partially compliant Compliant requirements: The PR ensures that GGML_HRX_LOG_DISPATCH does not change behavior, which addresses the root cause of the intermittent NaN issue Non-compliant requirements: None Requires further human verification: The actual behavior of the system with long-context requests and NaN logits should be verified through testing on the hardware 2159 - Partially compliant Compliant requirements: The PR ensures that GGML_HRX_LOG_DISPATCH does not change behavior, which addresses the root cause of the intermittent NaN issue Non-compliant requirements: None Requires further human verification: The actual behavior of the system with routed-expert models and sliding-window attention should be verified through testing on the hardware

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Test Coverage The test ensures that GGML_HRX_LOG_DISPATCH is only read in the two designated static void functions. However, the test relies on parsing source code to verify function signatures, which could be brittle if the code formatting changes or if there are edge cases in the regex patterns used for parsing. FLAG = "GGML_HRX_LOG_DISPATCH" SOURCE_DIRS = ("ggml/src", "src", "common", "tools") ALLOWED_FILE = "ggml/src/ggml-hrx/dispatch_registration/dispatch-registry.cpp" ALLOWED_FUNCTIONS = ("dispatch_log_requested_output", "dispatch_log_match") READ = re.compile(r'getenv\s*\(\s*"%s"\s*\)' % FLAG) FUNCTION = re.compile(r"^(static\s+)?([\w:<>\s\*&]+?)\s+(\w+)\s*\(([^;{]*)\)\s*\{", re.M) def enclosing_function(text, offset): """(name, return type with qualifiers, parameter text) of the function definition that contains offset.""" best = None for match in FUNCTION.finditer(text): if match.start() > offset: break best = match if best is None: return None return best.group(3), ((best.group(1) or "") + best.group(2)).split(), best.group(4) def check_file(path, text): """Problems for one source file: a list of strings.""" problems = [] for read in READ.finditer(text): line = text.count("\n", 0, read.start()) + 1 if path != ALLOWED_FILE: problems.append("%s:%d reads %s outside the dispatch log" % (path, line, FLAG)) continue found = enclosing_function(text, read.start()) if found is None or found[0] not in ALLOWED_FUNCTIONS: problems.append("%s:%d reads %s in %s, which is not a dispatch log function" % (path, line, FLAG, found[0] if found else "no function")) continue name, kind, parameters = found if kind != ["static", "void"]: problems.append("%s:%d %s is '%s', not 'static void': a log function must not return a decision" % (path, line, name, " ".join(kind))) for parameter in [p.strip() for p in parameters.split(",") if p.strip()]: by_value = "&" not in parameter and "*" not in parameter if not by_value and not parameter.startswith("const "): problems.append("%s:%d %s takes '%s', which it could write through" % (path, line, name, parameter)) return problems

[bong-water-water-bong]

VERDICT at fb5dc73: READY

This closes #2143 with a property rather than a measurement, and the reasoning is the review: "Measurements are taken with this flag set (the dispatch profile, the #1536 close test), so it must not be able to change what the backend does."

The test proves it structurally — in the pinned fork the flag is read in exactly two places, both static void taking const references and returning nothing, so they can print and cannot change which kernel is chosen. Asserting the call sites' shape is stronger than asserting one observed run: a future edit that gave either function a return value or a mutable argument would fail the test even if its behaviour happened to be unchanged that day.

That is the same idea as #1919 (the NPU_BATCH_CHECK oracle must leave the forward exactly as it found it) but applied to a flag that was found to be non-neutral by #2162's 18,782-token evidence. So the sequence is complete and in the right order: the finding was documented where the flag lives, the fork was fixed, and now the fix is pinned. A flag used for the dispatch profile and the #1536 close test is load-bearing for every measurement this fleet takes, which is why it deserves a permanent guard rather than a note.