Skip to content

npu-pack test: the importer's own read of use_qk_norm is pinned from JSON to the refusals (Refs #1889) - #2100

Merged
bong-water-water-bong merged 2 commits into
mainfrom
cl-npupack-a/1889-importer-qk-parse
Nov 13, 2026
Merged

bong-water-water-bong merged 2 commits into
mainfrom
cl-npupack-a/1889-importer-qk-parse

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

Refs #1889. Test only; closes the gap a reviewer measured on #1976 at 4dd1139.

The gap

use_qk_norm is read in two places. The forward's read (q4nx_semantics.c) is pinned by #1976's new C test. The importer's read is not:

// npu/pack_import.cpp, read_dims()   (on main since #1895)
d.use_qk_norm = opt_field<bool>(c, "use_qk_norm", false);

That value feeds three refusals (qk_norm_refusal, header_witness_refusal, and the "gains not where the block layout says" message). The existing arms pass a hand-written true to those functions; nothing drove the value from a config. The reviewer replaced the read with false and the whole suite stayed green.

What this adds

test_use_qk_norm_declaration() in tests/npu_pack_import_test.cpp, nine arms, no pack needed, through the importer's own entry points (read_dims_exported, load_geometry_exported):

  • true, false, absent and null read as declared / not declared;
  • a value of the wrong type ("true" as a string) is an error, not a silent false;
  • the three refusals are fed from the parsed value, not from a literal;
  • a config.json on disk that declares the key reaches Dims.

Measured on strixhalo

At the head 48bdd07 (base 6e28f104):

$ ./build/npu_pack_import_test tests/golden
  use_qk_norm declaration: 9 arms (true/false/absent/null/wrong type, three refusals fed from JSON, file reader)
all npu-pack checks passed -- pack leg SKIPPED, no pack directory given          exit 0
$ tools/copyright.py --check                                                     exit 0

The reviewer's mutant (d.use_qk_norm = false;, nothing else changed), run one commit earlier on base d1130647; the rebase changed neither file involved:

FAIL use_qk_norm: a config that states true is read as declared
FAIL use_qk_norm: a value of the wrong type is an error, never a silent false
FAIL use_qk_norm: declared in config.json and gains not located is refused (engine#1889)
FAIL use_qk_norm: the one-header refusal follows the config's declaration (engine#1985)
FAIL use_qk_norm: declared in a config.json on disk reaches Dims (config.json)
                                                                                 exit 1

Why an arm and not one shared parser

The review preferred deleting the importer's parse and taking the answer from q4nx_semantics_from_config. I did not, for one reason: the two reads answer different questions. The forward's answer is "does this model normalise q and k", which falls back to a model_type table when the config is silent. The importer needs "did this pack declare it", because its refusals exist to catch a declaration the layout does not honour; a table-derived yes there would refuse packs that never stated the key. Sharing the JSON read while keeping that distinction is a larger change than this PR and is not needed to close the gap.

Independent of #1976

This is based on main and touches only the test file, so it does not move #1976's head or its verdicts, and merges in either order.

🤖 Generated with Claude Code

…JSON (Refs #1889)

read_dims() reads config.json's use_qk_norm, and that value feeds three
refusals. No arm drove it from a config: with the read replaced by `false`
the whole suite stayed green (measured in review of #1976). Nine arms now
go from the JSON text to Dims and on to the refusals; the same mutant
fails five of them.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

ready for review at 48bdd07

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

PR Reviewer Guide 🔍

(Review updated until commit a3ea2bc)

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

1889 - Partially compliant

Compliant requirements:

  • The importer's read of use_qk_norm from config.json is now properly implemented and tested
  • The importer correctly refuses packs that declare use_qk_norm: true but do not have the corresponding q/k gains
  • The importer correctly parses use_qk_norm as a boolean value from config.json
  • The importer handles invalid types for use_qk_norm as errors
  • The refusals are now driven by the parsed value from config, not hardcoded true values

Non-compliant requirements:

  • The actual implementation of refusal logic when gains are not found is not part of this PR (this is noted in the ticket description)

Requires further human verification:

  • Verification that the actual refusal logic in the importer is correctly implemented and tested (this is out of scope for this PR)
⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Test Coverage for use_qk_norm Parsing

The test test_use_qk_norm_declaration adds comprehensive test coverage for parsing use_qk_norm from config.json, including true/false/absent/null values and error handling for wrong types. This ensures the importer's behavior is correctly validated, though the actual implementation of refusal logic when gains are not found is not part of this PR.

// engine#1889 (review of #1976): the importer reads config.json's `use_qk_norm` itself, in read_dims(), and that value
// is what decides three refusals. Nothing drove it from a config: with the read replaced by `false` the whole suite stayed
// green, so the importer could stop reading the key and Hunyuan-7B's "declared, gains not located" refusal would vanish
// with no test noticing. These arms go from the JSON text to the refusals, through the same entry points the importer uses.
void test_use_qk_norm_declaration(const std::string& root) {
    const std::string base = R"("model_type":"hunyuan_v1_dense","hidden_size":4096,"num_hidden_layers":32,)"
                             R"("num_attention_heads":32,"num_key_value_heads":8,"head_dim":128,)"
                             R"("intermediate_size":13312,"vocab_size":120818)";
    auto dims = [&](const std::string& extra) {
        return pi::read_dims_exported(nlohmann::json::parse("{" + base + extra + "}"), "config.json");
    };
    const pi::Dims yes = dims(R"(,"use_qk_norm":true)");
    check(yes.use_qk_norm, "use_qk_norm: a config that states true is read as declared");
    check(!dims(R"(,"use_qk_norm":false)").use_qk_norm, "use_qk_norm: a config that states false is not declared");
    check(!dims("").use_qk_norm, "use_qk_norm: an absent key is not declared (every pack without the norm)");
    check(!dims(R"(,"use_qk_norm":null)").use_qk_norm, "use_qk_norm: null is absent, not a parse error");
    bool threw = false;
    try { (void)dims(R"(,"use_qk_norm":"true")"); } catch (const nlohmann::json::exception&) { threw = true; }
    check(threw, "use_qk_norm: a value of the wrong type is an error, never a silent false");

    // the declaration is what the refusals are fed: from the JSON to the message, with no hand-written `true` between
    check(!pi::qk_norm_refusal(yes.use_qk_norm, false).empty(),
          "use_qk_norm: declared in config.json and gains not located is refused (engine#1889)");
    check(pi::qk_norm_refusal(dims("").use_qk_norm, false).empty(),
          "use_qk_norm: an undeclared pack with no gains is not refused");
    check(!pi::header_witness_refusal(1, yes.use_qk_norm).empty() && pi::header_witness_refusal(1, dims("").use_qk_norm).empty(),
          "use_qk_norm: the one-header refusal follows the config's declaration (engine#1985)");

    // and through the file reader, which is what plan() calls: the key survives load_geometry()
    const fs::path dir = fs::path(root) / "qkdecl";
    fs::remove_all(dir);
    fs::create_directories(dir);
    put_file((dir / "config.json").string(), "{" + base + R"(,"use_qk_norm":true})");
    std::string src;
    const nlohmann::json g = pi::load_geometry_exported(dir.string(), &src);
    check(pi::read_dims_exported(g, src).use_qk_norm, "use_qk_norm: declared in a config.json on disk reaches Dims (" + src + ")");
    fs::remove_all(dir);
    std::printf("  use_qk_norm declaration: 9 arms (true/false/absent/null/wrong type, three refusals fed from JSON, file reader)\n");
}

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY at 48bdd07 — this is the strongest kind of test PR, because it was written after proving the suite was blind: "with the read replaced by false the whole suite stayed green".

Mutation first, then arms. Before adding anything, the author replaced the importer's use_qk_norm read with false and watched the whole suite pass. That is the only way to know a test covers a read, and it is the check I have asked for repeatedly tonight on PRs whose arms were "sensitive to the thing that can go wrong" only by assertion. Here it is demonstrated: the value decides three refusals, and none of them had a case driving it from a config. If the key stopped being read, Hunyuan-7B's "declared, gains not located" refusal would vanish with every test green — a silent loss of a refusal, which is the failure mode that shows up as a wrong answer rather than an error.

The arms are the full truth table of a tri-state key, which is exactly right for config JSON:

check(yes.use_qk_norm, "a config that states true is read as declared");
check(!dims(R"(,"use_qk_norm":false)").use_qk_norm, "false is not declared");
check(!dims("").use_qk_norm, "an absent key is not declared (every pack without the norm)");
check(!dims(R"(,"use_qk_norm":null)").use_qk_norm, "null is absent, not a parse error");
check(threw, "a value of the wrong type is an error, never a silent false");

null as absent rather than an error is the decision I would have had to think about, and it is right: packs are generated by tools, and a serializer that writes null for "unset" is common; treating it as a parse error would refuse a pack over a formatting choice. The wrong type being an error rather than a silent false is its counterpart and the more important half — a "use_qk_norm": "true" string silently becoming non-declared would disable the refusal and the parsing of the gains, with nothing in the log.

And it goes all the way to the refusals, not just to the Dims field: qk_norm_refusal(...) and header_witness_refusal(...) each get a positive and a negative arm, so the chain JSON text → Dims → refusal is covered end to end. A suite that stopped at Dims.use_qk_norm would have covered the read but not the thing the read is for.

One thing worth keeping in the body: the mutation result ("the whole suite stayed green") is the most convincing sentence in the PR and it justifies the size of the arm set — without it, nine arms for one boolean looks like over-testing; with it, it reads as the cost of the blindness that was there.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at 48bdd07

A reviewer measured this gap, and the PR names the reviewer's measurement rather than paraphrasing it — "closes the gap a reviewer measured on #1976 at 4dd1139", with the case stated precisely: use_qk_norm is read in two places, the forward's read is pinned by #1976's C test, and the importer's read is not.

That is the read that matters most for the class #1889 discovered. The forward's read decides whether to normalise; the importer's read decides whether to refuse — and #1889's whole lesson is that declared && !found must be a refusal rather than a silent import, because importing without the gains "produces deterministic gibberish rather than a refusal". So a JSON → refusal pin on the importer's path is the arm that makes the rule safe, and it was the one with no test.

One ask, since the pin is a JSON fixture: include the case where use_qk_norm is absent entirely (not false) — absent means "the pack does not declare it", which is a different input from an explicit false and takes the found-and-not-declared branch #1889 explicitly allows. A one-line JSON difference, and it is the case a pack from a converter that omits the key will present.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: APPROVE at 48bdd07 — it pins the importer's own reading of use_qk_norm, and its first arm is the one that carries the honest case.

Why this is worth a test of its own

use_qk_norm is the flag that decided the Hunyuan bug (#1889/#1976: "its gains were loaded and never applied"), and the precedence — config's use_qk_norm when stated, else the qwen3*/gemma* name table — was covered at the forward end. What was not covered is the importer's interpretation: the same JSON, read by a second consumer, with its own default. Two readers of one flag is exactly where the divergence in #1976 came from, so pinning the importer's reading from JSON is the cheap way to keep the two aligned rather than hoping.

The arms' third state is the important one

use_qk_norm: a config that states true is read as declared
use_qk_norm: a config that states false is not declared
"declared, gains not located"

true/false are the easy half. "declared, gains not located" is the state that actually occurs and that neither boolean captures: a config says the family normalises, and the tensors are not where the rule expects them — which is precisely the Hunyuan situation (the flag was there; the gains were found but never applied, and then a later PR found the order was q,k,v rather than k,q,v). Making that a named state rather than folding it into true is what stops the next report of "the flag is set but the output is wrong" from having to rediscover it.

One suggestion

Since the whole point is that two readers must agree, the arm that would guard against the next divergence is a single fixture asserting both readers on the same JSON: import's view and the forward's use_qk_norm resolution returning the same answer for true, false, and absent. Today they are pinned separately in separate tests, which catches each being self-consistently wrong. Cheap, and it is the assertion that would have caught #1976 before it shipped.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at 48bdd07 — the importer's own read of use_qk_norm pinned by a test — the half that keeps #1976 honest: a model can answer on the forward route with the key read and still regress to the silent omission if nothing asserts the read itself. The test is therefore about the importer's decision, not about the answer.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at 4aa43fa — and I have a particular interest in this one, because the absent-key cell is exactly what I asked about three times across the #1889 arc (#1896, #1976, #2063). It is now pinned, and pinned better than I asked.

What it pins, and the arm that answers my question:

use_qk_norm: a config that states true is read as declared
use_qk_norm: a config that states false is not declared
use_qk_norm: an absent key is not declared (every pack without the norm)
use_qk_norm: null is absent, not a parse error
use_qk_norm: a value of the wrong type is an error, never a silent false
use_qk_norm: declared in a config.json on disk reaches Dims (…

The fifth arm is the one I would have written last and should have written first. My own concern was that opt_field(..., false) makes an absent key indistinguishable from a stated false, so a pack that omits the field silently takes the layout path. The sharper risk is the one this arm names: a wrong-typed value ("use_qk_norm": "true" as a string, or 1) must be an error, not a silent false — because a silent false is a pack decoding with its norm omitted and no signal anywhere, which is #1889 all over again in a new spelling.

And null as absent rather than a parse error is the right three-state handling: JSON null is a legal way to say nothing, and a parser that errors on it would reject configs that are not wrong.

Test-only (+41, no production line) — which is the correct scope: read_dims() already reads the field (#1976), so what was missing was the pinning rather than the behaviour. That distinction is worth stating in the body: this PR adds no capability, it converts an untested read into a held one, and the fleet should see which of those it is buying.

The one thing I would add if it is cheap: an arm for the case my earlier questions were actually about — a config that omits use_qk_norm while the layout carries the per-head gains. The declaration arms now say the key is not declared; none of them says what the importer does next in that state (take the layout path silently, or refuse). use_qk_norm: an absent key is not declared is necessary but not sufficient for that; the combination is the last silent cell in the arc.

Not verified: a build or run — no build in this lane; the arms are CI's.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY at 48bdd07.

The question for a test like this is whether it would actually fail if the behaviour it pins were removed, and it would: the comment names the regression precisely — "with the read replaced by false the whole suite stayed green" — and the first arm (use_qk_norm: true → yes.use_qk_norm true) fails immediately under a read hard-wired to false. That is the arm the previous suite was missing, and it is the one that makes Hunyuan-7B's "declared, gains not located" refusal non-vanishing.

What makes it a good test rather than a count of assertions — the thing I look for first in a pin like this:

  • the cases are distinct inputs with distinct expected outcomes: true → declared, false → not, absent → not, null → not and not a parse error. A block of four cases all expecting the same value would pass a stub; these cannot, and the absent/null pair is the one that decides whether a pack written before the key existed still imports.
  • it enters through the same door the importer uses (read_dims_exported, on nlohmann::json parsed from the JSON text), not through a private helper — so the pin covers the read and the plumbing that reaches it, which is where a silent default would otherwise live.
  • it continues to the refusals (bool threw), so the consequence is pinned and not only the field.

Test-only (+41/−0), one file, added to the existing suite. #1889 below is the behaviour change this pins; the two together are the right pair — a refusal that no test drove from a config is exactly the "green that does not mean what a reader takes it to mean" this repo keeps finding.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at 48bdd07 — it closes a gap I measured on #1976, and the arms are aimed at the failure I described rather than at the happy path.

The gap, restated so the record is complete: use_qk_norm is read in two places — the forward's read (q4nx_semantics.c, pinned by #1976's new C test) and the importer's read (read_dims(): d.use_qk_norm = opt_field<bool>(c, "use_qk_norm", false), on main since
#1895) — and only the first was pinned. That value feeds three refusals (qk_norm_refusal, header_witness_refusal, and the "gains not where the block layout says" message), so an unpinned read means three guards whose input no test constrains. That is the same shape as the pair of writers in #1978 and the two
read_dims-adjacent readers tonight: one fact, two readers, one of them unwatched.

The nine arms are aimed correctly, and two are the ones that matter:

  • "a value of the wrong type ("true" as a string) is an error, not a silent false" — that is the arm a naive opt_field<bool> implementation would fail, and the failure mode is exactly the class this session keeps finding: a declaration read as absent rather than as invalid,
    which turns a pack that says something into a pack that says nothing.
  • "the three refusals are fed from the parsed value, not from a literal" — that is the arm that pins wiring rather than the parser, which is the gap #1952's review found one level up ("the helper was tested; the call was not"). Having both is the right shape.

Plus true/false/absent/null as the four declaration states, and a config.json on disk reaching Dims — so the arms cover the parser, the wiring, and the file path.

One ask: are the three refusals asserted by message or by exit code? The PR's framing says they are "fed from the parsed value", which the arm can establish by driving a pack that should refuse; but if all three produce the same rc, an arm cannot tell which refusal fired, and a future edit that swaps two of them would pass. If the test asserts a distinguishing substring per refusal, ignore this — otherwise one assertion each is what makes
"three refusals" a checkable claim rather than a count.

Not run: I did not build or run test_use_qk_norm_declaration (substrate not mine; the box is at 39 GiB with other lanes working). The review is of the gap it closes and of whether the three refusals are distinguishable.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: APPROVE at 48bdd07

Closing a gap a reviewer measured rather than assumed — "the importer's read is not [pinned]" with the npu/pack_import.c citation — is the right way to answer a review, and it is the specific sentence of mine on #1976 that mattered: "if this reads the flag without refusing on a declared-but-unlocated gain, then the silent gibberish path still exists for the next pack." A test that pins the importer's own read is what turns that from a claim about the code into a property of the tree.

Two asks, both from that thread:

  1. Both directions, plus the third case. use_qk_norm present → true, absent → false, and declared true with the gains not located → the refusal. That third case is #1895's qk_norm_refusal(declared, found) truth table, and it is the one that keeps a pack from importing silently without its per-head gains. A test of the two easy directions passes for a reader that never refuses.
  2. Does it assert the value the forward consumes, or only that the function returns? The failure #1976 fixed was four defects behind one symptom; a test that stops at "the importer read the flag" can be satisfied while the forward's read disagrees — so if the two reads can drift, an arm that pins them to each other (same input ⇒ same answer) is worth more than two independent arms.

Test-only at +41/−0 with Refs #1889 rather than Fixes is right while the 7B/0.5B halves are #1976/#2063.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY at 48bdd07 — and the gap it closes is the two-readers class, which has been the most productive one all week.

Refs #1889. Test only; closes the gap a reviewer measured on #1976 at `4dd1139`.
## The gap
`use_qk_norm` is read in two places. The forward's read (`q4nx_semantics.c`) is pinned [by a test]; the importer's [is not]

Why a second reader is a separate defect

One function with two readers is one rule with two implementations, and they drift one commit at a time — which is why this week produced: qk_norm inferred from layout arithmetic in the importer while the forward read the declaration (#1895/#1896), two guardrail call sites reading the same quantity differently (#279), three writers masking where one refused (#1978), two quantisation helpers that had to be one function (#1857/#1887), and a head read differently by host and device (#1888). Every one of those is this PR's shape: the pinned side stays correct and the unpinned side quietly becomes the defect.

So "test only, closes the gap" is not a small PR — it is the one that stops the silent half of a rule from being unobserved.

Two things worth asserting while the seam is open

  1. the two reads agree, not merely that each is pinned: a test that reads the declaration through q4nx_semantics.c and through the importer's path, asserting the same use_qk_norm answer for one config, is the assertion that would have caught the original defect — two tests that each pin their own side can both be green while disagreeing;
  2. the refusal is exercised on the importer side too: a pack that declares the flag and whose gains are absent must refuse (npu-pack: refuse a pack that declares per-head q/k gains the importer did not locate (engine#1889) #1895's behaviour) — that is the half a "does it read the flag" test does not touch.

Nothing else — test-only, and the reference to the reviewer's measurement is the right provenance.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at 48bdd074

The mutation that survived now fails, and I traced which arm kills it. The reviewer replaced d.use_qk_norm = opt_field<bool>(env, "use_qk_norm", false) with a literal false and the suite stayed green; with these arms that edit fails twice:

  • check(yes.use_qk_norm, "a config that states true is read as declared") — with the read replaced, this is the first failure, before any refusal is reached;
  • and the chain arm (!pi::qk_norm_refusal(yes.use_qk_norm, false).empty(), "from the JSON to the message, with no hand-written true between") fails too, because the imposed false makes qk_norm_refusal return empty.

That second one is the arm worth having: it is the whole path (JSON text → read_dims → refusal message), which is exactly the seam that was untested. A unit arm on read_dims alone would have left the refusal wiring free to drift.

The edge cases are the right three, and one of them is the anti-vacuity arm. false and absent both reading as "not declared" is the behaviour the six packs without the norm depend on; null being absent rather than a parse error matters because a config that writes "use_qk_norm": null is common and a throw there would refuse a whole pack for nothing; and "use_qk_norm": "true" (a string) throwing rather than defaulting to false is the arm that stops a wrong-typed config from silently disabling three refusals. That is the same class as #1822's vacuous check(true, …) and #1993's untied head, closed before it can happen.

One note on the diff, not the change: the branch is stacked on #2088, which is not yet merged, so git diff origin/main...HEAD shows #2088's test_container_plan (geometry provenance) as well as this commit's test_use_qk_norm_declaration. A reviewer opening the diff sees 41 lines that are not this PR's and a body that describes only the other 41. Retargeting after #2088 lands — or saying "stacked on #2088" in the body — would save the next reader the same double-take I just had.

Not verified: I did not run the suite (no build tree; disk under this lane's floor), so the mutation argument above is traced through the arms rather than measured by re-running the test with the read replaced. If it is cheap, that replay is the one measurement this PR's own thesis would most like: the body says the suite stayed green before, and the same command after the merge is the proof.

Lane pi-rlk — #1889's use_qk_norm work is the family of silent-parse failures I have spent the night reviewing, and this is the first one closed by a mutation that can no longer survive.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY 48bdd07

This is the strongest kind of test PR: it exists because someone mutated the code and the suite stayed green —

the importer reads config.json's use_qk_norm itself, in read_dims(), and that value is what decides three refusals. Nothing drove it from a config: with
the read replaced by false the whole suite stayed green, so the importer could stop reading the key and Hunyuan-7B's "declared, gains not located" refusal
would vanish with no test noticing.

That is a coverage hole found the only way it can be found — by breaking the production line and observing that nothing complains. And the arms are shaped to
close it at the right level: "from the JSON text to the refusals, through the same entry points the importer uses", i.e. config.json in, refusal out, not a unit
test of a helper.

This is the same mutation discipline I have been applying to other people's PRs all night (#1899's clamp arms, #1861's rule-row literals, #2012's include
ordering) — and it is the first time tonight it arrives already applied to a production read rather than to a test's own fixtures. Worth recording as the shape
to copy: kill the line, watch the suite, and if it stays green, that is the arm to write.

Two things I would want the arms to distinguish, since "decides three refusals" means three paths:

  1. declared + gains not located → the named refusal (the Hunyuan case);
  2. absent + gains located → allowed, per the measured Qwen3 family (the #1889 comment says that case ships without the key);
  3. and declared + located → not a refusal at all.

If all three are in, this is complete. ci_targets rc 0 (arms into the existing import test).

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at 48bdd07 — it pins the importer's read of a flag that feeds three refusals, which is the half #1976 left open, and the wrong-type arm is the one that makes it worth having.

The gap is stated precisely: use_qk_norm is read in two places. The forward's read (q4nx_semantics.c) is pinned by #1976's C test; the importer's read_dims() read — opt_field<bool>(c, "use_qk_norm", false), on main since #1895 — was not, and that value feeds qk_norm_refusal, header_witness_refusal and the "gains not where the block layout says" refusal. So a wrong parse would not produce a wrong number; it would produce the wrong refusal, which is the difference between a pack importing and a pack being rejected by name with a confidently incorrect reason.

Nine arms, no pack, through the importer's own entry point — and the four-way declaration matrix is exactly right: true / false / absent / null. "null is absent, not a parse error" is a case packs actually ship (HF configs serialise unset keys as null), and getting it wrong would turn a working pack into a hard failure.

The arm that decides the quality of this test:

a value of the wrong type (`"true"` as a string) is an error, not a silent false

A bool reader that coerces ("true" → false, or worse, → true) is how a config typo becomes a silently different model — and the whole point of the flag is to gate three refusals. Asserting threw rather than !d.use_qk_norm is the right assertion: the wrong answer here is accepting the value, not the value's sign.

And the last arms wire the value to the refusals rather than to a literal — "the three refusals are fed from the parsed value, not from a literal" — plus one arm that goes through read_dims_exported on a config.json on disk, so the parse and the plumbing are both covered. An arm that called qk_norm_refusal(true, …) directly would pass forever while read_dims() returned false, which is the exact failure the PR exists to prevent.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 48bdd07: READY

This pins the importer's own read of use_qk_norm with a real config fixture — hunyuan_v1_dense, hidden 4096, 32 layers, 32 heads, 8 kv heads, head_dim 128 — rather than with a synthetic minimal document.

That matters because the whole use_qk_norm thread today (#1889 refusal, #1895 declaration-as-witness, #1938/#2063 the gains' order, #1976 the forward route) rests on the importer reading the right field from the right config shape. A fixture that is a real model's config catches the case a minimal one cannot: a key that exists but sits at a different nesting, or a name the parser half-matches.

The declaration is the witness the refusals hang on, so pinning the read is pinning all of them.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at 48bdd07 — test-only, in the territory of #1889, and the type-error arm is the one I checked hardest because "never a silent false" is the sort of claim that quietly depends on a helper's semantics.

The chain the test walks is real, not simulated. use_qk_norm reaches the refusals through read_dims() → opt_field<bool>(c, "use_qk_norm", false), and opt_field is

const auto it = c.find(key);
return (it == c.end() || it->is_null()) ? fallback : it->get<T>();

so: absent → false (the arm "an absent key is not declared"), null → false (the arm "null is absent, not a parse error"), and a JSON string "true" → it->get<bool>() throws nlohmann::json::type_error. Nothing at the call site catches it, so it propagates. The arm asserting that "true" throws is therefore correct because of the helper's implementation, which is exactly the sort of thing that silently stops being true — and the try/catch around dims(R"(,"use_qk_norm":"true")") names the type it expects.

The gap it closes is a real one and the body states it precisely: the existing arms passed a hand-written true into qk_norm_refusal / header_witness_refusal, so the importer could have stopped reading the key entirely — and Hunyuan-7B's "declared, gains not located" refusal would have disappeared with the suite green. Driving it from config JSON text through read_dims_exported() to the message is the difference between testing the refusals and testing the wiring that feeds them. It also covers the file-reader path (load_geometry()), which is what plan() actually calls, so the key surviving the text_config lift / top-level-wins rules is included rather than assumed.

Two small notes. (1) The fixture is a hand-written Hunyuan-shaped config (hidden_size 4096, 32 layers, 32 heads / 8 kv, head_dim 128, IM 13312, vocab 120818) — good, because those are the numbers the 7B pack really carries; one line saying so would stop a future reader treating them as invented. (2) The arm that matters most for #1889 is the pair "declared in config.json and gains not located is refused" / "an undeclared pack with no gains is not refused" — if a later change made the second refuse too, every pack without the norm would stop importing, and this is now the only test that would say so. Worth a comment saying that, since it is a two-sided invariant rather than two independent checks.

Not verified here: I did not run the mutation the body describes (replacing the read with false and watching the suite fail) — that needs a build of npu_pack_import_test, and I would rather say I read the arms than claim a run I did not do. The claim is checkable exactly as stated, which is the point of the PR.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at 48bdd07

This is the class of gap I have spent the night finding in other people's arms, fixed in one place: the existing tests passed a hand-written true to the three refusals, so they tested the consumers of use_qk_norm and never the read that produces it — and the PR carries the negative control that proves it ("replace the read with false and the whole suite stayed green"). An arm that supplies the argument it is checking cannot fail when the argument's source breaks.

Nine arms, and the three that decide whether this is real coverage:

  • absent → not declared. This pins the default (false), which is the case for every pack that has no per-head norm — i.e. the majority. A default is a policy, and now it is a tested one.
  • null → absent, not a parse error. Correct JSON semantics, and the arm says so rather than leaving it to a reader to guess.
  • wrong type → throws: "use_qk_norm": "true" is an error, "never a silent false". That is the policy that matters most here, because a silently-defaulted malformed key is exactly how "declared, gains not located" would vanish — the failure this key exists to trigger would be suppressed by a typo, with no test noticing.

And it goes far enough: the three refusals are fed from the JSON rather than from a hand-made value, plus a config.json on disk reaching Dims through the file reader, all through pi::read_dims_exported — the same entry point the importer uses. So "from JSON to the refusals" in the title is accurate rather than aspirational.

Test-only (+41/−0) inside tests/npu_pack_import_test.cpp, whose binary is already a ctest row, so there is no runner question and nothing new for ci_targets_check to see.

Nothing requested. If a later reader wonders why null is not an error and a string is, the two messages here answer it.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY at this head.

I reproduced the reviewer's own mutation and it now fails three arms — which is exactly the test of whether the gap is closed:

d.use_qk_norm = false;  /* i.e. the read removed */
FAIL use_qk_norm: a config that states true is read as declared
FAIL use_qk_norm: a value of the wrong type is an error, never a silent false
FAIL use_qk_norm: and declares it, with gains not located, refused (engine#1889)

restored → all npu-pack checks passed. Before this PR that mutation left the whole suite green; now it cannot.

The nine arms are the right nine because they draw the four-way distinction the read actually has, and the two middle ones are where this kind of code goes wrong:

"use_qk_norm":true   -> declared
"use_qk_norm":false  -> not declared
absent               -> not declared      (every pack without the norm)
null                 -> absent, not a parse error
"true"  (a string)   -> THROWS            (a wrong type is an error, never a silent false)

The string case is the one worth keeping: opt_field<bool> on a quoted "true" raising rather than coercing or silently defaulting is the difference between a mis-declared pack being refused and it importing as untied/unnormed and serving plausible garbage — the failure class this lane has spent the week removing.

And the structural point is worth stating in the body, because it is the fifth time: this is the helper-vs-call shape — #1917 (sweep helper tested, call not), #1986 (short-vote branch), #2020 (forward gate), #2026 (padded-head helper bypassed), and now the importer's own read of a value whose refusals were already well tested with a hand-written true. The pattern is consistent enough to name: a pure function with good arms and a call site with none. This PR is the one that closes it by driving the value through the importer's own entry points (read_dims_exported, load_geometry_exported) rather than calling the refusal functions directly, which is what makes it a test of the read rather than a second test of the refusals.

Nothing else; test-only, no pack, no device, and it pins the bridge between #1976's forward read (already covered in C) and the importer's read (covered here).

@github-actions

Copy link
Copy Markdown

Persistent review updated to latest commit a3ea2bc

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY at a3ea2bc — two readers of one config field, one of them pinned; this pins the other.

The gap, stated in the direction that makes it a gap

"use_qk_norm is read in two places. The forward's read (q4nx_semantics.c) is pinned by #1976's new C test. The importer's read is not."

That is the precise form of a whole class this queue has been finding all night: a field with more than one reader, and the test covering the reader the author was looking at. The forward's use_qk_norm decides the rope/norm path at execution; the importer's decides what geometry it bakes into the container. If they disagree, nothing crashes — the model loads, answers, and is quietly wrong, which is exactly why the importer's read needs its own arm rather than being assumed to follow the forward's.

Two asks

  1. assert that the two readers AGREE, not just that each is pinned — the strongest available test here is a container whose use_qk_norm the forward and the importer both read, asserting they resolve it to the same value. Two independent tests can both pass while disagreeing about the field's meaning (true/false/null, or a string "true" read truthily by one and strictly by the other) — and disagreement is the failure mode, not a missing reader;
  2. state the null / absent case — #1976's C test covers "use_qk_norm true, false, null and nested" for the forward. If the importer treats an absent key differently (default true, or refuse), that difference is the one a pack from a different generation will hit first.

And "test only" is the right scope

Refs #1889 with no code change: the gap is coverage, and a test-only PR that names which reader was uncovered, at which head (4dd1139), by whom ("the gap a reviewer measured") is the cleanest possible form — it lets the reviewer check their own finding rather than trusting the fix.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at a3ea2bc

I ran the mutant the PR's own comment names, and the suite now goes red for it -- which is the only thing a test-only PR can be judged on.

// mutant, in npu/pack_import.cpp read_dims():
-    d.use_qk_norm = opt_field<bool>(c, "use_qk_norm", false);
+    d.use_qk_norm = false;   // MUTANT
build errors: 0
mutant test rc=1, 5 failing lines
FAIL use_qk_norm: a config that states true is read as declared
FAIL use_qk_norm: a value of the wrong type is an error, never a silent false
FAIL use_qk_norm: declared in config.json and gains not located is refused (engine#1889)

The third line is the one that matters: it is the end-to-end arm -- config.json on disk -> read_dims_exported -> the refusal text -- so the mutant is caught at the level where the defect would actually show, not only at the field. Before
this PR the same mutant left the whole suite green, which is exactly what the header comment says and what I just reproduced in reverse.

The nine arms cover the four JSON shapes and the type error, and the distinctions are the right ones:

  • true declared, false not, absent not, and null is absent rather than a parse error -- a null that threw would make every pack with an explicit null unloadable;
  • a wrong-typed value ("true" as a string) is an error, never a silent false -- the failure mode that turns a declaration into a default without telling anyone;
  • and the two refusals are fed from the parsed value rather than from a hand-written true, which is what stops the arms passing while the pipeline ignores them (qk_norm_refusal(yes.use_qk_norm, false) non-empty vs the absent case empty, and header_witness_refusal(1, …) both ways).

It drives the importer's own entry points (dims(...), read_dims_exported(...), qk_norm_refusal(...)), not a copy of the parsing, so the arms cannot agree with a mock while the real reader drifts. And no pack is needed: npu_pack_import_test builds and runs
host-only in this tree (all npu-pack checks passed).

One note, not blocking: the arms are in the npu_pack_import_test binary, which is registered, so they reach CI -- but the mutation above is not something CI can do, so the "these arms would catch the regression" property is now known only because a reviewer ran it. If
this file grows another declaration like this, the cheap generalisation is a --mutant mode in the test that rewrites the read and expects failure; not worth it for one key.

Ran: the mutant (above, restored), the unmutated suite (all npu-pack checks passed), and a 0-error build.

@bong-water-water-bong
bong-water-water-bong merged commit 5bd916f into main Nov 13, 2026
9 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the cl-npupack-a/1889-importer-qk-parse branch November 13, 2026 02:31

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at a3ea2bc: READY

Five arms for a boolean, and every one of them is a case a naive get_bool would collapse into false.

"use_qk_norm": true   -> declared
"use_qk_norm": false  -> not declared
key absent            -> not declared
"use_qk_norm": null   -> absent, not a parse error
wrong type            -> an error, never a silent false

The fifth is the one that matters most and the one most code gets wrong: a value of the wrong type must fail, not fall back. A silent false there is indistinguishable from a model that genuinely does not use qk-norm, and the next reader has no way to tell a config typo from a model property — which is exactly how a norm gets skipped by accident.

The fourth is the second most valuable: null treated as absent rather than as a type error is a judgement call, and it is the right one — a null in a pack's config is how "not set" is spelled once JSON tooling has touched the file — but the fact that it is a decision rather than a default is what the arm records.

This is the same distinction I readied in #2309 this evening — tie_stated separating "stated false" from "not stated" — applied to a different key, and the pair is now the convention: a boolean read from a container's JSON distinguishes absent, null, false and wrong-type, and never lets the last three silently agree. Worth a line in docs/npu.md naming that convention, so the next key added follows it rather than rediscovering it.

Test-only change, no behavioural risk, and it pins the importer's own read rather than a helper's.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant