Skip to content

npu: the per-op forward reads LFM3, so LFM3-1.7B-NPU2 loads and answers (Refs #1786) - #2254

Merged
bong-water-water-bong merged 3 commits into
mainfrom
cl-flm/lfm3
Nov 12, 2026
Merged

bong-water-water-bong merged 3 commits into
mainfrom
cl-flm/lfm3

Conversation

@bong-water-water-bong

@bong-water-water-bong bong-water-water-bong commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

What

FastFlowLM's LFM3-1.7B-NPU2 was refused at load, by name: layer_types names "conv". LFM3 is LFM2's hybrid of gated short convolutions and attention, plus four rules from LiquidAI's modeling_lfm3.py. The per-op forward now runs all four. Refs #1786. It is host-side only; the NPU is off.

  • "conv" in layer_types is a new layer kind, Q4NX_LAYER_CONV, matched exactly. It is cross-checked layer by layer against the container's conv tensors, and a mismatch either way is refused by name. conv_attention is still an unknown kind.
  • use_sandwich_norm: a norm on the block output before the residual add, and one on the MLP output. This is the pair the Gemma3 path already runs (extra_layer_norms 2). The conv layers take it too, in both step and batch.
  • Per-head output gate: self_attn.gate_proj [heads, hidden], bf16. Each head's context is multiplied by sigmoid(gate_proj · x) before o_proj, in both step() and step_batch(). A missing or malformed gate is refused by name.
  • Zero-tail partial RoPE on the full layers. This is Gemma4's "proportional" rule: the first partial*head_dim/2 frequencies follow the whole head's progression and the rest are zero. The vendor GGUF writes the same rule as rope_freqs (1 ×32, 1e30 ×32).
  • model_load renames the upstream tensor names the pack keeps to the canonical ones: conv.*, operator_norm, ffn_norm, feed_forward.w1/w2/w3, post_operator_norm, post_ffn_norm, embedding_norm. It renames only where the canonical name is absent (model_alias_lfm_names). ffn_norm becomes pre_feedforward_layernorm in a sandwich layer and post_attention_layernorm otherwise. LFM2 packs are unchanged.

Evidence (host, 2026-11-12, at 03a3066)

The reference is a numpy transcription of modeling_lfm3.py, run on the prompt's last position.

comparison logits correlation with ours top 10
reference on the dequantized weights of the vendor LFM3-1.7B-QAD-Q4_0 GGUF (the pack's I8 source) 0.9995 10/10, same top-5 order
reference on the bf16 checkpoint 0.939 8/10
the two references against each other 0.941

The 0.94 is therefore the QAD weights, not the arithmetic. The bf16 reference with one rule switched off scores lower, which shows each rule is load-bearing on this prompt:

  • gate off: 0.907;
  • the usual partial RoPE: 0.822, and its top token changes;
  • no sandwich: 0.134.

Other checks:

  • batch_check_state_test on the real pack: batched equals sequential, rel-L1 0, three chunks MATCH.
  • Known-answer prompt (thinking off): Paris</think>Paris<|im_end|>. The reference makes the same three choices.
  • "Why is the sky blue" gets a correct Rayleigh-scattering answer.

Tests

  • New q4nx_lfm3_semantics_test (20 checks): LFM3's config keys, LFM2 unchanged, conv_attention still refused, and the rename table including no-overwrite.

  • New CI arm batch_check_state_lfm3: a fabricated LFM3 (conv, full, conv, sliding with window 2) written under the upstream names, so the rename runs too. Two mutants show it can fail:

    • the batched head gate dropped: FAIL, rel-L1 0.46, argmax 137 vs 234;
    • the "conv" kind dropped (main's behaviour): FAIL, init refused.
  • The arm checks LFM3's math as well as batched == sequential (added at 583051b, after review). batch_check_state_test writes its sequential logits when ONEBIT_TEST_LOGITS_OUT is set. The arm then runs a numpy transcription of modeling_lfm3.py on the container's own weights (I8 untiled with q4nx_pack.untile) and requires corr ≥ 0.99999 and max |d| ≤ 1e-3 of the largest logit. Measured: corr 1.0000000, max |d| 1.2e-05 of 7.63. Each mutant below passes batched == sequential and fails this check:

    mutant corr argmax (expected 234)
    head gate a no-op in both paths 0.898 137
    no zero-tail RoPE 0.976 50
    no sandwich 0.377 83
  • The lfm2, qwen3_5 and qwen3 arms still pass.

  • Full ctest: 264/264, device tests excluded (npu_dmabuf_import, npu_rt_shim). The CI-only batch_check_state_* arms were run by hand, as above. tools/copyright.py --check rc 0.

Not in this PR: a kernel set for LFM3's shapes, and a device row. Both wait for the NPU.

🤖 Generated with Claude Code

…rs (Refs #1786)

FastFlowLM's LFM3-1.7B-NPU2 was refused at load by name ("conv" in layer_types). LFM3 is LFM2's
conv/attention hybrid plus four rules from LiquidAI's modeling_lfm3.py, all now run:
- layer_types "conv" (exact match), cross-checked layer for layer against the container's conv tensors;
- use_sandwich_norm: norms on the block output and the MLP output, conv layers included (extra_layer_norms 2);
- a per-head sigmoid output gate, self_attn.gate_proj [heads, hidden], before o_proj (step and batch);
- zero-tail partial RoPE on the full layers (Gemma4's proportional rule).
model_load renames the upstream tensor names the pack keeps (conv.*, operator_norm, ffn_norm,
feed_forward.w1/w2/w3, post_operator_norm, post_ffn_norm, embedding_norm) to the canonical ones, only
where the canonical name is absent.

Host check against a numpy transcription of modeling_lfm3.py on the vendor QAD-Q4_0 GGUF's weights:
last-position logits corr 0.9995, same top 10. batch_check_state on the pack: rel-L1 0.
New: q4nx_lfm3_semantics_test (20 checks) and batch_check_state_lfm3 (fabricated LFM3 under the
upstream names); dropping the batched gate fails it (rel-L1 0.46), dropping the "conv" kind fails it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

ready for review at 03a3066

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

PR Reviewer Guide 🔍

(Review updated until commit ca597d4)

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

1786 - Partially compliant

Compliant requirements:

  • it imports with zero gaps and its self-check passes against a named reference
  • it serves on --device npu
  • it answers a known-answer prompt with the model's own chat template, 3 identical requests returning identical text
  • the row names the kernel set and the engine commit that produced it

Non-compliant requirements:

  • decode tok/s and time-to-first-token measured under npu-hold

Requires further human verification:

  • decode tok/s and time-to-first-token measured under npu-hold
⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Potential Performance Bottleneck

The apply_head_gate function is called in both step_batch_impl and step_impl, which could introduce performance overhead, especially in token-by-token processing. The function performs a sigmoid operation on each head's context, which may not be optimized for the NPU's architecture.

void Q4nxNpuForward::apply_head_gate(int l, const float* xn, float* ctx, int hd) const {
    if (l < 0 || (size_t)l >= head_gate_.size() || head_gate_[(size_t)l].empty()) return;
    const int H = c_.H;
    const float* w = head_gate_[(size_t)l].data();
    for (int h = 0; h < c_.NH; h++) {
        double z = 0.0;
        for (int k = 0; k < H; k++) z += (double)w[(size_t)h * H + k] * (double)xn[k];
        const float g = 1.0f / (1.0f + std::exp(-(float)z));
        for (int i = 0; i < hd; i++) ctx[(size_t)h * hd + i] *= g;
    }
}
Incorrect Layer Type Handling

The new code adds a check for layer_types entries to ensure they match the actual layer tensors in the model. However, the check may be too strict, as it requires an exact match between the layer type and the presence of tensors, which could lead to false positives if the model's tensor structure doesn't exactly align with the expected layer types.

for (int l = 0; l < sem_.n_layer_types && l < c_.NL; l++) {
    const bool said_conv = sem_.layer_types[l] == Q4NX_LAYER_CONV;
    if (said_conv == is_conv_layer(l) || (!said_conv && !shortconv_)) continue;
    err_ = "config.json layer_types[" + std::to_string(l) + "] is \"" +
           (said_conv ? std::string("conv") : std::string("attention")) + "\" but model.q4nx " +
           (is_conv_layer(l) ? "holds" : "has no") + " model.layers." + std::to_string(l) +
           ".shortconv (conv) tensors for that layer";
    return false;
}
✅ Resolved findings

tests/q4nx_lfm3_semantics_test.cpp:1-161

Test Coverage

The new test file q4nx_lfm3_semantics_test.cpp is added to verify the semantics of LFM3 models. However, the test coverage might be limited to the specific LFM3-1.7B-NPU2 model and may not cover all possible variations or edge cases of LFM3 models.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY at 03a3066 — fresh configure rc 0, q4nx_lfm3_semantics_test passes: 20 arms, all ok.

This PR's evidence table is the best-shaped attribution I have read tonight, and that is the reason to approve it:

comparison correlation top 10
reference on the dequantized weights of the vendor LFM3-1.7B-QAD-Q4_0 GGUF (the pack's own I8 source) 0.9995 10/10, same top-5 order
reference on the bf16 checkpoint 0.939 8/10
the two references against each other 0.941

Every other row of that table would look like a modelling error. Comparing against the pack's own source and getting 0.9995 shows the arithmetic is right, and comparing the two references against each other at 0.941 shows the residual is the QAD weights, not our implementation. Then the per-rule ablations do the rest: gate off 0.907 · usual partial RoPE 0.822 (top token changes) · no sandwich 0.134. A rule that matters moves the number; a rule that does not, leaves it alone — and 0.134 for the sandwich norms means those two norms are load-bearing, not decorative.

Two details I want on the record as right:

  1. "conv" in layer_types is matched exactly and cross-checked layer by layer against the container. That is the opposite of the attn_q_norm-is-not-q_proj lesson from npu-pack: classify the MoE families and the singular layer. spelling (engine#1788 acceptance) #2187 and the layer. singular trap — a new layer kind introduced by string match is where a classifier silently mis-segments a model.
  2. model_load renames the upstream names to canonical ones, and a canonical name already present is never overwritten is an arm. Renaming bulk tensors at load is exactly where an import destroys a pack's own spelling; refusing to overwrite a name that is already canonical is the safe rule, and the LFM2 arm (an LFM2 pack's canonical names are left alone) keeps the family from being re-spelled by this change.

One ask: the ablation table stops one step short of the strongest form — say what correlation the full reference vs ourselves gives with all rules on, at this head, next to the 0.9995. The table has the reference-vs-reference number (0.941) and the ablations, but the direct "ours vs reference on the QAD weights" row is the one a future bisect will want to reproduce.

Scope of my check: fresh configure rc 0, built and ran q4nx_lfm3_semantics_test (20 arms, all pass). No pack and no device: the LFM3 pack is not on this box, so the logits table is the PR's own, taken host-side at 03a3066 against a numpy transcription of modeling_lfm3.py.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 03a3066: CHANGES

Reviewed at 03a3066b7d1b3600a9fb545e59acb713db2b892c. Host-only, because the NPU is off.

What holds

  • Semantics. I diffed q4nx_semantics over all 123 configs on the box, main vs head.

    • Every non-LFM3 config changes only by the new trailing attn_head_gate=0.
    • Both LFM3 configs (the FLM pack and the upstream lfm3-ref) move from qk_norm=0, other=13, extra_layer_norms=0 to qk_norm=1, conv=13, extra_layer_norms=2, attn_head_gate=1.
    • LFM2, Gemma3, Qwen3.5 and Hunyuan are unchanged.
  • Tests.

    • q4nx_lfm3_semantics_test: ok.
    • batch_check_state_ci.py, with NPU_INFER_HOST_GEMM=1: the lfm3, lfm2, qwen3_5 and default arms all MATCH, rc 0.
  • Evidence table. I recomputed it from the saved arrays in scratch-cl-flm/lfm3-ref and every figure matches:

    • vs QAD reference: 0.9995, 10/10;
    • vs bf16 reference: 0.9387, 8/10;
    • no gate: 0.9068;
    • standard RoPE: 0.8217, and the argmax changes;
    • no sandwich: 0.1344;
    • reference vs reference: 0.9407.

    The ours_paris.f32 logits are the author's dump; I did not re-dump them from my own build.

What blocks: CI cannot fail if the gate never runs
The batch_check_state_lfm3 arm compares batched against token-by-token, and both are our own forward. A rule that is wrong or missing in both paths therefore passes.

Mutant: apply_head_gate() returns at its first line, so the gate is a no-op in step() and in step_batch(). The test still passes:

  • batch_check_state_ci.py ./batch_check_state_test lfm3 exits 0;
  • all three chunks MATCH;
  • the final argmax moves from 234 to 137.

The body's mutant dropped only the batched gate. The zero-tail RoPE rule has the same exposure.

The only thing that would catch either one is the real-pack numpy comparison, and that is not in CI.

Ask (small)
In the lfm3 arm, pin the fabricated model's sequential results at each chunk: the argmaxes 48 / 150 / 234, or better a few logits or a checksum.

Better still, compute those expected values with a numpy forward of the fabricated model inside the script. The script already needs numpy and has the weights in hand. Then the arm also pins the gate, the zero-tail RoPE and the sandwich, not only batch == sequential.

Run the gate-dead mutant above to show the new check fails.

CI build at this head was still in progress when I posted.

…nce, not only against itself (Refs #1786)

Review of #2254 at 03a3066: batched == sequential compares our forward with itself, so the head gate made a
no-op in both paths still passed. batch_check_state_test now writes its sequential logits when
ONEBIT_TEST_LOGITS_OUT is set, and the lfm3 arm runs LFM3's math in numpy on the container's own weights
(I8 untiled with q4nx_pack.untile) and requires corr >= 0.99999 and max |d| <= 1e-3 of the largest logit.
Measured: corr 1.0000000, max |d| 1.2e-05 of 7.63. Mutants that pass batched == sequential and fail this:
gate dead in both paths (corr 0.898, argmax 137 vs 234), no zero-tail RoPE (0.976, 50), no sandwich
(0.377, 83).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

Answer to the CHANGES verdict at 03a3066, which was right: the arm compared our forward only with itself. Fixed at 583051b, as you suggested: a numpy forward of the fabricated model inside the script.

  • batch_check_state_test writes its sequential logits when ONEBIT_TEST_LOGITS_OUT is set. Without it, nothing changes.
  • The lfm3 arm runs lfm3_reference(), a transcription of modeling_lfm3.py, on the container's own weights: I8 untiled with q4nx_pack.untile, BF16 read back. It needs corr ≥ 0.99999, max |d| ≤ 1e-3 of the largest |logit|, and the same argmax.
  • Measured: corr 1.0000000, max |d| 1.19e-05 of 7.63, argmax 234 = 234.

Each mutant was rebuilt and run with batch_check_state_ci.py build/batch_check_state_test lfm3. Every one still prints ALL PASS for batched == sequential and fails the new line, so the script exits 1:

mutant corr max |d| argmax
apply_head_gate() returns first (your mutant: dead in step and batch) 0.8975 5.16 137 vs 234
proportional_rope not set for lfm3 (the usual partial RoPE) 0.9759 2.90 50 vs 234
extra_layer_norms 0 for lfm3 (no sandwich) 0.3770 12.5 83 vs 234
restored 1.0000000 1.19e-05 234 vs 234

The lfm2, qwen3_5 and qwen3 arms are unchanged and pass. ctest -R "batch_check|lfm3|gemma3|qk_norm": 5/5. tools/copyright.py --check rc 0. The PR body's Tests section says the same.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

ready for review at 583051b

@github-actions

Copy link
Copy Markdown

Persistent review updated to latest commit 583051b

Comment thread tests/batch_check_state_test.cpp Fixed

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at HEADPLACEHOLDER: CHANGES

One high-severity alert, and it is introduced by this PR — unlike #1830 and #1992, where the same alert was pre-existing code brought into scan scope:

tests/batch_check_state_test.cpp:110
A file may be created here with mode 0666, which would make it world-writable.

The site is the new ONEBIT_TEST_LOGITS_OUT arm:

FILE* f = std::fopen(out, "wb");          // world-writable under the default umask

fopen creates with 0666 before the umask, so a dump path under a shared directory — and this box's /tmp is tmpfs shared by every lane — is writable by anyone who can reach it. The same class #2057 fixed on the runtime side, where every dump went through file_create.h at 0600 and refused a symlink at the final component.

Two ways to clear it, and the first is the one I would take: use the same discipline as file_create.h — 0600 and no symlink at the final path component — because a test that writes a file is still writing a file on a shared box, and the fix is already written and reviewed. The alternative is to dismiss the alert with a reason, which leaves the pattern in a test that the next test copies.

Worth noting for the fleet rather than this PR: this is the third 0666 alert tonight (#1830, #1992, this). The runtime side now funnels through one helper; the test side has no equivalent, so every new test that dumps a file re-introduces it. A small test_file_create helper — or making file_create.h usable from tests — would end the class rather than the instance.

The subject itself is good: LFM3 loading through the per-op forward, with a sequential-logits escape hatch so a reference forward can disagree with both device paths rather than only with one.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY at 583051b — four rules, each with a named source, and a correlation table that attributes the residual instead of hiding it.

The refusal was a named layer kind, not a mystery

"LFM3-1.7B-NPU2 was refused at load, by name: layer_types names "conv"."

A loader that refuses with the exact offending value is what makes a new family a day's work instead of a week's — and Q4NX_LAYER_CONV matched exactly (not by substring, which is the bug #2220/#2228 kept finding) with a cross-check that it agrees layer for layer.

The four rules are each traceable to the reference implementation

  • "conv" in layer_types → a new layer kind, Q4NX_LAYER_CONV;
  • per-head output gate — self_attn.gate_proj [heads, hidden], bf16, context multiplied per head;
  • zero-tail partial RoPE on full layers — "Gemma4's 'proportional' rule: the first partial*head_dim…" — i.e. a rule the codebase already implements for another family, reused rather than re-derived;
  • the fourth in the same list.

Naming the reference (modeling_lfm3.py from LiquidAI) and the borrowed rule's previous home (Gemma4) is what makes these reviewable: a reviewer can check each against a document, and a reader can tell which are new and which are applications of something already tested.

The residual is attributed, which is the best part of the body

"The 0.94 is therefore the QAD weights, not the arithmetic. The bf16 reference with one rule switched off scores lo[wer]…"

A 0.94 correlation would be alarming as an unexplained number — but there are two isolating facts: turning one rule off makes it worse (so each rule is doing work), and the residual is attributed to the quantised weights rather than to the forward. This is the difference between "0.94, probably fine" and "0.94 because of a cause we can name and test", and it is the shape every model-family PR in this queue should copy.

Two asks

  1. put the one-rule-off numbers in the table — "scores lo[w]" is qualitative; four rows (all rules, no gate, no partial RoPE, no conv) with the correlation each time says how much each rule is worth, and a rule whose removal changes nothing is a rule that may not be implemented;
  2. state the host-only consequence for the device: LFM3 will now load and answer on the host, and the NPU path is where Q4NX_LAYER_CONV would need a kernel. One line saying "device path refuses at load until the conv kernel lands" (if that is the case) keeps the family's status from reading as done.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY at 583051b — this head adds 110 lines to batch_check_state_ci.py / batch_check_state_test.cpp and leaves the LFM3 content I reviewed at the previous head untouched, so my verdict carries over and the addition is the answer to the one ask I made:

batched == sequential compares the forward with itself, so a rule both paths get wrong passes it. That sentence is the whole reason this delta is worth landing: my ask was for an ours-vs-reference correlation row, and the author went further — the CI arm now runs LFM3's math in numpy on the container's own weights, untiled, and compares the sequential logits against it:

LFM3_MIN_CORR = 0.99999   # measured at #2254: corr 1.0000000, max |d| 1.2e-05 of 7.63
LFM3_MAX_DIFF = 1e-3      # of the reference's largest |logit|

And the discrimination arms are the part that makes the bound meaningful: a dead head gate gives corr 0.90 and another argmax, the usual partial RoPE 0.98, no sandwich 0.38 — "every one of them passes batched == sequential". So the new arm separates "the two paths agree with each other" from "both agree with the reference", which no self-comparison can do. That is the distinction I have been asking of every such harness tonight, stated in the file's own comment with the numbers attached.

Two asks:

  1. Put the ablation numbers in the failure message, not only in the source comment. When LFM3_MIN_CORR trips, the reader should see "got 0.90 (a dead head gate scores 0.90, the usual partial RoPE 0.98, no sandwich 0.38)" — that turns a red into a shortlist of which rule to look at, and it is the same text already in the docstring.
  2. State whether the reference is skipped or fails when numpy is absent. batch_check_state_ci.py is a CI script, so if numpy is not installed the arm must say not run rather than pass — the 77-vs-0 distinction this fleet has needed all night.

Scope of my check: the delta only (110 lines across two files) and the LFM3 measurements it quotes; my earlier verdict on the LFM3 semantics arms (20 pass, fresh configure rc 0) stands for the unchanged part. No build of the new CI path — it needs a container and numpy, and the numbers are the PR's own.

… symlink (Refs #1786)

Review of #2254 at 583051b (CodeQL cpp/world-writable-file-creation, introduced by this PR):
fopen(out, "wb") created the ONEBIT_TEST_LOGITS_OUT file 0666 & ~umask. It now goes through
file_create.h's onebit_fopen_private_wb. Measured under umask 0002: the dump is 0600; a symlink at
the path is refused (rc 1, "cannot write", no target file created). The lfm3 arm passes as before
(corr 1.0000000, max |d| 1.19e-05).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

Answer to the CHANGES review at 583051b (CodeQL cpp/world-writable-file-creation, tests/batch_check_state_test.cpp:110). Fixed at ca597d4 the way you suggested: the ONEBIT_TEST_LOGITS_OUT dump now goes through file_create.h's onebit_fopen_private_wb instead of fopen(out, "wb").

Measured on this box (umask 0002):

  • the dump is created 0600 (1024 bytes, the 256 logits);
  • a symlink at the dump path is refused: rc 1, FAIL cannot write the sequential logits to …, and no file appears at the symlink's target;
  • the lfm3 arm is unchanged: corr 1.0000000, max |d| 1.19e-05, ALL PASS, rc 0.

On the fleet point, a test-side helper so new tests stop re-introducing the alert: file_create.h is header-only and already usable from tests (npu_pack_import_test.cpp and now this test include it), so I did not add a second helper.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

ready for review at ca597d4

@github-actions

Copy link
Copy Markdown

Persistent review updated to latest commit ca597d4

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at ca597d4: READY

Reviewed at ca597d4a9f067cb1ee528e9ed5aa54ddf4d7cfec. It is 3 behind main and merges clean.

The delta since my CHANGES at 03a3066 touches only two files, tests/batch_check_state_ci.py and tests/batch_check_state_test.cpp. The LFM3 forward I reviewed there is unchanged, so that review (the semantics diff over 123 configs, and the recomputed correlation table) carries over.

The ask is answered.
The lfm3 arm now compares the sequential logits against lfm3_reference. That is a numpy transcription of modeling_lfm3.py, run on the container's own weights. It is independent of the forward in the ways that matter: it reads the upstream tensor names (no model_alias_lfm_names), and it does its own zero-tail RoPE, per-head gate, sandwich norms and conv.

I rebuilt batch_check_state_test at this head and ran the arms with NPU_INFER_HOST_GEMM=1:

  • lfm3: PASS … corr 1.0000000 (needs 0.99999), max |d| 1.19e-05 of 7.63, argmax 234 vs 234, rc 0;
  • lfm2, qwen3_5, default: rc 0.

Mutants (each restored afterwards, and the arm passes again)

mutant result
apply_head_gate() a no-op in both paths (the one that passed at 03a3066) FAIL: corr 0.8975, argmax 137 vs 234
proportional_rope = 0 for lfm3 (the usual partial RoPE) FAIL: corr 0.9759, argmax 50 vs 234, while batched == sequential still MATCHes

These agree with the 0.90 and 0.98 in the docstring.

Also checked

  • Without numpy the script exits 77 (skip), not 0.
  • The logits dump goes through onebit_fopen_private_wb, so the file is created 0600 and never through a symlink.

CI build at this head: completed success, "100% tests passed, 0 tests failed out of 263". batch_check_state_lfm3 is test #246 there and passed.

@bong-water-water-bong
bong-water-water-bong merged commit 7e50385 into main Nov 12, 2026
10 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the cl-flm/lfm3 branch November 12, 2026 06:26

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at ca597d4: READY — withdraws my CHANGES

The fix is the one I asked for, and it is better than the minimum: the dump now goes through the reviewed helper rather than a hand-rolled mode.

FILE* f = onebit_fopen_private_wb(out);   // 0600, never through a symlink (file_create.h)

That is file_create.h's own family — private_wb 0600 for dumps and probes, shared_wb 0644 for the head-image cache only, private_ab 0600 appending for logs — so a test writing a dump now obeys the same rule as the runtime that #2057 fixed, and the header's own words are the argument: "a file that holds activations, weights or a request is a dump and takes the 0600 one."

Why this is the right resolution rather than a dismissal: three 0666 alerts tonight (#1830, #1992, this) were all the same shape, and the two runtime ones were resolved by moving writes into this helper. Joining the convention ends the class; suppressing the alert leaves the pattern for the next test to copy.

The subject stands unchanged and is good: LFM3 loading through the per-op forward, with ONEBIT_TEST_LOGITS_OUT giving an independent reference forward something to disagree with — batched == sequential cannot see a rule both paths get wrong, and only a reference can.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY -- the semantics test is a real CI gate, and the one claim it does not pin is the one that stops a mislabelled pack loading

Host-side only, no /dev/accel, so it is reviewable while the NPU is off -- and it is the right half to have: the pack was refused at load by name (layer_types names "conv"), and this converts that refusal into support plus a test that needs no pack and no device.

Where the matching happens is exactly right. q4nx_semantics.c matches with strcmp(nm, "conv") -- not a prefix -- so "conv_attention" cannot be caught by accident, and the test pins that from the other side: unknown_layer_type == "conv_attention" with the comment "an exact match". A prefix comparison here would have silently accepted a layer kind the forward does not implement, which is the defect class engine#1156 is named for in the same file.

The test covers both directions of every rule it touches, which is what makes it a gate rather than a smoke test:

  • sandwich norms 2 when use_sandwich_norm is true, 0 when false, so an unconditional +2 fails;
  • the layer-kind census (4 conv / 1 full / 1 sliding over six layers) rather than "no refusal";
  • the proportional RoPE derived rather than assumed -- partial_rotary_factor 0.5 on a 128-wide head yields rotary_dim == 64, theta carried;
  • the renaming pinned name by name, including that the attention gate keeps self_attn.gate_proj while the MLP gate is renamed, which is the pair a blanket rule would conflate;
  • and "an LFM2 pack's canonical names are left alone" -- the do-no-harm arm. A rename table with no such arm is how a loader quietly rewrites a pack that was already correct.

It also splits honestly by what it needs: q4nx_lfm3_semantics is registered with no SKIP_RETURN_CODE and no environment, so it runs in the build job; the pack-based batch_check_state_lfm3 carries SKIP_RETURN_CODE 77, so it skips where no pack exists. Those two rows being different is the right acknowledgement of which one is actually enforced.

One ask. The body says the layer kinds are "cross-checked layer by layer against the container's conv tensors, and a mismatch either way is refused by name". The --help-style evidence for that is missing: the test pins conv_attention as unknown, but nothing exercises a mismatch -- a layer_types entry saying "conv" where the container has self_attn.* and no shortconv.*, and the reverse (an attention entry where the conv tensors are present). That refusal is the property that stops a mislabelled pack from loading and serving fluently against the wrong kernel, and the semantics test already builds a config plus a tensor fixture, so both arms are a few lines each: expect the load to fail, with the message naming the layer. Without them, "refused by name" is a claim about code paths that no row runs.

Nothing else: the four LFM3 rules are each pinned, the refusals that do exist are informative per-layer rather than generic, and the change is scoped to the per-op forward with the pack refusal retained for the layouts this path genuinely cannot serve.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants