npu: four #1785/#803/#799 follow-ups in one PR — window11 accepts the host-head row; hybrid models stop re-dequantising attention weights per token; two docs corrections

docs/npu-p4-target-split.md

@@ -236,7 +236,7 @@ What is still not wired, and stated as such: `gdn_step`/`attn_step` dispatch nee
 ## 4. Gate (unchanged from #803)
 
 1. `window11`: 3 greedy prompts, text identical to the current binary.
-2. Logits KLD **0** on 8 prompts (the same prompts the hw-gate uses), same model dir and kernel set.
+2. Logits KLD **0** on 8 prompts, same model dir and kernel set in both arms. (This line used to ask for the hw-gate's prompts; the row below was run on eight fixed completions and says so.)
 
    **Measured (2026-11-11, on the NPU, under `npu-hold` tickets).** Baseline: the binary at `540e1fd`
    (before the decode-family move), run twice as the control. Candidate: `main` at `d0fa51c` (with #2166).

docs/npu-plan.md

@@ -152,7 +152,12 @@ path untouched.
 
 `scripts/npu-window11.sh` is the gate the plan change was validated with: three fixed greedy prompts plus
 the prof table, compared between a run with the plan and one without. `--compare` refuses to run without
-`--tol PCT` or `--control C1.log C2.log`; `--self-test` checks that policy.
+`--tol PCT` or `--control C1.log C2.log`; `--self-test` checks that policy. The prof rows of the two
+runs must be the same set of (phase, role, M), with one declared exception: a run with the decode lm head on
+the host (`NPU_HEAD_HOST`, [npu.md](npu.md)) prints `dec HEAD M=1` where a device-head run prints
+`dec HEAD M=16`. When that pair is the whole difference the comparison passes, prints both head rows and
+which side each is on, and scores every shared row as usual; any other missing or added row still fails
+(engine#1785).
 
 Tests: `tests/npu_plan_test.cpp` (ctest `npu_plan`) pins the loader and the refusals;
 `tests/npu_plan_generate_test.py` (ctest `npu_plan_generate`) runs the generator and `--check` on a
@@ -189,6 +194,9 @@ layers[0]     {"index": 0, "role": "gdn", "qkv": {"K": 5120, "N": 10240, "fused"
 
 ## Not yet
 
+The tracker (#799) closes with P4 (#803). What follows is the part of the plan that was **not built**, so that closing
+it does not read as "all seven phases shipped": each entry says what landed, what did not, and why.
+
 - **P4** (#803): **A1 landed** in #808 (`0a2cbb9`) — the `NpuTarget` interface (`npu/target/target.h`), the XDNA
   target behind it, and `open()` consuming the plan instead of duplicating the derivation. The step-by-step record is
   [npu-p4-target-split.md](npu-p4-target-split.md) §3, so this list does not snapshot it, but the two ends that touch
@@ -204,9 +212,17 @@ layers[0]     {"index": 0, "role": "gdn", "qkv": {"K": 5120, "N": 10240, "fused"
   ([npu-p4-target-split.md](npu-p4-target-split.md) §4). `XdnaTarget::load_block` remains a named no-op, so weight
   residency and decoding stay as they were — the cache fills lazily inside `gemv`, and turning that into an
   explicit block step is an optimisation whose evidence is a tokens/s row, not a structural move.
-- **P5** (#804): Hexagon target (t2 gemv on HVX/HMX, QEMU v73 oracle) has no engine rows yet.
-- **P6** (#805): the shared drafter protocol is on main; tau=0.6 and Hexagon acceptance rows are still missing.
-- **P7** (#806): the `kv` contract above is in the plan and its arithmetic is gated in CI (`ctest -R npu_kv_int8`,
+- **P5** (#804, closed 2026-11-07 on the scope the owner set: a toolchain check): the t2 GEMV is bit-exact in QEMU
+  v73 on the container's own layer-0 tensors and has a cycle model, and `tests/npu_t2_target_gate.py` (#1272) holds
+  any t2 target to the pinned accumulation order. **Not built:** a Hexagon `NpuTarget` in the engine (`npu/target/`
+  holds `xdna` only, and `--npu-target hexagon` is refused by name), silicon timing of our kernel (no board on the
+  fleet), and the decoder layer composed on the DSP.
+- **P6** (#805, closed 2026-11-07): the shared drafter protocol is on main and measured on XDNA at both gates, 8
+  prompts x 64 tokens, text identical 8/8 — tau 0.6 (the default): hit rate 78.77 %, depth-1 acceptance 78.34 %, 1.83
+  tokens per pass; tau 0: 46.69 %, 75.14 %, 2.87 ([npu-leaderboard.md](npu-leaderboard.md), the `27b-mtp-*` rows).
+  Leaf fill was rejected with a test (`tests/mtp_draft_test.cpp` [16]) rather than built. **Not measured:** acceptance
+  on a second target, because there is no second target (P5).
+- **P7** (#806, closed 2026-11-07 with the refusal in place, the writer not built): the `kv` contract above is in the plan and its arithmetic is gated in CI (`ctest -R npu_kv_int8`,
   which needs no device: see below), and the **target-side refusal** is enforced as well (`npu_target_caps`, so an
   `int8` plan cannot be run as f16 by any target, including P5's Hexagon). Still open, in the order it can be taken:
   the cache **writer/reader** in the

docs/npu.md

@@ -2194,6 +2194,19 @@ On FastFlowLM's Qwen3.5-2B-NPU2 (IM 6144, H 2048, a 151 MB tail that the earlier
 run in one ticket measured decode at 362–367 ms per token with the cache on (both arms) against 485–520 ms with
 it off in the quieter of the two off arms and 478–707 ms in the other, same text in all twelve runs (engine#1785).
 
+The full-attention layers of such a model had the same reload, for their own four buffers: `load_layer()`
+dequantised `q`, `k`, `v` and `o` into `wq_`/`wk_`/`wv_`/`wo_` on every decode token, and the only reader of
+those buffers (`q4nx_assemble_B`) is skipped once the layer's packed QKV and O images are on the device. The
+reload is now skipped when **both** images are resident (the batched path guards its O assemble with the QKV
+key, so a half-cached layer still loads everything) and the run is not `NPU_INFER_HOST_GEMM`.
+`NPU_ATTN_RELOAD=1` keeps the old per-token reload. On Qwen3.5-2B-NPU2 (6 full-attention layers of 24), an
+A B B A run in one ticket with `NPU_PHASE_PROF=1`, 46 decode tokens per arm, medians: `load_layer` 13.0 and
+11.6 ms per token with the reload against 0.04 and 0.03 ms without; the layer loop's host share 23.9 and 22.2
+ms against 10.6 and 11.1 ms; device wait unchanged (240.7 to 247.3 ms in all four). All 144 logits dumps of
+the other three arms are byte-identical to the first arm's, so this removes work and changes no value. The
+whole token moved from 347 and 345 ms to 332 and 341 ms, which on that box (load 20 to 27) is inside the
+spread between two arms of one mode; the host share is the number to read (engine#1785).
+
 ### The prefix cache's snapshot: two kinds of plane
 
 `snapshot_state`/`restore_state` (`npu/forward/include/q4nx_forward.h`) hand a prefill's state to the next

npu/forward/src/q4nx_forward.cpp

@@ -2063,12 +2063,22 @@ bool Q4nxNpuForward::load_layer(int l) {
     // copies of the big projections are never needed: skipping them here is the
     // difference between a first token that dequantises the whole 17.9 GB model and
     // one that opens 13.4 GB of images.  The norms and every small member still load.
+    // The four attention f32 buffers (wq_/wk_/wv_/wo_) are read only by q4nx_assemble_B(), which step() and
+    // step_batch() skip once this layer's packed QKV (key 0) and O (key 1) images are on the device -- the
+    // same argument as ffn_tail_cached() for the MLP and proj_cached for a GDN layer. A dense model never
+    // gets here on a cached layer (step()'s `dense_weights_cached`), but a hybrid one does on every token:
+    // its full-attention layers re-dequantised q, k, v and o each decode step, 17.5 ms of a 285 ms token on
+    // Qwen3.5-2B (6 of 24 layers), for buffers nothing read (engine#1785). BOTH keys must be cached:
+    // step_batch() guards its O assemble with the QKV key, so a half-cached layer still loads everything.
+    // NPU_ATTN_RELOAD=1 keeps the old per-token reload, for the A/B.
+    static const bool attn_reload = getenv("NPU_ATTN_RELOAD") != nullptr;
+    const bool attn_cached = !attn_reload && !host_gemm_ && xd_ && xd_->weight_cached(l, 0) && xd_->weight_cached(l, 1);
     if (is_conv_layer(l)) {
         // LFM2 conv layer: two projections where an attention layer has four. Same f32-until-packed rule.
         LhScope _t_p(LH_LOAD_PROJ);
         if (!deq(&lw->shortconv_in_proj_weight, wqkvlin_.data(), 3 * H, H)) return false;
         if (!deq(&lw->shortconv_out_proj_weight, woutlin_.data(), H, q4nx_pad_k(H))) return false;
-    } else if (!xd_->pb16i()) {
+    } else if (!xd_->pb16i() && !attn_cached) {
         if (!deq(&lw->q_proj_weight, wq_.data(),
                  NH * hd * (attn_gate_ ? 2 : 1), kpad_)) return false;   // q|gate rows
         if (!kv_shared) {

scripts/npu-window11.sh

@@ -31,6 +31,8 @@
 # so two runs can be compared without trusting a human to eyeball two terminals.  The gate a change has to
 # pass is: 3/3 transcripts byte-identical, the prof table's rows (phase, role, M) identical, and no row's
 # ms/launch outside the tolerance.
+# One row difference is a declared mode and passes: `dec HEAD M=1` (the lm head on the host, NPU_HEAD_HOST)
+# against `dec HEAD M=16` (on the device), when that pair is the whole difference (engine#1785).
 #
 # The tolerance is NOT a constant: engine#834 measured the same binary against itself on this box and the rows
 # moved up to 7.7 %, so a 2 % assertion cannot be evaluated here at all.  `--compare` therefore refuses to run
@@ -229,8 +231,26 @@ if ha or hb:
               f"{sum(1 for t in ka if t.startswith('[qkv-dump]'))} qkv dumps)")
 
 rows_a, rows_b = set(pa), set(pb)
-if rows_a != rows_b:
-    print(f"FAIL prof rows differ: only in A {sorted(rows_a - rows_b)}, only in B {sorted(rows_b - rows_a)}")
+# The one row difference that is a declared mode and not a regression (engine#1785, #2191): the decode lm
+# head computed on the host (NPU_HEAD_HOST) prints `dec HEAD M=1`, one host pass per token, where the device
+# head prints `dec HEAD M=16`, one row per launch. The two rows time different things, so they are reported
+# side by side and not scored against each other; the logits are the same numbers either way, which the text
+# rows above already assert. It is accepted only when that pair is the WHOLE difference: any other row
+# missing or added still fails, and so does a log with both head rows or with neither.
+HEAD_DEV, HEAD_HOST = ("dec", "HEAD", "16"), ("dec", "HEAD", "1")
+only_a, only_b = rows_a - rows_b, rows_b - rows_a
+head_moved = None
+if (only_a, only_b) == ({HEAD_DEV}, {HEAD_HOST}):
+    head_moved = ("A", HEAD_DEV, pa, "B", HEAD_HOST, pb)
+elif (only_a, only_b) == ({HEAD_HOST}, {HEAD_DEV}):
+    head_moved = ("B", HEAD_DEV, pb, "A", HEAD_HOST, pa)
+if head_moved:
+    dev_side, dk, dp, host_side, hk, hp = head_moved
+    print(f"prof rows: {len(rows_a & rows_b)} identical (phase, role, M), plus the decode head: on the device in "
+          f"{dev_side} (HEAD M=16, {statistics.median(dp[dk]):.3f} ms per launch) and on the host in {host_side} "
+          f"(HEAD M=1, {statistics.median(hp[hk]):.3f} ms per token); reported, not scored against each other")
+elif rows_a != rows_b:
+    print(f"FAIL prof rows differ: only in A {sorted(only_a)}, only in B {sorted(only_b)}")
     fail = 1
 else:
     print(f"prof rows: {len(rows_a)} identical (phase, role, M)")
@@ -341,6 +361,27 @@ with open(pb, "w") as f:
 rc, out = run(pa, pb, "--tol", "10")
 check("statistic in force: median-of-ratios" in out and "median-of-ratios" in out,
       "two rounds per row: the paired statistic is named and printed", out)
+
+# The decode head on the host (engine#1785): `dec HEAD M=1` against `dec HEAD M=16` is the one accepted row
+# difference, in either order, and only when it is the whole difference.
+hd, hh = ("dec", "HEAD", "16"), ("dec", "HEAD", "1")
+dev = write("dev.log", {**base, hd: 3.100})
+host = write("host.log", {**base, hh: 9.000})
+rc, out = run(dev, host, "--tol", "10")
+check(rc == 0 and "on the device in A" in out and "on the host in B" in out and "not scored" in out,
+      "device head against host head passes and says which side is which", out)
+rc, out = run(host, dev, "--tol", "10")
+check(rc == 0 and "on the device in B" in out and "on the host in A" in out, "the same in the other order", out)
+rc, out = run(dev, write("host_slow.log", {keys[0]: 2.600, keys[1]: 5.000, hh: 9.000}), "--tol", "10")
+check(rc == 1 and "outside tolerance" in out, "a host-head log is still scored on every shared row", out)
+rc, out = run(dev, write("host_extra.log", {**base, hh: 9.000, ("dec", "Z", "16"): 1.0}), "--tol", "10")
+check(rc == 1 and "prof rows differ" in out, "a host-head log with another row added still fails", out)
+rc, out = run(dev, write("nohead.log", base), "--tol", "10")
+check(rc == 1 and "prof rows differ" in out, "a log with no head row at all still fails", out)
+rc, out = run(dev, write("both.log", {**base, hd: 3.100, hh: 9.000}), "--tol", "10")
+check(rc == 1 and "prof rows differ" in out, "a log with both head rows still fails", out)
+rc, out = run(dev, write("host8.log", {**base, ("dec", "HEAD", "8"): 9.000}), "--tol", "10")
+check(rc == 1 and "prof rows differ" in out, "a head row at any other M is not the host head", out)
 print()
 print("self-test: " + ("PASS" if not failures else "FAIL"))
 sys.exit(1 if failures else 0)