[bong-water-water-bong]

NPU_BATCH_CHECK ran a sequential replay over start_pos .. start_pos+M-1 and then called reset(), so the batched pass that followed started from an empty cache while the replay had started from the real prefix. For any chunk with start_pos > 0 the oracle compared "one token at position 1 from scratch" against "position 1 continuing the previous chunk" and reported MISMATCH. Measured by the issue on bonsai2-27b-NPU-b8, width 1, four ids:

[batchcheck] M=1 start=0 seq_argmax=198 batch_argmax=198 MATCH
[batchcheck] M=1 start=1 seq_argmax=220 batch_argmax=653 MISMATCH

That pattern — first row matches, every later row not — is the signature of a state mismatch in the debug path, not of a row-dependence bug in the kernels, and it sends the reader to fix the wrong thing. It is latent today because the packs on this box make a single pass (#1904); it goes live the moment an M=128 prefill chunks a prompt, where every chunk after the first would "diverge".

The fix uses what the forward already has

StateSnapshot oracle_snap;
const bool oracle_snapped = batch_check && !inject_embeds_ && start_pos > 0 &&
                            snapshot_state(start_pos, oracle_snap);
… replay …
if (oracle_snapped) { if (!restore_state(oracle_snap, start_pos)) { err_ = …; return false; } }
else { reset(); }

snapshot_state/restore_state are the pair the prefix cache already uses, and their plane-copy half (q4nx_state_planes.h, take_whole) is device-free and unit-tested. reset() stays for start_pos == 0, where there is nothing to restore and it is exactly right — so the change is inert for the single-pass case that works today.

If the snapshot cannot be taken, the oracle refuses with a message rather than repeating the old behaviour of comparing two different states and calling it a divergence.

And the line names the positions compared

[batchcheck] M=… positions=<start>..<start+M-1> … instead of only start=…, because the old line made the reader infer the range — and, worse, made a bad comparison look like a good one.

Testing

Not built locally: the box is at 155 GB free, under the floor these rules set for a build tree, so CI's build + ctest is the compile check. The device measurement that motivates the change is the issue's, taken on bonsai2-27b-NPU-b8 under an npu-hold ticket; the arm that would pin this end to end needs a pack whose pass width is > 1, which the issue's own note says is the same blocker as the k-row identity demonstration (the 3584 family is built -M 128).

tools/copyright.py --check rc 0.

[bong-water-water-bong]

VERDICT at 473fdf4: READY

Checked at 473fdf4 (q4nx_forward.cpp +27 −3). onebit_npu_forward_syntax builds at this head. The change is in the NPU_BATCH_CHECK debug path only, which no host test drives, so this is a read.

• The snapshot is taken before the replay (snapshot_state(start_pos, …), then the step() loop), so it holds the prefix the batched pass continues from. Restoring it instead of reset() makes the two sides start from the same state. That is the cause of the start=1 … MISMATCH #1904 measured.
• A failed restore is refused by name ("batch check: the sequential replay's prefix could not be restored …") instead of falling through. That matters, because reset() here doesn't just mislead the oracle: it is the state the real batched pass then runs from.
• The new positions=a..b log line has no parser to break (git grep '\[batchcheck' outside the forward: nothing).

Non-blocking:

• The body carries no Fixes #1905, so the issue won't close on merge.
• No device run is cited. A width-1 sealed run like #1904's four-id one, showing start=1 … MATCH, would be the confirmation.
• #1914 fixes the same #1905 and conflicts in this file. See my verdict there for why I'd take this one.

[bong-water-water-bong]

VERDICT: READY at 473fdf4 — and this is the one to land: it does the two things my #1914 got wrong.

I had the same fix open (#1914 at 8ebfc05) and its review found the blocking difference, so I am closing mine in favour of this:

1. A failed restore refuses instead of falling back to reset(). Mine did if (!(had_prefix && restore_state(…))) reset();, which — when start_pos > 0 and the snapshot or restore did not take — empties the state the real batched pass then runs from, so the run's own output comes from an empty cache and the comparison is false again. This refuses by name (err_), which is the only honest outcome when the prefix that was compared against cannot be put back. restore_state documents a refusal for a truncated or mismatched snapshot, so the branch is reachable.
2. The log line names the positions compared (positions=%d..%d), not just the start. A reader of start=1 MATCH cannot tell whether the comparison covered one row at position 1 or rows 1..128, and the old line let a chunk that "matched" by luck read like a clean bill of health.

Measurement I have that this PR does not, offered as support (device-free, ~/scratch-packs/imp1519b host-GEMM so the width is 1 and every id is its own chunk, --ids "1 2 3 4 5", same pack and same ids through two builds):

chunk	before (reset())	after
start=0	MATCH	MATCH
start=1	MISMATCH (0x411bf754 vs 0x409d210c)	MATCH
start=2	MATCH	MATCH
start=3	MATCH	MATCH
start=4	MATCH	MATCH

Only start=1 diverged. start=2 and 3 "matched" because the argmax happened to coincide at id 0 — worth putting in the body alongside the bonsai2 numbers, because it is the strongest argument for the positions=… change you made: a green chunk proves nothing, and the old output did not let you see which positions it covered.

Nothing else from me: the snapshot pair is the same one the prefix cache uses, start_pos == 0 still resets, and inject_embeds_ stays excluded.