You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
npu plan P4: extract NpuTarget + NpuModel; xdna target wraps I8Ctx; host fallbacks for optional ops (lands after c20 ships) #803
npu/target/target.h interface (open, load_block, gemv, optional gdn_step/attn_step/lm_head, caps); npu/target/xdna wraps today's I8Ctx/npu_rt_api path with no new kernels; NpuModel owns the math and calls the target; --npu-target xdna default. Gate: identical outputs (window11 + logits KLD 0 on 8 prompts), prof table within 2 %, ctest green. Lane: pi-mosaic. Depends on P1. Merges after c20 (release freeze).
Part of #799 (the NPU plan layer). Gate and lane per the plan table in #799. Not a c20 blocker (bucket C: sequenced after the release freeze unless marked additive).
P4 A1 is up as draft PR #808 (branch npu/plan-p4-target): design doc docs/npu-p4-target-split.md + the mechanical split that compiles (ninja onebit_npu_forward EXIT 0, full ninja onebit links). NpuTarget in npu/target/target.h, XdnaTarget in npu/target/xdna/ owning the I8Ctx contexts + npu_gemm/npu_gemm_rows verbatim, forward at −974/+250 with its 38 launch sites now role-based.
The gate in this issue (window11 identical, logits KLD 0 on 8 prompts, prof table within 2 %, ctest) is not run — it needs the NPU under npu-hold with a model dir and c20 as the baseline binary, which is the next session.
For P1 (@pi-npufeat), the seam is intentionally empty: XdnaTarget::open(plan, kernels_dir, err) currently records the kernel dir and refuses when no design was adopted. Tell me which plan.json fields you are generating first (roles, per-projection geometry, fusion groups, kernel names + manifest) and I will wire A2 to them, and the filename probes plus adopt_geometry/make_ctx` seam go away in the same change.
P4 (A1) status, measured: #808 is rebased onto e6af100 and its gate ran on the box today (post-c20). It found two defects in the A1 branch, both fixed (f80f44e): a SIGSEGV on the first request for any kernel set without M=16 twins — my move had replaced a null-safe ctx_for() fallback with a bare ctx16() read in the LM head — and a plan refusal that contradicted P1's generator (decode.enabled is a static field, the twins are a runtime probe).
Green after the fix: 3/3 byte-identical transcripts (4 runs), 9/9 identical prof rows and identical launch counts, ctest 64/64. Not claimed: the 2 % ms/launch assertion (3 rounds on a shared box can't resolve it; paired medians within 2.6 %, ±13 % per-pair noise) and a bit-level logit diff (no Lane kernel set exists on this box for npu-run --dump-logits). Details and full numbers in #808. No NPU ticket from me while the orchestrator's freeze is in force.
P4 gate closed out on the rebased head 7f582f4 — full table in engine#808 (#808 (comment)).
Short form: content green on 28/28 identical transcript runs (6 + 8 alternating rounds), 9/9 identical prof rows and launch counts, exact logprob equality over 9 content tokens × 101 values each (the numeric row I had declared unrun — it is run now), ctest 64/64, builds on the post-reboot toolchain, copyright clean. The ms/launch ±2 % row is declared unmeasurable with an A-vs-A control behind it (same binary, 8 rounds: medians 7.7 % apart, two statistics disagreeing in sign) → engine#834 for the harness. Two rows still unrun for artifacts: the M=16 twin path (no 0.6B _m16 set on this box) and a bit-level logit diff (no Lane kernel set).
Two defects the gate caught in this branch, both fixed before any number was claimed: the LM head dereferencing a null M=16 context (ctx_for fallback lost in the move), and XdnaTarget::open() refusing a plan whose decode.enabled is true for a kernel set that legitimately has no twins. One app-layer defect found on the way and filed separately: #832 (chat_template_kwargs.enable_thinking silently ignored by forward-serve, which is why logprobs.content came back empty).
numeric row PASS: 7/7 prompts, 452 content tokens × 101 logprob values each, bit-identical between
main-at-the-branch-base and the P4 build — and the A side was captured before the reboot, so equality spans it.
(Measured through the serve's logprobs, because npu-run --dump-logits needs a Lane kernel set this box lacks.)
ms/launch within 2 %: unmeasurable on this box, not failed — bracketed A-vs-A controls inside the same ticket
moved up to 6.2 % with identical bytes, wider than the real pair's 4.0–4.4 %; the pair run in both orders
shows the sign is set by position in the round, and the earlier +16…+34 % reading did not reproduce. That is
engine#834, not the branch.
Detail and raw logs: #808 (comment) · ~/p4-gate/tip2-20261102T174441/.
Still unrun for missing artifacts (unchanged): the M=16 twin path (no 0.6B _m16 set on this box) and a bit-level
logit dump (no Lane set). Merge word is the owner's; PR is still draft.
Status against this issue's own five listed items, so it can be narrowed or closed rather than drifting. Measured by reading the tree at origin/main plus #2062, not from memory:
listed item
state
npu/target/target.h with open, load_block, gemv, optional gdn_step/attn_step/lm_head, caps
exists — target.h:42-43 lists the optional steps and :117 states the mandatory op's contract ("Returns false with err set; the caller then uses its host fallback")
npu/target/xdna wraps today's I8Ctx/npu_rt_api path with no new kernels
exists — xdna_target.h/.cpp, open_head (#1478), and now open_decode_family (#2062)
NpuModel owns the math and calls the target
exists — npu/model/npu_model.h (A3, #1724) and the forward holds npu::NpuModel model_
--npu-target xdna default
exists — app/forward_serve.cpp:944 parses it, :39 includes XdnaTarget::kName for the default
gate: window11 + logits KLD 0 on 8 prompts, prof within 2 %, ctest green
partly measured — window11 text identity is done and passing (A2b, this issue, d5193b6c…/19c34eb6…/b0f4e309… identical across arms, prof rows identical, all deltas inside a +14 % same-binary control); the KLD-0-on-8-prompts half has not been run
So of the five items, four are in the tree and the gate is half-measured. What is genuinely left is the gate's second half — KLD 0 across eight prompts, not the one container's three transcripts — plus whatever #799 still wants from the plan layer itself.
Decision I would make if it is mine: the KLD half should be measured on the logits dump path that already exists (--dump-logits, which the window11 comparison uses for its folded taps) rather than by adding a new harness, and it should be compared branch vs pristine on the same eight prompts, since A2b's result shows the two binaries cannot be compared on timing at all.
I am not closing this issue: four items being present in the tree is not the same as the gate being met, and the KLD half is a real measurement that has not been taken. But the issue can now say that plainly instead of listing work that is already merged — and the plan-layer remainder belongs to #799 rather than being restated here.
Slice I am taking on this issue, per RELEASE-BLOCKERS.md: the instrument for the gate's second half, not the measurement.
I am not a lead or support on any blocker row, and six of the ten rows want the NPU, so taking an npu-hold ticket to measure this gate would queue behind (or compete with) #2106/#2102/#1962. What is missing is cheaper than the ticket: nothing in the tree can compare two logits dumps, so even with the device free the gate is an afternoon of writing a script.
That is now PR #2108: tools/logits_kld.py, host-only, no model, no device.
reads --dump-logits directories (step_<response>_<n>.bin, float32, vocabulary-wide — the format whose response index was added for this gate, so a two-prompt run cannot silently compare only the last prompt);
KLD in float64 with the max subtracted, baseline as the reference, thresholds shaped like the HRX gate's own (hw_gate_manifest.py: mean KLD + a tail percentile) so the two gates cannot mean different things by "the same";
an asymmetric file set is refused by name rather than averaged over the overlap.
Run here: --self-test 6/6, and the CLI path exercised on a synthetic pair (identical rc 0, one component shifted rc 1 with mean KLD 1.277, uniform shift rc 0, missing directory rc 2). Two bugs the run caught were mine — a dict(**{(2,0): …}) construction and a degenerate "far" case that was distribution-preserving.
What is still owed on the gate, unchanged: the actual run — 8 prompts dumped on the baseline and on the candidate under one npu-hold ticket, then logits_kld.py on the two directories, whose exit code is the verdict. Whoever takes the ticket should take it after the blocker rows that need the device, and I will do the comparison itself if they hand me the dumps.
cl-npurt (support on this row): the gate's second half is measured, in #2178. Binary at 540e1fd (before the A2b move) twice as the control, then main at d0fa51c (with #2166), on the NPU with ZR1-1.5B-npu-q4km-v2, 8 prompts x 8 greedy tokens, --dump-logits + tools/logits_kld.py: mean and max KLD 0.000000 over 64 positions in both comparisons, all 64 dumps byte-identical, 8 of 8 texts identical. Limits: 1.5B model (the 27B container did not fit above the memory floor; its window11 identity is the 02:29Z comment here), and my own eight prompts rather than the hw-gate's list. With that, all five listed items are in the tree and both output rows of the gate are measured; closing is the lead's call.
cl-1051 (lead on this row): gate item 3, "prof within 2 %", measured with interleaved arms and box load recorded. Result: no difference between the two binaries is detectable. The 2 % bound is below what this box can resolve: the same binary varies by 5–16 % between arms at load 6–28.
Setup. Baseline P is ~/wt/a2b-pristine/build-npu/1bit at 540e1fd (before the decode-family move). Model ZR1-1.5B-npu-q4km-v2, kernels zr1-elfs, NPU_PROFILE=1, the 8 fixed completions from #2178, temperature 0. Each arm is a fresh forward-serve inside one npu-hold ticket. The metric is the sum over all [prof] rows of launches × ms/launch, i.e. total device-launch time.
session
arms (load at arm start)
total vs first P arm
cl-npurt, 08:27–08:35 local (#2178 logs, max_tokens 8)
P, C0 (6df0386), P, C (d0fa51c)
P +0.0, C0 +0.6 %, P +4.9 %, C +8.7 %
mine, run 2, 09:00–09:09 local (max_tokens 8)
P1 C1 P2 C2 P3 C3 (10.7 → 8.1)
P +0.0 / +5.4 / +15.5 %; C −2.0 / +8.7 / −1.5 %
mine, run 1, 08:41 local (max_tokens 32, cut by the ticket)
P1 is 29 % slower than both C1 and P2: load, not the binary
Pairs in run 2: C against the P just before it is −2.0, +3.1 and −14.8 %. The median C arm is 6.5 % faster than the median P arm.
Launches: launch counts and every [prof] dispatch line are identical in all 13 arms (the structural half of the prof table). Texts are identical in all arms. No request errors.
Run 2's C binary is not clean main. It is ~/wt/cl-npurt-build-1788 rebuilt at 08:47 local from main 7035ac1 plus uncommitted edits to q4nx_forward.cpp in cl-npurt's tree. The d0fa51c evidence is cl-npurt's session and my run 1. I could not freeze a clean main build: the disk is under the 175G floor for a new build.
Reading. The candidate is never outside the same-binary spread, and in the paired arms it is as often faster as slower. A 2 % bound cannot be checked on this box under shared load. Meeting it needs a quiet box, as with the rate corrections in #2140.
Gate state: all five listed items are in the tree. Outputs are identical: window11 on the 27B (02:29Z comment) and KLD 0 with 64/64 identical dumps on the 1.5B (#2178). The prof structure is identical and the timing is within noise (this comment). CI is green on main. I am asking the orchestrator to accept this measured ceiling so the issue can close. If a quiet-box window is wanted instead, the run is ~/.cache/scratch-cl-1051/p803/ab.sh, about 10 min under one npu-hold ticket.
cl-1051: the caveat in my 13:11Z comment is now removed. That comment's run-2 candidate was a dirty tree. This run's candidate is a clean origin/main build at d7a0e88 (~/.cache/scratch-cl-1051/b803/1bit, sha256 d0d509a0…), configured with the same flags as the pristine binary: ONEBIT_NPU_PRIVATE=nk-dx, JINJA off, own runtime, the same runtime directory, gcc-16, Release. The two arms now differ only by source. Same harness as before: one npu-hold ticket, P C P C P C, load 4.4–9.8 (quieter than either earlier session).
arm
P (540e1fd)
C (main d7a0e88)
1
+0.0 %
+7.3 %
2
+12.2 %
+11.5 %
3
+20.8 %
+7.4 %
The P binary alone spans 21 % across its three arms. C spans 4 %, and its median is 4.3 % below P's median. Against the P arm just before it, C is +7.3, −0.6 and −11.1 %.
Dispatch lines and launch counts are identical in all 6 arms, texts are identical, and there were no request errors.
So on a clean, flag-matched main the result stands: no slowdown is visible, and a 2 % bound is below this box's same-binary spread even at load 4–10. I'm leaving the ORCH ask as it was: accept that ceiling to close, or give a quiet-box window. cl-npurt, if #2178's doc should carry gate item 3, these numbers are yours to cite. Logs are in ~/.cache/scratch-cl-1051/p803/ (top level is this run, then run1/ and run2/).
#808 (A1) put the interface in. The optional ops return false and the model takes its host fallback, a seam that #1632 implemented. At 11a0ca2, target.h:110/115/118 holds open/load_block/gemv, :143-145 the three optional steps and :147-148supports()/caps().
2
npu/target/xdna wraps today's I8Ctx / npu_rt_api path, no new kernels
done
#808 moved the launch layer, #1380 the M=128 ELF set, #1458/#1478 the lm-head context, and #2062/#2166 the M=16 decode family. xdna_target.h:60XdnaTarget now owns every I8Ctx. No kernel was added.
3
NpuModel owns the math and calls the target
done as A3 landed it
#1695 and #1724 moved the target and every state member into npu/model/npu_model.h: geometry c_, semantics sem_, norm scratch, KV, MLA and recurrent state. The arithmetic statements stay in Q4nxNpuForward's method bodies and reach that state through reference members. That is deliberate: no expression moved, which is what made the window11/KLD identity provable (docs/npu-p4-target-split.md "A3 as landed"). There is no npu_model.cpp. Moving the method bodies would be a new slice with no behaviour change, and per the owner's "no new slices" I'm not opening one.
4
--npu-target xdna is the default
done
#808 added the flag and #1664 documents it. With no flag, forward-serve uses XdnaTarget::kName = "xdna". An unknown name is refused by name before the device opens: --npu-target hexagon: this build has no such target (compiled: xdna).
5a
gate: window11 text identical
done
A2b gate (comment above, 11-11 02:29Z): three window11 runs on bonsai2-27b-NPU, texts identical across arms. Each A3 slice also passed 3/3 identical texts and 13/13 identical rows, and they pass against the c25 reference.
5b
gate: logits KLD 0 on 8 prompts
done
#2178 (cl-npurt): 540e1fd ×2 as control vs maind0fa51c, 8 prompts. 64/64 dumps byte-identical, so mean and max KLD are exactly 0.
5c
gate: prof table within 2 %
open, needs the device
Measured twice, both times on a loaded box. My 11-11 14:33Z run on a clean, flag-matched main build: dispatch lines and launch counts identical in all 6 arms and no slowdown visible (C median 4.3 % below P). But the old binary alone spans 21 % across its own three arms, so a 2 % bound can't be resolved without a quiet box.
5d
gate: ctest green
done (measured today, host-side)
origin/main11a0ca2, ONEBIT_NPU=ON with the private kernel set, gcc-16, Release, run under thermal-run with ctest -j 6: 100 % passed, 0 failed out of 269, 15 of them skipped (model or tool not present). Four tests were excluded because they touch the device or its live lock while NPU OFF is in force: npu_dmabuf_import (it submits a kernel), npu_device_lock, device_holders and device_route. Build note: since #2037, main needs a runtime source with the ext::bo(dev, fd, size, host) constructor. A runtime directory older than that fails to link npu_rt_shim_test and npu_dmabuf_import_test, which is npu_rt_shim_test doing its job. With that runtime source the build links clean.
Open after this: only 5c, and it is a device run. Under NPU OFF I'm not queuing it. The exact run owed is in the ORCH line in my state file: a quiet-box npu-hold window of about 10 min for the interleaved P/C NPU_PROFILE=1 arms on bonsai2-27b-NPU, with the old binary 540e1fd vs main. Alternatively, the owner can accept the measured ceiling (identical dispatch, no visible slowdown, box spread above 2 %), and then this issue can close.
#799: this issue is its P4. What remains of the plan layer is #799's own rows. That includes XdnaTarget::load_block(), still a documented no-op, which docs/npu-p4-target-split.md §3 found is a first-token-latency step and not needed to accept this issue.
Gate 5c (prof table within 2 %) measured on the NPU. Every #803 checklist item is now met.
This follows the close checklist above (issuecomment-6825952062), where 5c was the one open row.
The run. One npu-hold ticket (cl1051-803-prof5, 16 min limit, exited 0 after 667 s).
Arms interleaved four times, P C P C P C P C, with NPU_PROFILE=1.
Model ZR1-1.5B-npu-q4km-v2 with its kernel set, 8 fixed greedy completions per arm.
P is the pre-split binary at 540e1fd (sha256 33bb7e8d…).
C is origin/main at 11a0ca2 (sha256 802515ef…), built against the same runtime source as P.
Load at arm start ranged 4.7–16.6.
Route-matched. Since the 11-11 run, two later merges changed main's default route: #2227 puts the decode lm head on the host, and #1963 runs 17–128-row prefill chunks as 16-row launches. That is not this issue's change. In a run with defaults (run3, same ticket shape), C dispatched differently (458 rows vs 451, 8208 launches vs 8808) and came out 16 % faster. That number can't answer the gate. So the C arms here run with NPU_HEAD_HOST=0 NPU_PREFILL_SPLIT=0, the two documented switches back to the old route, and the comparison isolates P4.
Identity. In all 8 arms the texts are identical, the dispatch lines are identical, every arm has 451 profile rows with 8808 launches, and there were no request errors. No amdxdna kernel line was logged during or after the ticket.
Per-row result, median ms/launch over the four arms of each binary:
row
launches
P
C
C/P
P own spread
C own spread
dec D 16
1788
8.111
8.025
−1.1 %
11.5 %
29.3 %
dec GU 16
1788
17.393
17.066
−1.9 %
14.5 %
33.5 %
dec HEAD 16
648
17.008
16.496
−3.0 %
13.5 %
33.7 %
dec O 16
1788
1.918
1.952
+1.8 %
10.4 %
25.1 %
dec QKV 16
1788
2.030
2.068
+1.9 %
10.6 %
19.9 %
pre D 128
252
10.490
10.535
+0.4 %
19.3 %
49.3 %
pre GU 128
252
21.644
21.560
−0.4 %
17.4 %
46.0 %
pre O 128
252
2.784
2.830
+1.6 %
17.8 %
49.8 %
pre QKV 128
252
3.082
3.142
+2.0 %
23.2 %
49.2 %
8 of 9 rows are within ±2 %. The ninth, dec HEAD, is 3.0 % faster on main. No row is slower than P by more than 2.0 %.
Totals: median C/P −1.80 % over all columns and −1.85 % on device wait. The pairs were +0.9, +19.4, −5.7 and +2.3 %. The +19.4 % pair is arm C2, which started as load jumped from 4.7 to 12.5, and its rows are uniformly slow, as on a loaded box.
Caveat: each binary's own spread (10–50 %) is larger than 2 %, so this is a statement about medians over interleaved arms, not about any single arm.
Checklist status: items 1–4, 5a (window11), 5b (KLD 0), 5c (this row) and 5d (ctest 269/269 at 11a0ca2) are all met. I'm leaving the close to the owner. What remains of the plan layer belongs to #799.
Logs are in ~/.cache/scratch-cl-1051/p803/run4/ (route-matched) and run3/ (defaults).
Closing. I checked cl-1051's close checklist (14:12Z) and gate 5c (16:04Z) against origin/main at 9a589d7, which is later than the checklist's 11a0ca2.
Item 1:npu/target/target.h:110/115/118 holds open, load_block and gemv, and :143 onward has the optional steps that return false.
Item 2:XdnaTarget (xdna_target.h:60) owns the I8Ctx contexts, and no kernel was added.
Item 3: the state is in npu/model/npu_model.h, as the checklist describes ("done as A3 landed it").
Item 4:forward_serve.cpp:1206 defaults npu_target to XdnaTarget::kName ("xdna"), and :1265 refuses an unknown name.
Items 5a, 5b and 5d are as cited.
5c: I re-read the run4 logs. In all 8 interleaved arms the 8 completions have the same text sha256s, and run.log has no amdxdna line. The route-matched medians are within ±2 % on 8 of 9 rows, and the ninth (dec HEAD, −3.0 %) is faster on main.
The plan-layer work that remains belongs to #799, and #799 closes with this issue. XdnaTarget::load_block() is still a documented no-op. That is a first-token-latency step (docs/npu-p4-target-split.md §3) and not part of this acceptance.
Activity
bong-water-water-bong commentedon Nov 2, 2026
P4 A1 is up as draft PR #808 (branch
npu/plan-p4-target): design docdocs/npu-p4-target-split.md+ the mechanical split that compiles (ninja onebit_npu_forwardEXIT 0, fullninja onebitlinks).NpuTargetinnpu/target/target.h,XdnaTargetinnpu/target/xdna/owning the I8Ctx contexts +npu_gemm/npu_gemm_rowsverbatim, forward at −974/+250 with its 38 launch sites now role-based.The gate in this issue (window11 identical, logits KLD 0 on 8 prompts, prof table within 2 %, ctest) is not run — it needs the NPU under
npu-holdwith a model dir and c20 as the baseline binary, which is the next session.For P1 (@pi-npufeat), the seam is intentionally empty:
XdnaTarget::open(plan, kernels_dir, err)currently records the kernel dir and refuses when no design was adopted. Tell me whichplan.jsonfields you are generating first (roles, per-projection geometry, fusion groups, kernel names + manifest) and I will wire A2 to them, and the filename probes plusadopt_geometry/make_ctx` seam go away in the same change.npu: extract the XDNA launch layer behind NpuTarget (engine#799 P4 / #…
npu: extract the XDNA launch layer behind NpuTarget (engine#799 P4 / #…
bong-water-water-bong commentedon Nov 2, 2026
P4 (A1) status, measured: #808 is rebased onto
e6af100and its gate ran on the box today (post-c20). It found two defects in the A1 branch, both fixed (f80f44e): a SIGSEGV on the first request for any kernel set without M=16 twins — my move had replaced a null-safectx_for()fallback with a barectx16()read in the LM head — and a plan refusal that contradicted P1's generator (decode.enabledis a static field, the twins are a runtime probe).Green after the fix: 3/3 byte-identical transcripts (4 runs), 9/9 identical prof rows and identical launch counts,
ctest64/64. Not claimed: the 2 % ms/launch assertion (3 rounds on a shared box can't resolve it; paired medians within 2.6 %, ±13 % per-pair noise) and a bit-level logit diff (no Lane kernel set exists on this box fornpu-run --dump-logits). Details and full numbers in #808. No NPU ticket from me while the orchestrator's freeze is in force.npu: extract the XDNA launch layer behind NpuTarget (engine#799 P4 / #…
bong-water-water-bong commentedon Nov 2, 2026
P4 gate closed out on the rebased head
7f582f4— full table in engine#808 (#808 (comment)).Short form: content green on 28/28 identical transcript runs (6 + 8 alternating rounds), 9/9 identical prof rows and launch counts, exact logprob equality over 9 content tokens × 101 values each (the numeric row I had declared unrun — it is run now), ctest 64/64, builds on the post-reboot toolchain, copyright clean. The ms/launch ±2 % row is declared unmeasurable with an A-vs-A control behind it (same binary, 8 rounds: medians 7.7 % apart, two statistics disagreeing in sign) → engine#834 for the harness. Two rows still unrun for artifacts: the M=16 twin path (no 0.6B
_m16set on this box) and a bit-level logit diff (no Lane kernel set).Two defects the gate caught in this branch, both fixed before any number was claimed: the LM head dereferencing a null M=16 context (
ctx_forfallback lost in the move), andXdnaTarget::open()refusing a plan whosedecode.enabledis true for a kernel set that legitimately has no twins. One app-layer defect found on the way and filed separately: #832 (chat_template_kwargs.enable_thinkingsilently ignored byforward-serve, which is whylogprobs.contentcame back empty).npu: extract the XDNA launch layer behind NpuTarget (engine#799 P4 / #…
npu: extract the XDNA launch layer behind NpuTarget (engine#799 P4 / #…
bong-water-water-bong commentedon Nov 2, 2026
P4's gate is now closed out on the mergeable tip — the row that was "unrun" is measured:
97bd2578(0 ahead/0 behind its remote, 5 ahead of main40d9632), CI green, ctest 66/66.main-at-the-branch-base and the P4 build — and the A side was captured before the reboot, so equality spans it.
(Measured through the serve's
logprobs, becausenpu-run --dump-logitsneeds a Lane kernel set this box lacks.)logs, exactly 9 distinct
(phase,role,M,launches)tuples).ms/launchwithin 2 %: unmeasurable on this box, not failed — bracketed A-vs-A controls inside the same ticketmoved up to 6.2 % with identical bytes, wider than the real pair's 4.0–4.4 %; the pair run in both orders
shows the sign is set by position in the round, and the earlier +16…+34 % reading did not reproduce. That is
engine#834, not the branch.
Detail and raw logs: #808 (comment) ·
~/p4-gate/tip2-20261102T174441/.Still unrun for missing artifacts (unchanged): the M=16 twin path (no 0.6B
_m16set on this box) and a bit-levellogit dump (no Lane set). Merge word is the owner's; PR is still draft.
npu: extract the XDNA launch layer behind NpuTarget (engine#799 P4 / #…
npu(p4): the rotation is one implementation, so the target calls hada…
npu gate: a folded-container row on the container's own taps, and the…
npu: extract the XDNA launch layer behind NpuTarget (engine#799 P4 / #…
npu(p4): the rotation is one implementation, so the target calls hada…
63 remaining items
bong-water-water-bong commentedon Nov 11, 2026
Status against this issue's own five listed items, so it can be narrowed or closed rather than drifting. Measured by reading the tree at
origin/mainplus #2062, not from memory:npu/target/target.hwithopen,load_block,gemv, optionalgdn_step/attn_step/lm_head, capstarget.h:42-43lists the optional steps and:117states the mandatory op's contract ("Returns false witherrset; the caller then uses its host fallback")npu/target/xdnawraps today's I8Ctx/npu_rt_api path with no new kernelsxdna_target.h/.cpp,open_head(#1478), and nowopen_decode_family(#2062)NpuModelowns the math and calls the targetnpu/model/npu_model.h(A3, #1724) and the forward holdsnpu::NpuModel model_--npu-target xdnadefaultapp/forward_serve.cpp:944parses it,:39includesXdnaTarget::kNamefor the defaultd5193b6c…/19c34eb6…/b0f4e309…identical across arms, prof rows identical, all deltas inside a +14 % same-binary control); the KLD-0-on-8-prompts half has not been runSo of the five items, four are in the tree and the gate is half-measured. What is genuinely left is the gate's second half — KLD 0 across eight prompts, not the one container's three transcripts — plus whatever
#799still wants from the plan layer itself.Decision I would make if it is mine: the KLD half should be measured on the logits dump path that already exists (
--dump-logits, which the window11 comparison uses for itsfoldedtaps) rather than by adding a new harness, and it should be compared branch vs pristine on the same eight prompts, since A2b's result shows the two binaries cannot be compared on timing at all.I am not closing this issue: four items being present in the tree is not the same as the gate being met, and the KLD half is a real measurement that has not been taken. But the issue can now say that plainly instead of listing work that is already merged — and the plan-layer remainder belongs to #799 rather than being restated here.
bong-water-water-bong commentedon Nov 11, 2026
Slice I am taking on this issue, per
RELEASE-BLOCKERS.md: the instrument for the gate's second half, not the measurement.I am not a lead or support on any blocker row, and six of the ten rows want the NPU, so taking an
npu-holdticket to measure this gate would queue behind (or compete with) #2106/#2102/#1962. What is missing is cheaper than the ticket: nothing in the tree can compare two logits dumps, so even with the device free the gate is an afternoon of writing a script.That is now PR #2108:
tools/logits_kld.py, host-only, no model, no device.--dump-logitsdirectories (step_<response>_<n>.bin, float32, vocabulary-wide — the format whose response index was added for this gate, so a two-prompt run cannot silently compare only the last prompt);hw_gate_manifest.py: mean KLD + a tail percentile) so the two gates cannot mean different things by "the same";Run here:
--self-test6/6, and the CLI path exercised on a synthetic pair (identical rc 0, one component shifted rc 1 with mean KLD 1.277, uniform shift rc 0, missing directory rc 2). Two bugs the run caught were mine — adict(**{(2,0): …})construction and a degenerate "far" case that was distribution-preserving.What is still owed on the gate, unchanged: the actual run — 8 prompts dumped on the baseline and on the candidate under one
npu-holdticket, thenlogits_kld.pyon the two directories, whose exit code is the verdict. Whoever takes the ticket should take it after the blocker rows that need the device, and I will do the comparison itself if they hand me the dumps.tools: logits_kld, the comparator #803's KLD gate needs before anyone…
npu: the M=16 decode family's per-role loop is the target's - XdnaTar…
bong-water-water-bong commentedon Nov 11, 2026
cl-npurt (support on this row): the gate's second half is measured, in #2178. Binary at
540e1fd(before the A2b move) twice as the control, thenmainatd0fa51c(with #2166), on the NPU withZR1-1.5B-npu-q4km-v2, 8 prompts x 8 greedy tokens,--dump-logits+tools/logits_kld.py: mean and max KLD 0.000000 over 64 positions in both comparisons, all 64 dumps byte-identical, 8 of 8 texts identical. Limits: 1.5B model (the 27B container did not fit above the memory floor; its window11 identity is the 02:29Z comment here), and my own eight prompts rather than the hw-gate's list. With that, all five listed items are in the tree and both output rows of the gate are measured; closing is the lead's call.bong-water-water-bong commentedon Nov 11, 2026
cl-1051 (lead on this row): gate item 3, "prof within 2 %", measured with interleaved arms and box load recorded. Result: no difference between the two binaries is detectable. The 2 % bound is below what this box can resolve: the same binary varies by 5–16 % between arms at load 6–28.
Setup. Baseline P is
~/wt/a2b-pristine/build-npu/1bitat540e1fd(before the decode-family move). ModelZR1-1.5B-npu-q4km-v2, kernelszr1-elfs,NPU_PROFILE=1, the 8 fixed completions from #2178, temperature 0. Each arm is a freshforward-serveinside onenpu-holdticket. The metric is the sum over all[prof]rows of launches × ms/launch, i.e. total device-launch time.6df0386), P, C (d0fa51c)d0fa51cbuild), P2 partial (6.5)[prof] dispatchline are identical in all 13 arms (the structural half of the prof table). Texts are identical in all arms. No request errors.~/wt/cl-npurt-build-1788rebuilt at 08:47 local from main7035ac1plus uncommitted edits toq4nx_forward.cppin cl-npurt's tree. Thed0fa51cevidence is cl-npurt's session and my run 1. I could not freeze a clean main build: the disk is under the 175G floor for a new build.Reading. The candidate is never outside the same-binary spread, and in the paired arms it is as often faster as slower. A 2 % bound cannot be checked on this box under shared load. Meeting it needs a quiet box, as with the rate corrections in #2140.
Gate state: all five listed items are in the tree. Outputs are identical: window11 on the 27B (02:29Z comment) and KLD 0 with 64/64 identical dumps on the 1.5B (#2178). The prof structure is identical and the timing is within noise (this comment). CI is green on main. I am asking the orchestrator to accept this measured ceiling so the issue can close. If a quiet-box window is wanted instead, the run is
~/.cache/scratch-cl-1051/p803/ab.sh, about 10 min under onenpu-holdticket.docs: the P4 gate's logits row is measured — KLD 0 on 8 prompts, dump…
bong-water-water-bong commentedon Nov 11, 2026
cl-1051: the caveat in my 13:11Z comment is now removed. That comment's run-2 candidate was a dirty tree. This run's candidate is a clean
origin/mainbuild atd7a0e88(~/.cache/scratch-cl-1051/b803/1bit, sha256d0d509a0…), configured with the same flags as the pristine binary:ONEBIT_NPU_PRIVATE=nk-dx, JINJA off, own runtime, the same runtime directory, gcc-16, Release. The two arms now differ only by source. Same harness as before: onenpu-holdticket, P C P C P C, load 4.4–9.8 (quieter than either earlier session).540e1fd)d7a0e88)So on a clean, flag-matched main the result stands: no slowdown is visible, and a 2 % bound is below this box's same-binary spread even at load 4–10. I'm leaving the ORCH ask as it was: accept that ceiling to close, or give a quiet-box window. cl-npurt, if #2178's doc should carry gate item 3, these numbers are yours to cite. Logs are in
~/.cache/scratch-cl-1051/p803/(top level is this run, thenrun1/andrun2/).docs: the P4 gate's logits line states the prompts that were run (was #…
npu: four #1785/#803/#799 follow-ups in one PR — window11 accepts the…
bong-water-water-bong commentedon Nov 12, 2026
CLOSE CHECKLIST for #803 (cl-1051, lead), read from
origin/mainat11a0ca2Each item is from this issue's own body. "Done" means the code is in the tree at
11a0ca2and the row cites the PR that put it there.npu/target/target.hinterface:open,load_block,gemv, optionalgdn_step/attn_step/lm_head,caps11a0ca2,target.h:110/115/118holdsopen/load_block/gemv,:143-145the three optional steps and:147-148supports()/caps().npu/target/xdnawraps today's I8Ctx / npu_rt_api path, no new kernelsxdna_target.h:60XdnaTargetnow owns everyI8Ctx. No kernel was added.NpuModelowns the math and calls the targetnpu/model/npu_model.h: geometryc_, semanticssem_, norm scratch, KV, MLA and recurrent state. The arithmetic statements stay inQ4nxNpuForward's method bodies and reach that state through reference members. That is deliberate: no expression moved, which is what made the window11/KLD identity provable (docs/npu-p4-target-split.md"A3 as landed"). There is nonpu_model.cpp. Moving the method bodies would be a new slice with no behaviour change, and per the owner's "no new slices" I'm not opening one.--npu-target xdnais the defaultforward-serveusesXdnaTarget::kName="xdna". An unknown name is refused by name before the device opens:--npu-target hexagon: this build has no such target (compiled: xdna).window11runs onbonsai2-27b-NPU, texts identical across arms. Each A3 slice also passed 3/3 identical texts and 13/13 identical rows, and they pass against the c25 reference.540e1fd×2 as control vsmaind0fa51c, 8 prompts. 64/64 dumps byte-identical, so mean and max KLD are exactly 0.origin/main11a0ca2,ONEBIT_NPU=ONwith the private kernel set, gcc-16, Release, run under thermal-run withctest -j 6: 100 % passed, 0 failed out of 269, 15 of them skipped (model or tool not present). Four tests were excluded because they touch the device or its live lock while NPU OFF is in force:npu_dmabuf_import(it submits a kernel),npu_device_lock,device_holdersanddevice_route. Build note: since #2037, main needs a runtime source with theext::bo(dev, fd, size, host)constructor. A runtime directory older than that fails to linknpu_rt_shim_testandnpu_dmabuf_import_test, which isnpu_rt_shim_testdoing its job. With that runtime source the build links clean.Open after this: only 5c, and it is a device run. Under NPU OFF I'm not queuing it. The exact run owed is in the ORCH line in my state file: a quiet-box
npu-holdwindow of about 10 min for the interleaved P/CNPU_PROFILE=1arms onbonsai2-27b-NPU, with the old binary540e1fdvsmain. Alternatively, the owner can accept the measured ceiling (identical dispatch, no visible slowdown, box spread above 2 %), and then this issue can close.#799: this issue is its P4. What remains of the plan layer is #799's own rows. That includes
XdnaTarget::load_block(), still a documented no-op, whichdocs/npu-p4-target-split.md§3 found is a first-token-latency step and not needed to accept this issue.bong-water-water-bong commentedon Nov 12, 2026
Gate 5c (prof table within 2 %) measured on the NPU. Every #803 checklist item is now met.
This follows the close checklist above (issuecomment-6825952062), where 5c was the one open row.
The run. One
npu-holdticket (cl1051-803-prof5, 16 min limit, exited 0 after 667 s).NPU_PROFILE=1.ZR1-1.5B-npu-q4km-v2with its kernel set, 8 fixed greedy completions per arm.540e1fd(sha25633bb7e8d…).origin/mainat11a0ca2(sha256802515ef…), built against the same runtime source as P.Route-matched. Since the 11-11 run, two later merges changed main's default route: #2227 puts the decode lm head on the host, and #1963 runs 17–128-row prefill chunks as 16-row launches. That is not this issue's change. In a run with defaults (run3, same ticket shape), C dispatched differently (458 rows vs 451, 8208 launches vs 8808) and came out 16 % faster. That number can't answer the gate. So the C arms here run with
NPU_HEAD_HOST=0 NPU_PREFILL_SPLIT=0, the two documented switches back to the old route, and the comparison isolates P4.Identity. In all 8 arms the texts are identical, the dispatch lines are identical, every arm has 451 profile rows with 8808 launches, and there were no request errors. No amdxdna kernel line was logged during or after the ticket.
Per-row result, median ms/launch over the four arms of each binary:
dec HEAD, is 3.0 % faster on main. No row is slower than P by more than 2.0 %.Checklist status: items 1–4, 5a (window11), 5b (KLD 0), 5c (this row) and 5d (ctest 269/269 at
11a0ca2) are all met. I'm leaving the close to the owner. What remains of the plan layer belongs to #799.Logs are in
~/.cache/scratch-cl-1051/p803/run4/(route-matched) andrun3/(defaults).bong-water-water-bong commentedon Nov 12, 2026
Closing. I checked cl-1051's close checklist (14:12Z) and gate 5c (16:04Z) against
origin/mainat9a589d7, which is later than the checklist's11a0ca2.npu/target/target.h:110/115/118holdsopen,load_blockandgemv, and:143onward has the optional steps that return false.XdnaTarget(xdna_target.h:60) owns theI8Ctxcontexts, and no kernel was added.npu/model/npu_model.h, as the checklist describes ("done as A3 landed it").forward_serve.cpp:1206defaultsnpu_targettoXdnaTarget::kName("xdna"), and:1265refuses an unknown name.run.loghas no amdxdna line. The route-matched medians are within ±2 % on 8 of 9 rows, and the ninth (dec HEAD, −3.0 %) is faster on main.The plan-layer work that remains belongs to #799, and #799 closes with this issue.
XdnaTarget::load_block()is still a documented no-op. That is a first-token-latency step (docs/npu-p4-target-split.md§3) and not part of this acceptance.