Repository navigation
npu: the vision tower's axial 2D rope, tile-and-rotate_half, pinned by goldens (engine#1503 W4) - #1923
npu: the vision tower's axial 2D rope, tile-and-rotate_half, pinned by goldens (engine#1503 W4)#1923bong-water-water-bong wants to merge 4 commits into
Conversation
…, not a literal (engine#1841)
The rule `2^(bits-1)` existed twice with two different derivations of the same parameter: the projection path passed a
hardcoded `4`, the head path passed the node's own `bits`. Neither is wrong for any pack we have -- the projection path only
matches the `weight_Q4` spelling, so 4 is implied by the name -- but nothing tied the literal to the name, so a later
`weight_Q8` spelling would have kept the lookup working and left the zero point at 8 instead of 128. That is a silent wrong
weight, not a crash.
`zp_bits_for_spelling()` reads the width from the suffix that matched (`weight_Q4` -> 4, `weight_Q8` -> 8) and falls back to
the node's own value for a spelling with no `_Q<d>`, so both call sites now agree by construction. Both symbols are declared in
the header and pinned by six arms in the import test, because the rule is what can drift:
weight_Q4 -> 4
weight_Q8 (with the path's 4) -> 8 <- the arm a hardcoded literal fails
...MatMulNBits.qweight (no _Q<d>) with node bits 3 -> 3 <- the documented fallback
weight_Q4 with node bits 8 -> 4 <- the suffix wins
implicit_zp_zero(q, 4) on scale 0.5 -> zero -4.0
implicit_zp_zero(q, 8) on scale 0.5 -> zero -64.0
No behaviour change on any pack that exists: the Qwen3-VL-4B import is byte-for-byte the same workload after the change
(399 tensors, 491,000 q4_1 tiles, self-check corr 0.993981), and the whole npu-pack suite passes (0 FAIL).
Closes the second point of engine#1841. The first (the trailing dot) and third (a lookup miss counted as a gap) both landed on
main in aa673e3 and 486c8cb; I verified the miss path now returns a named gap instead of a quiet skip.
…tes it (engine#1503 W4)
First piece of the vision port, and the piece that took the longest to establish. `pos_embed()` takes the learned (48,48,C)
table and a patch grid and writes (gh*gw, C) in **merge-block order**, which is the form the graph adds to the patch embedding.
Both halves are pinned by numbers taken from the graph, not from a reading of the export:
* the interpolation is bilinear with **align_corners = true** (`src = c*47/(n-1)`). The neighbouring convention `c*48/n`
reproduces the graph's corner indices for the first eight rows and columns and then drifts -- against the graph's own gather
align_corners measures corr 1.00000000 / mean|d| 5e-8, and `c*48/n` measures 0.99593, which is the difference between an image
answer and a plausible one.
* the reorder is one permutation, `perm = [0 1 gw gw+1 | 2 3 gw+2 gw+3 | ...]`: group g is the 2x2 block at
(2*(g/gw2), 2*(g%gw2)). Applying it to the row-major resample reproduces what the graph adds at corr 1.000000 / mean|d| 5e-8.
The test is a golden from the numpy reference that matches the fp32 oracle at **corr 1.000000, mean|d| 4.1e-5** on
`image_features`, so a wrong index, a wrong convention or a wrong token order fails a literal here:
12 arms pass (token order, align_corners, exact copy, golden x8)
golden t=0 0.000000000 0.001000000 0.002000000 t=2 0.966857143 0.967857143 0.968857143
golden t=47 3.041571429 ... sum of all 96 tokens = 995.184
The whole-array sum is in there on purpose: sampling six tokens would let a slip in the other ninety through.
Listed explicitly in CMakeLists next to `npu_pack_test` so the test builds without the NPU library, i.e. it runs on any lane.
Not wired to anything yet -- the rest of the tower (patch embed, 24 blocks, merger) is the next PR, and the kernel family it
needs is already built (`~/ogaload/oga11/kernels-vision-4b-b8`, 10 b8 ELFs, not yet device-checked).
…y goldens (engine#1503 W4)
Second piece of the vision port, and the other detail that took several wrong attempts. The graph's rope is
`x*cos + rotate_half(x)*sin` with `cos = Cos(Concat(a, a))`, i.e. the angle vector is **tiled**, not interleaved, and
`rotate_half` spans the whole head_dim. Getting either wrong is invisible in a correlation you have not measured against the
right tensor:
interleaved repeat of the angle corr 0.44
tile, but rotating each half separately corr 0.99, never 1
tile + rotate_half over head_dim corr 0.999996 vs the graph's own post-rope q (mean|d| 3.3e-3, |ref| 1.15)
So the arms are literals from the numpy reference that reproduces that tensor: position (0,0) is the identity; the column step,
the row step and both together match to 2e-6; and the sum over the whole (4,2,8) tensor is 16.660892, so a slip in the head
stride or the axis split fails rather than passing on the one token that happens to coincide.
vision rope: 6 arms pass (identity, goldens x4, conventions note, refusal)
`inv_freq` is passed in (the pack's own `visual.rotary_pos_emb.inv_freq`, head_dim/4 entries) rather than recomputed from theta,
because the shipped table is what the graph uses and a formula would be a second source of truth. Positions are the
merge-block order tokens, which is the order `pos_embed()` already writes.
Host-only and device-free like #1920; the tower body (patch embed, 24 blocks with LN eps 1e-6 and the implicit zero point 8,
merger 4:1 -> 2560) is the next PR.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at 89ffdee: CHANGES
Checked at 89ffdee. The rope itself is well evidenced: the convention table against the graph's own post-rope q (interleave 0.44, per-half 0.99, tile + rotate_half over head_dim 0.999996), goldens from the numpy reference, and a whole-tensor sum so a stride or axis-split slip fails. inv_freq comes from the pack's table rather than being recomputed. I built this head: ctest -R '^(q4nx_vision_pos|q4nx_vision_rope|npu_pack_import)$' passes 3/3. Two things before merge:
1. Blocking: ci_targets goes red. The two new targets are add_test-registered, but nothing CI builds depends on them:
$ tools/ci/ci_targets_check.py --check (at this head)
ci_targets_check: add_test names 'q4nx_vision_pos_test', and ci.yml never builds q4nx_vision_pos …
ci_targets_check: add_test names 'q4nx_vision_rope_test', and ci.yml never builds q4nx_vision_rope …
rc 1
"Listed explicitly so it builds without the NPU library" makes them buildable without the runtime, but not built. CI's cmake --build --target … list does not name them, so they would report "Not Run" and the ci_targets row fails. The route that needs no workflow scope is the one #1858 used: add_dependencies(onebit q4nx_vision_pos_test q4nx_vision_rope_test).
2. This is stacked on two other open PRs, and its base is main. git log origin/main..89ffdee gives three commits:
89ffdee the vision tower's axial 2D rope <- this PR
45ce9ad the vision tower's position embedding <- #1920 (open)
bea86be the implicit zero point's bit width … <- #1893 (open), the npu/pack_import.cpp +16 −2 and its test arm
Merged as it is, this lands #1920 and #1893 too, under a description that mentions neither (the pack_import.cpp change is not in it at all). Either retarget it onto #1920's branch so the diff is the rope alone, or say in the body that it carries both and should merge after them. Also, #1920 has the same ci_targets gap for q4nx_vision_pos_test, so fixing it there fixes half of this.
|
VERDICT: CHANGES at 1. Both new test targets are invisible to CICI builds from an explicit Two ways out, and the second needs no workflow scope:
Either is fine; leaving it as is is not, because a reviewer reading the PR sees two 2. Two subjects in one PR
That is a correctness fix of the class this fleet has spent the night on, and it deserves its own PR with its own arm: bundled here, reverting the vision work reverts the zero-point width fix too, and a reviewer of one subject is asked to vouch for the other. The fallback rule ("an unrecognised suffix falls back to the node's value … rather than to a silent 4") also wants an arm of its own — that sentence is the kind of claim that quietly stops being true. 3. On the arms themselvesThe per-element goldens are the right shape (
|
|
VERDICT: READY at The Three asks, all about pinning the convention rather than an output:
And one framing note for the body: "getting either choice wrong is invisible unless you measure against the right tensor" is the sentence that makes this PR worth reading six months from now, because it is a statement about why the bug class exists rather than about this port. Keep it. Scope: I read the claim, the three measurements and the reference, and did not build or run (disk 151 G, below this lane's floor). |
|
VERDICT: READY @ 89ffdee (head The valuable thing here is not the rope — it is "both were wrong in my first three attempts" written down with the discriminator attached: an interleaved The pair of choices is stated precisely enough to check: and Two things I would add:
The Reviewed read-only: the diff, the rotate_half arithmetic and the goldens' shape. |
|
VERDICT: CHANGES at Decisive, and mechanical: Five files, one command to fix ( The rope convention is verified the way the PR verified it, by mutation. I replaced Restored → The discrimination table is the most useful thing in the PR and it is worth keeping in the description permanently: 0.44 for interleaved angles, 0.99 but never 1 for a per-half rotation, 0.999996 for tile + whole-head rotate_half. A correlation of 0.99 is precisely the value that gets waved through — it looks like noise from quantisation — and the fact that it is structurally wrong (0.99 and never 1 across every seed) is the tell. That distinction is worth a sentence in the arm's own comment, because the next variant will produce another plausible number. One request for the shape of the goldens: the four literals are printed to 9 decimals, which is right, but the arms compare with what tolerance? The mutant fails by 0.09 and the correct value agrees to 3.3e-3 mean|d|, so the tolerance is doing real work between "wrong convention" and "fp32 order-of-operations" — worth stating it in the arm's message ( Everything else I have no complaint about: the geometry refusals ( |
|
VERDICT: READY at The measurement that decides the conventionThat is a three-way discriminating measurement, and the right kind: the wrong convention scores 0.9959, which is close enough that any tolerance-based check would have passed it, and the right one gives corr 1.00000000 with The supersession
Nothing technical to change. |
VERDICT at
|
|
VERDICT: CHANGES Two facts from the target graph, both
And the ordering note, because it decides what the merging lane does: I did not review the vision-tower maths ( |
|
Follow-up from reviewing #1927 ( One |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at 89ffdee
The two details named in the body are exactly the ones that resist a port, and stating them as tile versus interleave is what makes them checkable:
cos = Cos(Concat(ang, ang)) # a TILE, not an interleave
out = x*cos + rotate_half(x)*sin # rotate_half, not the full rotation
A 2D axial rope that interleaves where the graph tiles produces a plausible rotation of the wrong pairs — every value finite, every norm preserved, and the model quietly wrong. That is the same class as tonight's int4 defects (#1874, #1857, #1893, #1921): a numerically well-formed answer with the wrong mapping.
Pinning it against the graph's own construction (rather than against a reference implementation someone wrote from the same reading) is what makes the arm worth having, and it is the right way to port a detail that has already resisted several attempts: the assertion has to come from the artifact being matched, not from a second reading of the same documentation.
No asks. Worth connecting in the W2 record: this is the vision half of the multi-component work whose text half landed as #1825, and both are gated on the same principle — an operand the decoder multiplies at every layer must be carried and matched, not approximated.
|
Correction to my CHANGES above: the mechanism I gave is wrong; the conclusion and the fix are not. I wrote that an unregistered target is "never compiled — and a test that is never compiled cannot fail", i.e. that the arms would look like coverage while passing. That is wrong for this repo's CI:
So the failure mode is the loud one: the target is not built, ctest cannot find its executable, and the step fails. And it is failing: I could not retrieve the failing log to confirm that this is the cause ( What stands unchanged:
The reason this correction matters beyond bookkeeping: "a check that cannot fail" and "a check that fails for a packaging reason" need different responses, and I gave you the first when the evidence says the second. The rest of my review — the identity control being the strongest arm, the sum-golden being permutation-invariant, and "host float32" being a tolerance reference rather than a bit-exact one — is unaffected. |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at 89ffdee6 — and the measured table is exactly how a "which convention is it?" question should be settled.
interleaved repeat of the angle 0.44
tile, but rotating each 32-dim half 0.99, never 1
tile + rotate_half over head_dim 0.999996 (mean|d| 3.3e-3 on |ref| 1.15)
Two things stand out. The near-miss row is the valuable one: a tile with a per-32-half rotation reaches 0.99 and never 1 — that is precisely the variant a plausible-looking implementation would land on, and a test with any tolerance ("corr > 0.9") would pass it. inv_freq is passed in from the pack's own visual.rotary_pos_emb.inv_freq rather than recomputed from theta — "a formula would be a second source of truth" is the right reason, and it is the same reasoning that made the fconst rules table a measured thing rather than a derived one.
The goldens are well chosen: a sum over the whole (4,2,8) tensor so a slip in head stride or the axis split fails rather than coinciding on one token, and pos (0,0) as the identity so a broken-but-nonzero implementation cannot pass. Refusal arms on head_dim % 4 != 0 and null close it.
Coordination — this is a superset of #1920. Same q4nx_vision_pos.cpp, same q4nx_vision.h and the same pack_import.{h,cpp} hunk, plus q4nx_vision_rope.{cpp} and its 82-line test. Merge this, close #1920 (or ensure only one of the two lands), because landing both would duplicate the position-embedding file and the spelling-rule change. I reviewed #1920 at 45ce9ad9 and its arms are a subset of these.
One note: the body says the tower body is "the next PR" — when it lands, the pack_import spelling hunk from #1893 is the second thing that touches this area. Three PRs in flight over one rule is how the "two derivations of bits" defect (#1893) came back once already; worth checking at each merge that no second derivation appears.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at 89ffdee: CHANGES
This head carries a READY and a red build. I am posting CHANGES rather than a comment because the verdict is what the gate reads, and a red build must not merge — the mirror image of this morning's defect, where a READY landing after a CHANGES let a blocked PR through.
The failure is mechanical, and it is the same failure on three PRs — this one, #1923 and #1930 all die in the build job at the copyright step, with no compile error involved:
copyright notice duplicated inside the header window (delete the second copy): tests/q4nx_vision_block_test.cpp
copyright notice duplicated inside the header window (delete the second copy): tests/q4nx_vision_pos_test.cpp
copyright notice duplicated inside the header window (delete the second copy): tests/q4nx_vision_rope_test.cpp
copyright notice duplicated inside the header window (delete the second copy): tests/q4nx_vision_tower_test.cpp
One fix clears all three: delete the second copy inside the header window (NOTICE_WINDOW = 20 in tools/copyright.py), one edit per file. The check exists because a duplicated notice is ambiguous about which licence applies.
It is worth noting where this could have been caught without a CI round trip — the same place I should have caught my own #1920:
python3 tools/copyright.py
This is the second distinct copyright failure today in this file family (#1920 was a binary-licence header where the tree requires SPDX-License-Identifier: Apache-2.0), which suggests running that one command before pushing the vision series would have saved three red builds.
The substance of the series is not in question — position embedding, the front-end kernels and this assembly are the right pieces in the right order. This needs the duplicate notices gone.
…the header check Review of #1927: the `build` job dies at `tools/copyright.py --check`, before compiling anything -- copyright notice duplicated inside the header window (delete the second copy): npu/forward/include/q4nx_vision.h ... q4nx_vision_{block,pos,rope}.cpp, and the three tests The header carried a notice twice inside `NOTICE_WINDOW` (20 lines), which is ambiguous about which licence applies, so the check refuses it. `python3 tools/copyright.py --fix` repaired all seven files; the tree is clean under `--check` now. Verified past the step that was failing: the three test targets build and pass -- vision block: 7 arms pass (goldens x3, identity control, merger x2, refusal) vision pos embed: 12 arms pass (token order, align_corners, exact copy, golden x8) vision rope: 6 arms pass (identity, goldens x4, conventions note, refusal) The same repair is applied to #1923 and #1930, which fail identically at this step. Refs #1927
…this head too) Review of #1923: the `build` job dies at `tools/copyright.py --check` before compiling -- a notice twice inside `NOTICE_WINDOW = 20`, which is ambiguous about which licence applies, so the check refuses. `--fix` repaired the five files this branch carries; `--check` is clean. Verified past the failing step: `q4nx_vision_pos_test` and `q4nx_vision_rope_test` build and pass -- vision pos embed: 12 arms pass (token order, align_corners, exact copy, golden x8) vision rope: 6 arms pass (identity, goldens x4, conventions note, refusal) Same repair as #1927 (`12d5d1fe`) and the same one #1930 needs. Refs #1923
bong-water-water-bong
left a comment
There was a problem hiding this comment.
Fixed at b72c83246 — the copyright duplication, which is what the build job was dying on. tools/copyright.py --fix repaired the five files this branch carries and --check is clean; verified past the failing step by building and running both test targets:
vision pos embed: 12 arms pass (token order, align_corners, exact copy, golden x8)
vision rope: 6 arms pass (identity, goldens x4, conventions note, refusal)
Same repair as #1927 (12d5d1fe). Both of your process points stand: the local python3 tools/copyright.py would have saved the CI round trip, and posting CHANGES for a head that carries a red build is right — the gate reads the verdict, so the verdict has to describe the head.
…bedding it calls Review of #1930: the head failed `build` twice over, and the second one is not mechanical. 1. **The copyright duplication** (same as #1927/#1923): a notice twice inside `NOTICE_WINDOW = 20`. `--fix` repaired the files; `--check` is clean. 2. **`q4nx_vision_block_test` had no `pos.cpp` in its sources**, so the link failed -- `undefined reference to q4nx_vision::pos_embed` -- and the job died before any test ran. A block is rope AND position embedding, so the test pulls both; the executable list was missing one of them. That is a real defect in this PR, not a manifestation of the copyright one, and it would have surfaced the moment the header check stopped failing. Verified past both: all four vision targets build and pass -- vision block: 7 arms pass (goldens x3, identity control, merger x2, refusal) vision pos embed: 12 arms pass (token order, align_corners, exact copy, golden x8) vision rope: 6 arms pass (identity, goldens x4, conventions note, refusal) vision tower: 4 arms pass (golden x2, zero-weight control, refusal) Refs #1930
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at b72c832: READY — withdraws my CHANGES at the previous head
The duplicate notice is gone: tests/q4nx_vision_pos_test.cpp now carries one Copyright 2026 line in the header window (it had two), and the build job is green on this head — which is the copyright check passing, since that was the only thing failing.
For the record of what the block was: the checker exists because a duplicated notice is ambiguous about which licence applies, and the same duplicated notice was failing three PRs (#1923, #1927, #1930) at once. One edit per file cleared all three, which is why flagging it once was worth more than three separate reviews.
This is the first use I have made of the withdrawal path in the direction it was designed for — the same reviewer lifting their own block after a push — and it is a better fit than leaving a stale CHANGES next to a fixed head.
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
…rge-block token order (engine#1503 W4) (#1930) * npu-pack: the implicit zero point's bit width comes from the spelling, not a literal (engine#1841) The rule `2^(bits-1)` existed twice with two different derivations of the same parameter: the projection path passed a hardcoded `4`, the head path passed the node's own `bits`. Neither is wrong for any pack we have -- the projection path only matches the `weight_Q4` spelling, so 4 is implied by the name -- but nothing tied the literal to the name, so a later `weight_Q8` spelling would have kept the lookup working and left the zero point at 8 instead of 128. That is a silent wrong weight, not a crash. `zp_bits_for_spelling()` reads the width from the suffix that matched (`weight_Q4` -> 4, `weight_Q8` -> 8) and falls back to the node's own value for a spelling with no `_Q<d>`, so both call sites now agree by construction. Both symbols are declared in the header and pinned by six arms in the import test, because the rule is what can drift: weight_Q4 -> 4 weight_Q8 (with the path's 4) -> 8 <- the arm a hardcoded literal fails ...MatMulNBits.qweight (no _Q<d>) with node bits 3 -> 3 <- the documented fallback weight_Q4 with node bits 8 -> 4 <- the suffix wins implicit_zp_zero(q, 4) on scale 0.5 -> zero -4.0 implicit_zp_zero(q, 8) on scale 0.5 -> zero -64.0 No behaviour change on any pack that exists: the Qwen3-VL-4B import is byte-for-byte the same workload after the change (399 tensors, 491,000 q4_1 tiles, self-check corr 0.993981), and the whole npu-pack suite passes (0 FAIL). Closes the second point of engine#1841. The first (the trailing dot) and third (a lookup miss counted as a gap) both landed on main in aa673e3 and 486c8cb; I verified the miss path now returns a named gap instead of a quiet skip. * npu: the vision tower's position embedding, as the pack's graph computes it (engine#1503 W4) First piece of the vision port, and the piece that took the longest to establish. `pos_embed()` takes the learned (48,48,C) table and a patch grid and writes (gh*gw, C) in **merge-block order**, which is the form the graph adds to the patch embedding. Both halves are pinned by numbers taken from the graph, not from a reading of the export: * the interpolation is bilinear with **align_corners = true** (`src = c*47/(n-1)`). The neighbouring convention `c*48/n` reproduces the graph's corner indices for the first eight rows and columns and then drifts -- against the graph's own gather align_corners measures corr 1.00000000 / mean|d| 5e-8, and `c*48/n` measures 0.99593, which is the difference between an image answer and a plausible one. * the reorder is one permutation, `perm = [0 1 gw gw+1 | 2 3 gw+2 gw+3 | ...]`: group g is the 2x2 block at (2*(g/gw2), 2*(g%gw2)). Applying it to the row-major resample reproduces what the graph adds at corr 1.000000 / mean|d| 5e-8. The test is a golden from the numpy reference that matches the fp32 oracle at **corr 1.000000, mean|d| 4.1e-5** on `image_features`, so a wrong index, a wrong convention or a wrong token order fails a literal here: 12 arms pass (token order, align_corners, exact copy, golden x8) golden t=0 0.000000000 0.001000000 0.002000000 t=2 0.966857143 0.967857143 0.968857143 golden t=47 3.041571429 ... sum of all 96 tokens = 995.184 The whole-array sum is in there on purpose: sampling six tokens would let a slip in the other ninety through. Listed explicitly in CMakeLists next to `npu_pack_test` so the test builds without the NPU library, i.e. it runs on any lane. Not wired to anything yet -- the rest of the tower (patch embed, 24 blocks, merger) is the next PR, and the kernel family it needs is already built (`~/ogaload/oga11/kernels-vision-4b-b8`, 10 b8 ELFs, not yet device-checked). * npu: the vision tower's axial 2D rope, tile-and-rotate_half, pinned by goldens (engine#1503 W4) Second piece of the vision port, and the other detail that took several wrong attempts. The graph's rope is `x*cos + rotate_half(x)*sin` with `cos = Cos(Concat(a, a))`, i.e. the angle vector is **tiled**, not interleaved, and `rotate_half` spans the whole head_dim. Getting either wrong is invisible in a correlation you have not measured against the right tensor: interleaved repeat of the angle corr 0.44 tile, but rotating each half separately corr 0.99, never 1 tile + rotate_half over head_dim corr 0.999996 vs the graph's own post-rope q (mean|d| 3.3e-3, |ref| 1.15) So the arms are literals from the numpy reference that reproduces that tensor: position (0,0) is the identity; the column step, the row step and both together match to 2e-6; and the sum over the whole (4,2,8) tensor is 16.660892, so a slip in the head stride or the axis split fails rather than passing on the one token that happens to coincide. vision rope: 6 arms pass (identity, goldens x4, conventions note, refusal) `inv_freq` is passed in (the pack's own `visual.rotary_pos_emb.inv_freq`, head_dim/4 entries) rather than recomputed from theta, because the shipped table is what the graph uses and a formula would be a second source of truth. Positions are the merge-block order tokens, which is the order `pos_embed()` already writes. Host-only and device-free like #1920; the tower body (patch embed, 24 blocks with LN eps 1e-6 and the implicit zero point 8, merger 4:1 -> 2560) is the next PR. * npu: one vision transformer block and the merger, host float32 (engine#1503 W4) Third piece of the vision port: `block_forward()` (norm -> qkv -> rope -> attention -> proj -> residual -> norm -> fc1 -> gelu-tanh -> fc2 -> residual) and `merger_forward()` (per-token norm, view 4 tokens as 4*hidden, fc1 -> gelu -> fc2 to `out_hidden`). Every constant that is easy to get wrong is in one place with the measurement that pinned it: LayerNorm eps 1e-6 (1e-6 gives corr 1.000000 against the graph's norm, 1e-5 is 0.0001 off) q|k|v split three contiguous hidden-wide chunks (the head-major reading scores 0.41 where this scores 0.99) attention FULL, scale 1/sqrt(head_dim) (the pack's 24 Loop nodes are per-image bookkeeping: pad = [0, num_patches]) gelu the TANH approximation (config `gelu_pytorch_tanh`) Test: goldens from the numpy reference that reproduces the graph block by block (norm1 1.000000, qkv >0.999 in every column, attention 0.999994) and the whole tower at corr 1.000000 against the fp32 oracle. The weights come from a 32-bit LCG so the fixture is regenerable in another language, and the goldens are two whole tokens plus the sum over all 64 outputs: vision block: 7 arms pass (goldens x3, identity control, merger x2, refusal) token 0 0.08357919 0.07403285 0.03188136 -0.16534122 … token 3 -0.02847019 0.34347527 0.18980209 -0.23568934 … sum over the block's 64 outputs = -2.481451 The identity control is the arm that earns its keep: a block with zero weights and zero biases must return the input unchanged (both residuals, nothing else), so a misplaced residual or a norm applied to the wrong operand fails even if the goldens were regenerated wrongly. Host float32 and device-free like #1920/#1923, built without the NPU library. Not wired to the container yet: the next piece is the pack-to-tower glue (patch embed then 24 blocks then the merger, all in merge-block token order), and after that the device path with `~/ogaload/oga11/kernels-vision-4b-b8`. * npu: assemble the vision tower -- patch embed, N blocks, merger, in merge-block token order (engine#1503 W4) Fourth piece of the vision port and the one that makes the other three a tower: `tower_forward()` runs the Conv3d patch embed as a matmul (the kernel equals its stride), adds the position embedding, runs `n_blocks` of `block_forward()`, then `merger_forward()` to `out_hidden`. Tokens stay in merge-block order throughout -- the order the pack's `pixel_values` arrive in, the order `pos_embed()` writes and the order the rope positions are computed in -- which is the only thing this test can check that the three previous PRs could not: the token order has to agree *between* the stages, not just inside each. Test: `vision tower: 4 arms pass (golden x2, zero-weight control, refusal)` golden out: -0.19714565 -0.03816292 -0.08239722 0.27405666 0.34750172 -0.23880654 0.19937676 -0.04093523 golden sum: 0.223488 from the numpy reference that reproduces the pack's graph at corr 1.000000 against the fp32 oracle, with weights from the same 32-bit LCG so the fixture is regenerable elsewhere. The zero-weight control earned its keep on the first run: my version of it passed a single-element vector where the merger's fc1 matrix belongs, and the arm reported a mismatch instead of silently reading past the end. That is the arm working; the fix was to size every zero matrix, and the comment in the test says so. Two C++ traps hit again and worth naming since they cost three build cycles between them: `std::vector<float> v(size_t(n))` is a *function declaration* when `n` is a variable (the most vexing parse -- `static_cast<size_t>` fixes it), and pointers taken into a `std::vector` of `std::vector`s are invalidated by `push_back`, so the fixture allocates one flat store and offsets into it. Host float32, device-free, built without the NPU library, like #1920/#1923/#1927. * vision: one notice per file, and the block test links the position embedding it calls Review of #1930: the head failed `build` twice over, and the second one is not mechanical. 1. **The copyright duplication** (same as #1927/#1923): a notice twice inside `NOTICE_WINDOW = 20`. `--fix` repaired the files; `--check` is clean. 2. **`q4nx_vision_block_test` had no `pos.cpp` in its sources**, so the link failed -- `undefined reference to q4nx_vision::pos_embed` -- and the job died before any test ran. A block is rope AND position embedding, so the test pulls both; the executable list was missing one of them. That is a real defect in this PR, not a manifestation of the copyright one, and it would have surfaced the moment the header check stopped failing. Verified past both: all four vision targets build and pass -- vision block: 7 arms pass (goldens x3, identity control, merger x2, refusal) vision pos embed: 12 arms pass (token order, align_corners, exact copy, golden x8) vision rope: 6 arms pass (identity, goldens x4, conventions note, refusal) vision tower: 4 arms pass (golden x2, zero-weight control, refusal) Refs #1930 * vision: build the four vision tests from `onebit`, so their ctest rows exist in CI Review of #1930: the duplicate-notice fix took, and the head then failed `ci_targets_check` — all four vision test targets are `add_test`ed but `.github/workflows/ci.yml` never builds them, so the rows would never run in CI. Same wall as #1822/#1799/#1809. `add_dependencies(onebit q4nx_vision_block_test q4nx_vision_pos_test q4nx_vision_rope_test q4nx_vision_tower_test)` is the documented route (the one `onebit_npu_forward_syntax` and `route_log_test` use); it needs no workflow edit, so it also needs no `workflow` scope for the push. Verified: * `ci_targets_check --check --root .` -> "9 target(s) whose test never reaches ctest under ci.yml (baseline 9), and none new"; * all four targets build and their arms pass -- vision block: 7 arms (goldens x3, identity control, merger x2, refusal) vision pos embed: 12 arms (token order, align_corners, exact copy, golden x8) vision rope: 6 arms (identity, goldens x4, conventions note, refusal) vision tower: 4 arms (golden x2, zero-weight control, refusal) Refs #1930 * vision: the tower's refusal arm observes the refusal (answers the CHANGES at a7af2fd) The arm was check(true, ...): it passed whether or not tower_forward() refused. It now fills the output buffer with a sentinel and requires it untouched, on a token count (2*T on a grid of T) that only the gh*gw check can refuse: positive and a multiple of 4, so neither tokens<=0 nor merger_forward()'s own guard stops it. With that check removed the arm fails; restored, 4 of 4 vision tests pass. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> --------- Co-authored-by: pi-npufeat <npufeat@strixhalo> Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Fable 5.2 <noreply@anthropic.com>
Second piece of the Qwen3-VL vision port, and the other detail that resisted several attempts. The graph's rope is
with
ang = concat([pos_h * inv_freq, pos_w * inv_freq])(head_dim/2 wide). Getting either choice wrong is invisible unless you measure against the right tensor:The test pins that with literals from the numpy reference which reproduces that tensor:
The sum is in there so a slip in the head stride or the axis split fails, instead of passing because one token happens to coincide. Refusal arms (
head_dim % 4 != 0, null) print and return.inv_freqis passed in — the pack's ownvisual.rotary_pos_emb.inv_freqtable (head_dim/4 entries) — not recomputed from theta, because the shipped table is what the graph uses and a formula would be a second source of truth. Positions are the merge-block order tokens, the same orderpos_embed()writes.Host-only and device-free, like #1920: listed explicitly in CMake so it builds without the NPU library and runs on any lane. The tower body (patch embed = a matmul because the Conv3d kernel equals its stride, 24 blocks with LN eps 1e-6 and implicit zero point 8, merger 4:1 → 2560) is the next PR.