[IE] UnrollGroupQuantize: make unrolled slice locations unique per consumer - #302
Conversation
|
Follow-up: I filed #303 documenting the next VPUX compiler blocker the same model (Qwen3-0.6B, asymmetric per-group INT4) hits once this PR is applied — a sub-byte (u4) per-group zero-point SEGMENTED slice that |
|
Note: I've closed #303 — it surfaces on per-group asymmetric INT4, which the dev team flagged as unsupported on NPU (openvinotoolkit/openvino#34450). This PR is narrower: it fixes a |
…nsumer
GenericUnrollBase::splitValue named each unrolled slice after the value being
split (appendLoc(val.getLoc(), "slice_{idx}")). When one value feeds more than
one unrolled consumer -- e.g. a grouped-INT4 per-group scale that is the
dequantization scale of both the weight and the activation DynamicDequantize --
the low-index slices of that shared value get byte-identical locations across
consumers, and StopLocationVerifierPass aborts with "Found N duplicated names
after full verification".
Derive each slice's location from the consuming op plus an operand tag and chunk
index (takeOpLoc(consumerOp, "{operandTag}_slice_{idx}")) so slices of a shared
value stay unique. The per-chunk reduced FakeQuantize/DynamicDequantize ops keep
their consumer-rooted "slice_{idx}" locations (already unique by construction).
Adds a LIT regression test (unroll_group_quantize_shared_param_locations.mlir)
covering a shared scale feeding two DynamicDequantize consumers.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
4983451 to
1c00f41
Compare
|
Rebased onto current The rebase. One conflict, and it happened to be on the exact line this PR rewrites: UD2026.28 The correction. This PR originally cited an end-to-end compile failure — Qwen3-0.6B So that justification is withdrawn. What does still hold, and is why I've kept the PR open rather than closing it: the underlying %2 = IE.Slice %arg2 [0, 0, 0] [1, 1536, 1] ... loc(#loc28) // "weight_dq"
%12 = IE.Slice %arg2 [0, 0, 0] [1, 1536, 1] ... loc(#loc28) // "matmul_dq"
#loc28 = loc(fused[#loc7, #loc10]) #loc10 = loc("slice_d0_0")The new LIT test fails on unpatched That reframes this from "fixes a compile blocker" to "fixes a latent invariant violation, with a Two practical notes:
Separately, and not part of this PR: building AI assistance disclosure: this work was AI-assisted (Claude Code) for the analysis, the |
Details:
Problem.
UnrollGroupQuantize(via the sharedGenericUnrollBase::splitValue) names eachunrolled slice after the value being split — currently
appendLoc(val.getLoc(), "slice_d{axis}_{idx}"). When a single value feeds more than oneunrolled consumer — e.g. a grouped-INT4 per-group scale that is the dequantization scale of
both the weight
DynamicDequantizeand the activation/matmulDynamicDequantize— theslices of that shared value get byte-identical locations across the two consumers, which
violates the location-uniqueness invariant
StopLocationVerifierPassenforces.This still reproduces on current
develop. Running the pass on the reduction added by this PR,the two consumers' slices of the same shared scale resolve to the same location:
The
d{axis}component added in UD2026.28 does not separate this case: both consumers unroll theshared value along the same axis, so the axis tag is identical on both sides.
Fix. Derive each slice's location from the consuming op plus an operand tag, keeping the
existing axis and chunk index —
takeOpLoc(consumerOp, "{operandTag}_slice_d{axis}_{idx}").Slices of a shared value then stay unique because they are rooted at the (distinct) consumers,
while the axis tag continues to do its own job of separating one value split along several axes.
All
splitValuecall sites are updated to pass the consumer + operand tag. The per-chunk reducedops (FakeQuantize / DynamicDequantize) keep their consumer-rooted
slice_{idx}locations, whichare already unique by construction.
With the fix, the same two slices become:
Test. New LIT test
tests/lit/NPU/dialect/IE/passes/unroll_group_quantize_shared_param_locations.mlir: a sharedscale feeding two
DynamicDequantizeconsumers; asserts each consumer's slices resolve to afused location rooted at that consumer + operand tag. It fails on unpatched
developand passeswith this change.
Validation:
Manual validation — the automatic checks on this PR have never run (fork PRs appear to require a
maintainer to approve workflows). Per CONTRIBUTING's allowance for manual results:
vpux-opt --init-compiler="platform=NPU4000" --unroll-group-quantize --mlir-print-debuginfo | FileCheckdevelop: FAIL (FileCheck exit 1)4089686065a245d648cdd2b99c31884f53cb7a5e(thecommit pinned by
validation/openvino_config.jsonondevelop), npu_compiler at this PR'shead, RelWithDebInfo, MSVC 19.44, Ninja. Hardware: Intel Core Ultra 7 258V (Lunar Lake),
NPU 4000.
Correction to this PR's original claim:
This PR originally reported that it unblocked an end-to-end compile of Qwen3-0.6B grouped-INT4
via the NPUW-LLM path, which had been aborting with
StopLocationVerifierPass Pass failed : Found 40 duplicated names after full verification.That is no longer true on current
develop, and I am withdrawing it. I re-ran the originalreproducer (same model, same harness, same NPU_USE_NPUW/NPUW_LLM configuration) against
developwith and without this change. The two runs are equivalent: neither reports duplicatednames, both proceed past
StopLocationVerifierPass, and both fail later at the same place —UnrollDistributedOps Pass failed : Can't convert 20 Bit to Byte(
vpux/utils/core/mem_size.hpp:128), which is the sub-byte lowering limitation on thisconfiguration, not something this PR addresses.
So something between UD2026.20 and UD2026.28 changed the pipeline such that this particular model
no longer reaches the colliding configuration. I did not trace which change.
What this PR is now: a fix for a location-uniqueness violation that demonstrably still exists
in
develop, plus a regression test that fails without it — not a fix for a currently-observablecompile blocker. If you would rather address the invariant differently, it can be reshaped or
closed.
Tickets:
AI Assistance:
splitValuerefactor, the LITregression test, the rebase conflict resolution, and the before/after experiment design.
produced by building both configurations from source and running them on real hardware
(Intel Core Ultra 7 258V / NPU 4000). The before/after comparison was a single-variable control
— only
unroll_group_quantize.cppwas reverted, in the same build tree with identical flags —and was checked to have actually rebuilt (differing binaries) before the results were trusted.