Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 18 additions & 1 deletion npu/pack_import.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1650,7 +1650,17 @@ FconstGeom locate_fconst_with_header(const std::string& tok, const std::string&
// engine#1831: only when the vote did NOT record the block's headers. When it did, its list is the measurement
// (`plan()` reads 2-header vs 4-header from `hdr.size()`, and `post_attn_hdr()` is `hdr.back()`), and rewriting it
// to one entry turned every Qwen3 4-header 1.9 block into a 2-header one whose "post-attention" header was k_norm's.
if (!g.hdr.empty()) return g;
if (!g.hdr.empty()) {
/* engine#1985: the vote's list is a measurement, but it is not the only witness. A pack that declares per-head q/k
* gains cannot have a 2-header block, so a one-header vote on such a pack is the #1831 defect coming back through
* the vote instead of through the residue step -- measured: Qwen3-1.7B read as 2-header, qk_norm true -> false, 280
* refused projections, where main imported it with 0 the day before. Refuse naming both rather than pick. */
if (const std::string why = header_witness_refusal(int(g.hdr.size()), dims.use_qk_norm); !why.empty()) {
g.why = why;
g.stride = 0;
}
return g;
}
const std::vector<int64_t> pos = eps_header_positions(tok, size_t(1) << 30, dims.eps_bits);
// The scan window is the first GiB (about 8 blocks at h3584), so a real section boundary repeats ~8 times against
// 1-2 for a payload coincidence; 3 separates them (requiring `layers/2` was too strict and dropped the real one).
Expand Down Expand Up @@ -2779,6 +2789,13 @@ QkGainOffsets qk_gain_offsets(int64_t gap_start, int64_t head_dim, int64_t gap_b
return o;
}

std::string header_witness_refusal(int vote_hdrs, bool declared_qk) {
if (!declared_qk || vote_hdrs != 1) return std::string();
return "the epsilon-anchor vote recorded ONE in-block header, and this pack declares use_qk_norm, whose per-head q/k "
"gains live in the two sections a 2-header block does not have (engine#1985). The two witnesses disagree and "
"neither is being preferred";
}

std::string qk_norm_refusal(bool declared, bool found) {
if (declared && !found)
return "this pack declares use_qk_norm and its per-head q/k gains are not where the block layout says "
Expand Down
13 changes: 13 additions & 0 deletions npu/pack_import.h
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,19 @@ struct QkGainOffsets {
QkGainOffsets qk_gain_offsets(int64_t gap_start, int64_t head_dim, int64_t gap_bytes);
std::string qk_norm_refusal(bool declared, bool found);

// Why a pack's two layout witnesses cannot be reconciled, or "" when they can (engine#1985).
//
// `vote_hdrs` is the number of in-block headers the epsilon-anchor vote recorded (0 when it could not win); `declared_qk`
// is config.json's `use_qk_norm`. A pack that states per-head q/k gains keeps them in the two sections between q/k/v and
// o_proj, which a 2-header block does not have -- so a ONE-header vote on such a pack is self-contradictory, and engine#1985
// is what trusting it costs: `Qwen3-1.7B` read as a 2-header block, `qk_norm` flipped true -> false, and every attention
// projection was refused (280 gaps) where yesterday's main imported the same pack with 0.
//
// Neither witness wins: the vote measured where anchors repeat, and the declaration states what the block must contain.
// The reader refuses and names both, because picking either one silently is the defect. Exposed so the three cases can be
// pinned without a 2.6 GB pack.
std::string header_witness_refusal(int vote_hdrs, bool declared_qk);

// A hybrid pack's embedding table lives in its own graph (`embedding.onnx` + `.data`, gathered with
// GatherBlockQuantized). Returns [V, H] f32 row-major, empty when the pack has no such component, with `why` naming the
// reason. Exposed so the dequantisation can be tested against known codes and scales instead of only on a 3 GB pack.
Expand Down
24 changes: 24 additions & 0 deletions tests/npu_pack_import_test.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -4267,7 +4267,31 @@ void test_qk_gain_offsets() {
check(n4.k_norm == 0 && n4.q_norm == 0, "qk gains: the byte count alone is not enough, head_dim has to agree");
}

/* engine#1985: a REGRESSION, not a new bug -- `Qwen3-1.7B` imported with 0 gaps on yesterday's main and refused with 280
* on today's, because #1838's early return trusted an epsilon-anchor vote that had recorded ONE in-block header on a block
* that has three. The tell in the report is `qk_norm: true -> false`: a pack that declares per-head q/k gains keeps them in
* the two sections a 2-header block does not have, so the two witnesses disagree and neither may be preferred silently.
*
* The arms below pin the refusal and, just as importantly, the three cases it must NOT fire on: a vote that found three
* headers (a correct 4-header pack), a genuine 2-header pack that declares nothing, and no vote at all (0), which is the
* residue step's case and not a disagreement. */
void test_header_witness_refusal() {
const std::string a = pi::header_witness_refusal(1, true);
check(!a.empty(), "header witness: a one-header vote on a pack declaring use_qk_norm is refused");
check(a.find("engine#1985") != std::string::npos, "header witness: the refusal carries its issue");
check(a.find("use_qk_norm") != std::string::npos, "header witness: it names the declaration it contradicts");
check(a.find("ONE in-block header") != std::string::npos, "header witness: and what the vote measured");
check(a.find("neither is being preferred") != std::string::npos,
"header witness: and that no witness is being chosen, which is the point");
check(pi::header_witness_refusal(3, true).empty(), "header witness: a three-header vote is consistent with the declaration");
check(pi::header_witness_refusal(1, false).empty(), "header witness: a 2-header pack that declares nothing is fine");
check(pi::header_witness_refusal(0, true).empty(),
"header witness: NO vote is not a disagreement (the residue step decides it)");
std::printf(" header witness: 8 arms (one-header vote vs declaration refuses; 3/1-without-declaration/0 allowed)\n");
}

int main(int argc, char** argv) {
test_header_witness_refusal();
test_encode_tile_code_range();
test_qk_norm_refusal();
test_qk_gain_offsets();
Expand Down
Loading