Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions docs/npu.md
Original file line number Diff line number Diff line change
Expand Up @@ -1875,6 +1875,13 @@ a stream that was sent after the generation had ended; reasoning that the templa
decision-only pack. "What ends an answer" and [npu-chat-template.md](npu-chat-template.md) describe the rules
that came out of them.

**Refused by name at load: the `lfm2` packs.** `LFM2.5-1.2B-NPU2` has all four dense kernel shapes on the box
and its container loads, but ten of its sixteen layers are gated short convolutions (`model.layers.N.shortconv.*`),
which the loader reads and the forward does not run. The serve exits at load with `forward init failed:
model.q4nx has 10 gated short-convolution layer(s) of 16 (the first is model.layers.0.shortconv.*, LFM2's hybrid),
a layer kind this forward does not implement …`. Before that refusal existed it ran those layers as attention,
failed its warm-up with a size error that named neither the layer nor the kind, and announced itself ready.

Not established by any row here: that a pack goes through the importer (`npu-pack` still reports these
containers as recognised, not importable), anything about a vision or audio tower, and any pack whose geometry
has no kernel set on the box. Of the 57 `*-NPU2` repos with a readable geometry on 2026-11-11, 19 had all four
Expand Down
16 changes: 16 additions & 0 deletions npu/forward/include/model.h
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,22 @@ static inline int model_tensor_name_alt(const char* name, char* out, size_t cap)
memcpy(out + tl, rest, rl + 1);
return 1;
}
// How many of the first `n_layers` layers are gated short-convolution layers (LFM2's hybrid: a layer is one iff it
// carries shortconv.in_proj, see LayerWeights), and, through `first`, the index of the first of them (-1 if none).
// The loader reads those tensors; whether anything RUNS them is the forward's business, and a forward that does not
// must refuse the pack by name instead of running a convolution layer as attention (engine#1786: FastFlowLM's
// LFM2.5-1.2B-NPU2 loaded, failed its warm-up with "MLP dequant : data_size 0 ...", and was announced as ready).
static inline int model_shortconv_layers(const ModelWeights* mw, int n_layers, int* first) {
int count = 0;
if (first) *first = -1;
if (!mw || !mw->layers) return 0;
for (int l = 0; l < n_layers; l++) {
if (mw->layers[l].shortconv_in_proj_weight.shape[0] == 0) continue;
if (count == 0 && first) *first = l;
count++;
}
return count;
}
void model_print_info(ModelWeights* mw);
int model_find_tensor(const char* name, ModelWeights* mw);

Expand Down
16 changes: 16 additions & 0 deletions npu/forward/src/q4nx_forward.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -936,6 +936,22 @@ bool Q4nxNpuForward::init_impl(xrt::device* dev, const char* model_path,
std::to_string(mw_->config.num_layers) + " layers in model.q4nx";
return false;
}
// A layer kind nothing here implements is refused BY NAME (the same rule as an unknown layer_types entry below).
// LFM2 declares its hybrid through `full_attn_idxs`, not `layer_types`, so that check cannot see it: its ten
// gated short-convolution layers carry no self_attn.* tensors, and run as attention they fail in the first
// dequant with a size error that names neither the layer nor the kind.
{
int first_conv = -1;
const int n_conv = model_shortconv_layers(mw_, cfg.num_layers, &first_conv);
if (n_conv > 0) {
err_ = "model.q4nx has " + std::to_string(n_conv) + " gated short-convolution layer(s) of " +
std::to_string(cfg.num_layers) + " (the first is model.layers." + std::to_string(first_conv) +
".shortconv.*, LFM2's hybrid), a layer kind this forward does not implement (it runs "
"full_attention, sliding-window and linear/GDN layers), so serving it would run those layers as "
"attention; file the kind you need instead of converting it";
return false;
}
}

// forward dims mirror the config the weights were loaded with
c_.H = cfg.hidden_size;
Expand Down
31 changes: 31 additions & 0 deletions tests/npu_forward_loader_test.c
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,37 @@ int main(void) {
model_free(m);
}
}
{ /* LFM2's hybrid: a layer with shortconv.in_proj is a convolution layer, and the count names the first */
const char* h =
"{\"model.layers.0.self_attn.q_proj.weight\":{\"dtype\":\"BF16\",\"shape\":[4],\"data_offsets\":[0,8]},"
"\"model.layers.1.shortconv.in_proj.weight\":{\"dtype\":\"BF16\",\"shape\":[4],\"data_offsets\":[8,16]},"
"\"model.layers.1.shortconv.conv.weight\":{\"dtype\":\"BF16\",\"shape\":[4],\"data_offsets\":[16,24]},"
"\"model.layers.1.shortconv.out_proj.weight\":{\"dtype\":\"BF16\",\"shape\":[4],\"data_offsets\":[24,32]},"
"\"model.layers.2.self_attn.q_proj.weight\":{\"dtype\":\"BF16\",\"shape\":[4],\"data_offsets\":[32,40]},"
"\"model.layers.3.shortconv.in_proj.weight\":{\"dtype\":\"BF16\",\"shape\":[4],\"data_offsets\":[40,48]},"
"\"model.norm.weight\":{\"dtype\":\"BF16\",\"shape\":[4],\"data_offsets\":[48,56]}}";
ModelWeights* m = load(write_file("lfm2.q4nx", strlen(h), h, 56));
check(m != NULL, "a container with short-convolution layers loads (the loader reads them)");
if (m) {
int first = -2;
check(model_shortconv_layers(m, 4, &first) == 2 && first == 1,
"two of its first four layers are convolution layers, and the first is layer 1");
check(model_shortconv_layers(m, 1, &first) == 0 && first == -1, "none among the layers before it");
check(model_shortconv_layers(m, 4, NULL) == 2, "the count does not need the index");
model_free(m);
}
const char* plain = "{\"model.layers.0.self_attn.q_proj.weight\":{\"dtype\":\"BF16\",\"shape\":[4],\"data_offsets\":[0,8]}}";
ModelWeights* q = load(write_file("dense.q4nx", strlen(plain), plain, 8));
if (q) {
int first = -2;
check(model_shortconv_layers(q, 4, &first) == 0 && first == -1, "an attention-only container has none");
model_free(q);
} else {
check(0, "an attention-only container loads");
}
int first = -2;
check(model_shortconv_layers(NULL, 4, &first) == 0 && first == -1, "no model, none");
}
{ /* shorter than the length prefix */
char path[128];
snprintf(path, sizeof path, "%s/short.q4nx", dir);
Expand Down
Loading