Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions docs/arch-gaps.md
Original file line number Diff line number Diff line change
Expand Up @@ -1087,3 +1087,14 @@ the single undocumented entry out of 37.
MLA-style geometry (`kv_lora_rank` 512, `q_lora_rank` 1536, `qk_rope_head_dim` 64, `v_head_dim` 128, 32 heads /
16 KV, IM 4096). **Not an alias of `deepseek_v41`**: nothing at the pin decodes AW-SECD weights, so mapping it to
that architecture would name a backend that cannot read the file. Reported, not alerted (`"watch": "report"`).

## 2026-11-11: `OpensciForCausalLM` — reviewed remote code (engine#2171)

* `OpensciForCausalLM` (`model_type` `opensci`, 121 models in the 2026-11-11 census) — the open-sci reference models
(for example `open-sci/open-sci-1.7b-nemotron-cc-1T-oh-dcft`: hidden 2048, 24 layers, 32 heads / 32 KV,
`qk_layernorm`, vocab 50304). **Published as remote code**: `config.json` carries an `auto_map` to its own
`configuration_opensci.OpensciConfig` and `modeling_opensci.OpensciForCausalLM`, so no pinned backend accepts the
class without that code. **Not an alias** of any supported family. Reported, not alerted (`"watch": "report"`).
The other spelling, `OpenSciForCausalLM` (7 models, same `model_type` and the same `auto_map`), is a separate exact
key: `census_watch.py` matches architecture classes exactly, so that spelling needs its own entry and is not covered
by this one.
28 changes: 28 additions & 0 deletions docs/npu.md
Original file line number Diff line number Diff line change
Expand Up @@ -1916,6 +1916,34 @@ route loads and a captured `--device npu` run stands behind it; the regenerated
GGUF-on-NPU forward), while every D3-* pack in this body declares the dense `qwen3_5` family, so
the two are not the same answer and the registry keeps them apart.

The answers are also machine-readable, in `registry/flm_answers.json`, because CI keeps no watcher state: every scheduled
run sees the whole org as new, and before that file each family answered above printed `!! NPU FLM` again and re-filed
the census issue the day after it closed (engine#2171; the 2026-11-11 run alerted on 31 packs, every one already
answered or tracked). A pack whose declared architecture is listed there prints `. NPU FLM` / `. FLM` with the issue
that tracks it and does not alert; a family with no entry still alerts. The test fails on an entry for a family the
registry serves on the NPU route, so a family that starts to load drops its answer. Keyed by the architecture each
pack declares (a config with no `architectures` declares its `model_type`):

| declared architecture | packs on 2026-11-11 | answer | tracked by |
|---|---|---|---|
| `MiniCPMV4_7ForConditionalGeneration` | MiniCPM-V-4.7-1B, MiniCPM-V-4.7-35B-A3B | not served: a VLM class the pin does not register | engine#1788 |
| `MiniCPMOcrForConditionalGeneration` | MiniCPM-OCR-1.0 | not served: an OCR/vision pack; the route claims text decoders only | engine#1786 |
| `HYV3ForCausalLM` | Hy-Memory-30B-A3B | not served: `hy_v3`, hrx only | engine#1786 |
| `DFlashDraftModel` | Nemotron-3.5-Bolt-E4B-DFlash | not served: a draft head, not a decoder | engine#1780 |
| `Gemma4ForConditionalGeneration`, `Gemma4UnifiedForConditionalGeneration` | Nemotron-3.5-Bolt-E4B, Gemma4-E4B-IT, Gemma4-E2B-IT, Gemma4-12B-IT | not served: `gemma4`, hrx only | engine#1786 |
| `HunYuanDenseV1ForCausalLM` | Hy-MT2-1.8B | not served: `hunyuan-dense`, hrx only | engine#1786 |
| `Gemma3ForCausalLM`, `gemma3_text` | Gemma3-270M, Gemma3-1B, Gemma3-4B, Gemma3-4B-Text, medgemma-4b-it, medgemma-1.5-4b-it, Translategemma-4B-Instruct | not served: `gemma3`, hrx only | engine#1786 |
| `Lfm2ForCausalLM` | LFM2-1.2B, LFM2-2.6B, LFM2-1.2B-Tool, LFM2-2.6B-Transcript, LFM2.5-1.2B, LFM2.5-1.2B-Thinking | not in the registry's NPU set: `lfm2`, hrx only (LFM2.5-1.2B on the per-op route is engine#2170, which adds no `NPU_ARCHS` entry) | engine#1786 |
| `GptOssForCausalLM` | GPT-OSS-20B, GPT-OSS-Safeguard-20b | not served: `gpt-oss`, hrx only | engine#1786 |
| `Qwen3_5MoeForConditionalGeneration` | Qwen3.6-35B-A3B | not served for this class: the route serves `qwen3_5_moe` through `Qwen3_5MoeForCausalLM` (GGUF on the NPU); the VLM wrapper class is hrx only, and FastFlowLM's own pack is not measured on the route | engine#1786 |
| `Qwen2_5_VLForConditionalGeneration` | Qwen2.5-VL-3B-Instruct | not served: `qwen2vl`, hrx only (the route serves the text-only `qwen2` class) | engine#1786 |
| `Phi3ForCausalLM` | Phi4-mini-Instruct | not served: `phi3`, hrx only | engine#1786 |
| `Gemma3TextModel` | Embedding-Gemma-300M | not served: `gemma-embedding`, an embedding model, hrx only | engine#1786 |
| `WhisperForConditionalGeneration` | Whisper-V3-Turbo | not served: a speech encoder-decoder the registry does not map | engine#1783 |

`FastFlowLM/smolvla-NPU2` is answered by its repo id (`repos` in the same file): it has no config and no `*.q4nx`
file, so there is no architecture to key on and nothing for the route to load.

### FastFlowLM's own containers, served as published: measured rows (engine#1786)

The tables above answer "does the registry map this family". This one answers a different question for the packs
Expand Down
91 changes: 91 additions & 0 deletions registry/flm_answers.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
{
"about": "Recorded answers for FastFlowLM packs this engine does not serve, read by tools/census_watch.py (engine#2171). CI keeps no watcher state, so without this file every scheduled run re-alerted on families docs/npu.md had already answered. A pack whose declared architecture (or, for a repo with no config, whose id) is listed here prints `. NPU FLM` / `. FLM` with the issue that tracks it and does not alert. Hand-written: add an entry only with a row for it in docs/npu.md, and remove it when the family is served (tests/census_watch_test.py fails on an answer for a family the registry serves on the NPU route) or when the issue that tracks it closes without serving it (no test can see that: check the `issue` numbers here whenever one of them closes).",
"architectures": {
"MiniCPMV4_7ForConditionalGeneration": {
"answer": "not served: a vision-language pack whose class the pin does not register; serving it on the NPU route is engine#1788",
"doc": "docs/npu.md#fastflowlms-npu2-drops-the-families-the-route-does-not-serve",
"issue": 1788
},
"MiniCPMOcrForConditionalGeneration": {
"answer": "not served: an OCR/vision pack whose class the pin does not register; the route claims text decoders only",
"doc": "docs/npu.md#fastflowlms-npu2-drops-the-families-the-route-does-not-serve",
"issue": 1786
},
"HYV3ForCausalLM": {
"answer": "not served: hy_v3 is mapped for hrx only; no hy_v3 model type or kernels in the NPU route",
"doc": "docs/npu.md#fastflowlms-npu2-drops-the-families-the-route-does-not-serve",
"issue": 1786
},
"DFlashDraftModel": {
"answer": "not served: a draft head, not a decoder; running the DFlash drafts as NPU sources is engine#1780",
"doc": "docs/npu.md#fastflowlms-npu2-drops-the-families-the-route-does-not-serve",
"issue": 1780
},
"Gemma4ForConditionalGeneration": {
"answer": "not served: gemma4 is mapped for hrx only; no gemma4 model type or kernels in the NPU route",
"doc": "docs/npu.md#fastflowlms-npu2-drops-the-families-the-route-does-not-serve",
"issue": 1786
},
"Gemma4UnifiedForConditionalGeneration": {
"answer": "not served: gemma4 is mapped for hrx only; no gemma4 model type or kernels in the NPU route",
"doc": "docs/npu.md#fastflowlms-npu2-drops-the-families-the-route-does-not-serve",
"issue": 1786
},
"HunYuanDenseV1ForCausalLM": {
"answer": "not served: hunyuan-dense is mapped for hrx only",
"doc": "docs/npu.md#fastflowlms-npu2-drops-the-families-the-route-does-not-serve",
"issue": 1786
},
"Gemma3ForCausalLM": {
"answer": "not served: gemma3 is mapped for hrx only (medgemma, translategemma and the Gemma3 text packs declare it)",
"doc": "docs/npu.md#fastflowlms-npu2-drops-the-families-the-route-does-not-serve",
"issue": 1786
},
"gemma3_text": {
"answer": "not served: gemma3 is mapped for hrx only (medgemma, translategemma and the Gemma3 text packs declare it)",
"doc": "docs/npu.md#fastflowlms-npu2-drops-the-families-the-route-does-not-serve",
"issue": 1786
},
"Lfm2ForCausalLM": {
"answer": "not served by the registry yet: lfm2 is mapped for hrx only; LFM2.5-1.2B on the per-op route is engine#2170, which adds no NPU_ARCHS entry",
"doc": "docs/npu.md#fastflowlms-npu2-drops-the-families-the-route-does-not-serve",
"issue": 1786
},
"GptOssForCausalLM": {
"answer": "not served: gpt-oss is mapped for hrx only",
"doc": "docs/npu.md#fastflowlms-npu2-drops-the-families-the-route-does-not-serve",
"issue": 1786
},
"Qwen3_5MoeForConditionalGeneration": {
"answer": "not served for this class: the route serves qwen3_5_moe through Qwen3_5MoeForCausalLM (GGUF on the NPU); the VLM wrapper class FastFlowLM declares is mapped for hrx only, and its own Q4NX pack is not measured on the route",
"doc": "docs/npu.md#fastflowlms-npu2-drops-the-families-the-route-does-not-serve",
"issue": 1786
},
"Qwen2_5_VLForConditionalGeneration": {
"answer": "not served: qwen2vl is mapped for hrx only; the route serves the text-only qwen2 class",
"doc": "docs/npu.md#fastflowlms-npu2-drops-the-families-the-route-does-not-serve",
"issue": 1786
},
"Phi3ForCausalLM": {
"answer": "not served: phi3 is mapped for hrx only",
"doc": "docs/npu.md#fastflowlms-npu2-drops-the-families-the-route-does-not-serve",
"issue": 1786
},
"Gemma3TextModel": {
"answer": "not served: gemma-embedding (an embedding model, not a decoder) is mapped for hrx only",
"doc": "docs/npu.md#fastflowlms-npu2-drops-the-families-the-route-does-not-serve",
"issue": 1786
},
"WhisperForConditionalGeneration": {
"answer": "not served: a speech encoder-decoder the registry does not map; the Whisper encoder on our runtime is engine#1783",
"doc": "docs/npu.md#fastflowlms-npu2-drops-the-families-the-route-does-not-serve",
"issue": 1783
}
},
"repos": {
"FastFlowLM/smolvla-NPU2": {
"answer": "not a pack: a config-less VLA repo with no *.q4nx file, so there is no architecture to map and nothing for the route to load",
"doc": "docs/npu.md#fastflowlms-npu2-drops-the-families-the-route-does-not-serve"
}
}
}
5 changes: 5 additions & 0 deletions registry/significant.json
Original file line number Diff line number Diff line change
Expand Up @@ -172,6 +172,11 @@
"model_type": "deepseek_v4_flash_superexpert",
"family": "DeepSeek V4.1 Flash SuperExpert, published as remote code and with AW-SECD-compressed weights (8.5 GB, claimed 0.962 retention); MLA-style geometry (kv_lora_rank 512, q_lora_rank 1536, qk_rope_head_dim 64, v_head_dim 128, 32 heads / 16 KV, IM 4096). Not an alias of deepseek_v41: nothing at the pin decodes AW-SECD weights, so mapping it would name a backend that cannot read it",
"watch": "report"
},
"OpensciForCausalLM": {
"model_type": "opensci",
"family": "OpenSci (1.7B 'nemotron-hq' recipe). A REMOTE-CODE class: its config carries an auto_map pointing at modeling_opensci.py / configuration_opensci.py, so the pinned backends cannot accept the architecture without that custom code. Not an alias of any supported family, and not mappable until the code is vendored or upstreamed.",
"watch": "report"
}
}
}
12 changes: 9 additions & 3 deletions registry/support_pins.json
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,15 @@
"why": "The official gguf Q4_0/Q4_1 -> Q4NX converter with per-architecture configs (configs/*.json, 18 of them: gemma3, gemma4, gpt-oss, lfm2, llama, nanbeige, phi4, qwen2, qwen2vl, qwen3, qwen3.5_{0.8b,2b,4b,9b,a3b,omni}, qwen3vl). docs/npu.md's repack path runs it (scripts/repack_gguf.py, ONEBIT_Q4NX_CONVERTER) and fails its declared-scale guard when the converter or config drops a critical scale. A per-arch config here is what makes a family packable on our side at all. Reviewed 2026-11-08 (d1d5232 -> 93b8398, 2 commits, \"Qwen3.5 / Qwen3.5-MoE / Omni converters on stock llama.cpp GGUFs\"): three changes touch the family this route serves. (a) full-attention layers are now detected from `attn_q.weight` instead of \"every Nth layer\", because the d3 layouts are irregular -- the commit names full attention at 2, 5, 8, 9, 10, 11 for d3:edge, which is the same irregularity our own converter takes from `layer_types`; (b) an MTP block (nextn_predict_layers > 0, counted in block_count) no longer leaks out as a bogus extra decoder layer; (c) the Qwen35 value-head reorder is decided from the v/qk head ratio instead of feed_forward_length <= 6144, and a ratio > 2 raises. Stock Qwen3.5 0.8B/2B/4B (Q4_K, Q4_1) are tested 124/124 in that PR, and the output is byte-identical for GGUFs without an MTP block, so the repack path's declared-scale guard is unaffected; the entry point and configs/ layout that scripts/repack_gguf.py invokes are unchanged."
},
"vllm-project/semantic-router": {
"commit": "a9219240e6f9dd4d9e1bea83596565cc8f18989b",
"reviewed": "2026-11-09",
"why": "The Decision 3.0 (d3) source: the router calls d3 for routing, guardrails and tool selection, and FastFlowLM v1.0.11 serves those weights through /v1/systemone with the same HTTP API as the upstream d3 server. Engine #653 is our side of that; a change here is what a new d3 pack would follow. Reviewed 2026-11-08 (632f087 -> b7adcfa, 152 commits, 300 files): **the d3 contract is unchanged** -- `d3_format.py`, `d3_server.py` and `d3_runtime.py` are not in the delta at all, and the only System One files touched are dashboard frontend pages (`SystemOnePlaygroundPage.tsx`, `systemOneTargets.ts`). What moved is router-side: `config/fragments/signal/decision/vela{2,3}-labels-spans.yaml` (their own newer decision models), a `config/recipes/decision-balance/` recipe, and a `canonical_decision_model` CRD. So what this route must speak is unchanged, and the vela2/vela3 names are upstream's packs, not ours. Reviewed 2026-11-09 (b7adcfa -> a921924, 9 commits, engine #1667): the d3 contract is unchanged again -- no d3_* file is in the delta. The one System One file is the router's client, pkg/systemone/backend.go (#5144): a backend marked trusted_gateway is now refused before dispatch (errTrustedGatewayUnsupported), because System One dispatch does not carry the request's gateway credential. The wire protocol string stays vllm-sr/systemone@1, and FastFlowLM's /v1/systemone is not a trusted gateway, so nothing this route speaks or serves moves. The rest is router-side (header_mutation append, the lifecycle client's 5xx retry, contrastive jailbreak embedding validation, adaptive thinking on Responses backends, a repeated [DONE] in protocolcodec) plus a Claude Code plugin and maintainer docs."
"commit": "15b4fb5ad46b5c40a3bd99f35d0feb28997dac68",
"reviewed": "2026-11-11",
"why": "The Decision 3.0 (d3) source: the router calls d3 for routing, guardrails and tool selection, and FastFlowLM v1.0.11 serves those weights through /v1/systemone with the same HTTP API as the upstream d3 server. Engine #653 is our side of that; a change here is what a new d3 pack would follow. Reviewed 2026-11-08 (632f087 -> b7adcfa, 152 commits, 300 files): **the d3 contract is unchanged** -- `d3_format.py`, `d3_server.py` and `d3_runtime.py` are not in the delta at all, and the only System One files touched are dashboard frontend pages (`SystemOnePlaygroundPage.tsx`, `systemOneTargets.ts`). What moved is router-side: `config/fragments/signal/decision/vela{2,3}-labels-spans.yaml` (their own newer decision models), a `config/recipes/decision-balance/` recipe, and a `canonical_decision_model` CRD. So what this route must speak is unchanged, and the vela2/vela3 names are upstream's packs, not ours. Reviewed 2026-11-09 (b7adcfa -> a921924, 9 commits, engine #1667): the d3 contract is unchanged again -- no d3_* file is in the delta. The one System One file is the router's client, pkg/systemone/backend.go (#5144): a backend marked trusted_gateway is now refused before dispatch (errTrustedGatewayUnsupported), because System One dispatch does not carry the request's gateway credential. The wire protocol string stays vllm-sr/systemone@1, and FastFlowLM's /v1/systemone is not a trusted gateway, so nothing this route speaks or serves moves. The rest is router-side (header_mutation append, the lifecycle client's 5xx retry, contrastive jailbreak embedding validation, adaptive thinking on Responses backends, a repeated [DONE] in protocolcodec) plus a Claude Code plugin and maintainer docs. Reviewed 2026-11-11 (a9219240 -> 15b4fb5a, 88 commits; engine#2171): **the d3 contract is unchanged** -- `vela3_d3/package.py`, `model.py` and `decode.py` have the same blob at both commits. Three commits touch the family: #5886 adds router config fragments for the decision model (config/fragments/signal/systemone/*, no runtime code), #6113 records ROCm golden answers for the d3 sizes (a new registry/golden_answers_vela3_d3.json and the test that reads it), and #6192 versions the System One HTTP API (systemone/0.2, vllm-sr/5.0; vllm-sr/4.0 stays a selectable compatible superset, `/v1/models` now reports api_version 5.0.0, a request may send `roles`, and `roles: null` makes an object state the user's content for span questions). Its own description says model input stays byte-identical on every family. Nothing here reaches npu/systemone.h, which copies the pack's d3_format.py/d3_runtime.py (choice/noul/score prompts and answers; no span questions and no api_version field). From this review on the pin watches `paths` only: the d3 family directory, the runtime's systemone.py and api/ (the HTTP contract), and config/schemas/systemone; router-side commits no longer re-file the census issue every day.",
"paths": [
"src/model-runtime/vllm_srun/families/vela3_d3",
"src/model-runtime/vllm_srun/systemone.py",
"src/model-runtime/vllm_srun/api",
"config/schemas/systemone"
]
}
},
"d3_sources": {
Expand Down
Loading
Loading