bench: the Qwen3-1.7B same-pack pair against FastFlowLM's runtime becomes two leaderboard records

bench/npu-leaderboard/lfm2-12b-perop-decode-20261111.json

@@ -0,0 +1,28 @@
+{
+  "id": "lfm2-12b-perop-decode-20261111",
+  "host": "strixhalo",
+  "model": "LFM2-1.2B-NPU2",
+  "route": "npu",
+  "quant": "Q4NX symmetric 4-bit tiles (FastFlowLM's published container, listed as Q4_0), per-op int8 GEMM, blocked family",
+  "metric": {
+    "name": "decode",
+    "value": 4.28,
+    "unit": "tok/s"
+  },
+  "settings": {
+    "device": "accel0",
+    "engine": "1bit built at the code of main 1131acf0 (engine#2170)",
+    "container": "FastFlowLM/LFM2-1.2B-NPU2 model.q4nx, sha256 2755ab40cd927e3c…, served as published; the only edit is \"q4nx_weight_layout\": \"b8\" in a copy of config.json",
+    "kernels": "full_i8 blocked: QKV K2048 N3072, QKV K2048 N6144 (the conv layers' in_proj), O K2048 N2048, GU K2048 N16384, D K8192 N2048, each with its M=16 twin",
+    "ticket": "npu-hold -l cl-flm-lfm2-rows, both LFM2 packs in one 107 s ticket",
+    "sampling": "temperature 0, streamed, max_tokens 64",
+    "prompt_tokens": 20,
+    "load1": "17.5 at start, 14.7 at end",
+    "known_answer": "3 requests of 'What is the capital of France? Answer with one word.' returned 'Paris' three times, identical"
+  },
+  "instrument": "streamed chat completion of 64 tokens, usage read back; decode rate from the server's own timings; two requests, 4.25 and 4.30 tok/s, same text both times",
+  "repeats": 2,
+  "timestamp": "2026-11-11T13:32:47Z",
+  "source": "~/.cache/scratch-cl-flm/rows.log and out-rows-LFM2-1.2B-NPU2/ on strixhalo (row-serve.sh + row-probe.py, lane cl-flm; engine#1786)",
+  "notes": "A hybrid: 10 of 16 layers are gated short convolutions, run as two device GEMMs with the gate and a 3-tap causal conv on the host. This is the per-op route's speed, not what the array does on a geometry with a fused set: FastFlowLM 1.0.12's runtime decoded the same file at 42 to 83 tok/s in the paired tickets of the same day. What would make it wrong: two requests, box load 15 to 19 during the ticket (the same pack measured 3.7 tok/s in a paired ticket earlier that day)."
+}

bench/npu-leaderboard/lfm2-12b-perop-first-token-20261111.json

@@ -0,0 +1,28 @@
+{
+  "id": "lfm2-12b-perop-first-token-20261111",
+  "host": "strixhalo",
+  "model": "LFM2-1.2B-NPU2",
+  "route": "npu",
+  "quant": "Q4NX symmetric 4-bit tiles (FastFlowLM's published container, listed as Q4_0), per-op int8 GEMM, blocked family",
+  "metric": {
+    "name": "first token (warm)",
+    "value": 0.756,
+    "unit": "s"
+  },
+  "settings": {
+    "device": "accel0",
+    "engine": "1bit built at the code of main 1131acf0 (engine#2170)",
+    "container": "FastFlowLM/LFM2-1.2B-NPU2 model.q4nx, sha256 2755ab40cd927e3c…, served as published; the only edit is \"q4nx_weight_layout\": \"b8\" in a copy of config.json",
+    "kernels": "full_i8 blocked: QKV K2048 N3072, QKV K2048 N6144 (the conv layers' in_proj), O K2048 N2048, GU K2048 N16384, D K8192 N2048, each with its M=16 twin",
+    "ticket": "npu-hold -l cl-flm-lfm2-rows, both LFM2 packs in one 107 s ticket",
+    "sampling": "temperature 0, streamed, max_tokens 64",
+    "prompt_tokens": 20,
+    "load1": "17.5 at start, 14.7 at end",
+    "known_answer": "3 requests of 'What is the capital of France? Answer with one word.' returned 'Paris' three times, identical"
+  },
+  "instrument": "time from request to the first streamed chunk on a 20-token prompt; two requests, 0.747 and 0.764 s",
+  "repeats": 2,
+  "timestamp": "2026-11-11T13:32:47Z",
+  "source": "~/.cache/scratch-cl-flm/rows.log and out-rows-LFM2-1.2B-NPU2/ on strixhalo (row-serve.sh + row-probe.py, lane cl-flm; engine#1786)",
+  "notes": "Same ticket as lfm2-12b-perop-decode-20261111. Batched prefill, no prefix cache hit (a different prompt had been served before it). The server's own prompt_ms was 745 and 762 ms."
+}

bench/npu-leaderboard/lfm25-12b-perop-decode-20261111.json

@@ -0,0 +1,28 @@
+{
+  "id": "lfm25-12b-perop-decode-20261111",
+  "host": "strixhalo",
+  "model": "LFM2.5-1.2B-NPU2",
+  "route": "npu",
+  "quant": "Q4NX symmetric 4-bit tiles (FastFlowLM's published container, listed as Q4_0), per-op int8 GEMM, blocked family",
+  "metric": {
+    "name": "decode",
+    "value": 4.2,
+    "unit": "tok/s"
+  },
+  "settings": {
+    "device": "accel0",
+    "engine": "1bit built at the code of main 1131acf0 (engine#2170)",
+    "container": "FastFlowLM/LFM2.5-1.2B-NPU2 model.q4nx, sha256 b341d42021e617a5…, served as published; the only edit is \"q4nx_weight_layout\": \"b8\" in a copy of config.json",
+    "kernels": "full_i8 blocked: QKV K2048 N3072, QKV K2048 N6144 (the conv layers' in_proj), O K2048 N2048, GU K2048 N16384, D K8192 N2048, each with its M=16 twin",
+    "ticket": "npu-hold -l cl-flm-lfm2-rows, both LFM2 packs in one 107 s ticket",
+    "sampling": "temperature 0, streamed, max_tokens 64",
+    "prompt_tokens": 20,
+    "load1": "18.6 at start, 17.5 at end",
+    "known_answer": "3 requests of 'What is the capital of France? Answer with one word.' returned 'Paris' three times, identical"
+  },
+  "instrument": "streamed chat completion of 64 tokens, usage read back; decode rate from the server's own timings; two requests, 4.20 and 4.19 tok/s, same text both times",
+  "repeats": 2,
+  "timestamp": "2026-11-11T13:31:50Z",
+  "source": "~/.cache/scratch-cl-flm/rows.log and out-rows-LFM2.5-1.2B-NPU2/ on strixhalo (row-serve.sh + row-probe.py, lane cl-flm; engine#1786)",
+  "notes": "A hybrid: 10 of 16 layers are gated short convolutions, run as two device GEMMs with the gate and a 3-tap causal conv on the host. This is the per-op route's speed, not what the array does on a geometry with a fused set: FastFlowLM 1.0.12's runtime decoded the same file at 42 to 83 tok/s in the paired tickets of the same day. What would make it wrong: two requests, box load 15 to 19 during the ticket (the same pack measured 4.6 tok/s at load 3 to 4 and 3.5 to 3.7 at load 8 to 10 earlier that day, so load is not the only term)."
+}

bench/npu-leaderboard/lfm25-12b-perop-first-token-20261111.json

@@ -0,0 +1,28 @@
+{
+  "id": "lfm25-12b-perop-first-token-20261111",
+  "host": "strixhalo",
+  "model": "LFM2.5-1.2B-NPU2",
+  "route": "npu",
+  "quant": "Q4NX symmetric 4-bit tiles (FastFlowLM's published container, listed as Q4_0), per-op int8 GEMM, blocked family",
+  "metric": {
+    "name": "first token (warm)",
+    "value": 0.929,
+    "unit": "s"
+  },
+  "settings": {
+    "device": "accel0",
+    "engine": "1bit built at the code of main 1131acf0 (engine#2170)",
+    "container": "FastFlowLM/LFM2.5-1.2B-NPU2 model.q4nx, sha256 b341d42021e617a5…, served as published; the only edit is \"q4nx_weight_layout\": \"b8\" in a copy of config.json",
+    "kernels": "full_i8 blocked: QKV K2048 N3072, QKV K2048 N6144 (the conv layers' in_proj), O K2048 N2048, GU K2048 N16384, D K8192 N2048, each with its M=16 twin",
+    "ticket": "npu-hold -l cl-flm-lfm2-rows, both LFM2 packs in one 107 s ticket",
+    "sampling": "temperature 0, streamed, max_tokens 64",
+    "prompt_tokens": 20,
+    "load1": "18.6 at start, 17.5 at end",
+    "known_answer": "3 requests of 'What is the capital of France? Answer with one word.' returned 'Paris' three times, identical"
+  },
+  "instrument": "time from request to the first streamed chunk on a 20-token prompt; two requests, 1.094 and 0.763 s",
+  "repeats": 2,
+  "timestamp": "2026-11-11T13:31:50Z",
+  "source": "~/.cache/scratch-cl-flm/rows.log and out-rows-LFM2.5-1.2B-NPU2/ on strixhalo (row-serve.sh + row-probe.py, lane cl-flm; engine#1786)",
+  "notes": "Same ticket as lfm25-12b-perop-decode-20261111. Batched prefill, no prefix cache hit (a different prompt had been served before it). The server's own prompt_ms was 766 and 760 ms, so the 1.094 s of the first request is client-side time before the first chunk, not prefill."
+}

bench/npu-leaderboard/lfm25-12b-thinking-perop-decode-20261111.json

@@ -0,0 +1,28 @@
+{
+  "id": "lfm25-12b-thinking-perop-decode-20261111",
+  "host": "strixhalo",
+  "model": "LFM2.5-1.2B-Thinking-NPU2",
+  "route": "npu",
+  "quant": "Q4NX symmetric 4-bit tiles (FastFlowLM's published container, listed as Q4_0), per-op int8 GEMM, blocked family",
+  "metric": {
+    "name": "decode",
+    "value": 4.3,
+    "unit": "tok/s"
+  },
+  "settings": {
+    "device": "accel0",
+    "engine": "1bit built at the code of main 1131acf0 (engine#2170)",
+    "container": "FastFlowLM/LFM2.5-1.2B-Thinking-NPU2 model.q4nx, sha256 b9c4bacba8ed7251…, served as published; the only edit is \"q4nx_weight_layout\": \"b8\" in a copy of config.json",
+    "kernels": "full_i8 blocked: QKV K2048 N3072, QKV K2048 N6144 (the conv layers' in_proj), O K2048 N2048, GU K2048 N16384, D K8192 N2048, each with its M=16 twin",
+    "ticket": "npu-hold -l cl-flm-lfm25tk-row, 234 s",
+    "sampling": "temperature 0, max_tokens 320, not streamed",
+    "prompt_tokens": "the pack's own template (chat_template.jinja) on 'What is the capital of France? Answer with one word.'",
+    "load1": "10.1 at start, 13.0 at end",
+    "known_answer": "3 requests returned the same message three times: 287 reasoning tokens split into reasoning_content, then content that begins 'Paris' and runs to the token limit (finish_reason length)"
+  },
+  "instrument": "usage.completion_tokens over the server's own predicted_ms, 320 tokens per request; three requests, 4.30, 4.25 and 4.37 tok/s, the median",
+  "repeats": 3,
+  "timestamp": "2026-11-11T14:14:56Z",
+  "source": "~/.cache/scratch-cl-flm/rows-25tk.log and out-rows-lfm25tk/ on strixhalo (row-serve.sh + think-row.py, lane cl-flm; engine#1786)",
+  "notes": "The thinking variant of the LFM2.5 hybrid. The reply does not stop after its answer inside 320 tokens. FastFlowLM 1.0.12's own runtime, same file, same prompt, same budget, one ticket later (two requests, identical): content empty, finish length, and 830 characters of reasoning that begin 'Okay derivative of the function f(x) = x^2 + 3x - 4. Find its vertex.' -- a different problem from the one asked -- at 65.5 tok/s. So on this pack the two runtimes do not produce comparable text, and this row is not a parity row: it says this route reasons about the question and reaches 'Paris'. What would make it wrong: one prompt."
+}

bench/npu-leaderboard/qwen3-17b-fused-decode-flm-pair-20261111.json

@@ -0,0 +1,26 @@
+{
+  "id": "qwen3-17b-fused-decode-flm-pair-20261111",
+  "host": "strixhalo",
+  "model": "Qwen3-1.7B-NPU2",
+  "route": "npu",
+  "quant": "Q4NX q4_1 tiles (FastFlowLM's published container, unmodified), sealed fused set 2048x6144x16x8",
+  "metric": {
+    "name": "decode",
+    "value": 39.25,
+    "unit": "tok/s"
+  },
+  "settings": {
+    "device": "accel0",
+    "engine": "1bit 80aceca152a909d5 (lemonade-onebit-rc-2026-11-07-c32)",
+    "container": "FastFlowLM/Qwen3-1.7B-NPU2 model.q4nx, sha256 b1c1fe9d6c4a6d0c…, the same file both runtimes loaded",
+    "ticket": "one npu-hold ticket, 38 s, arms in the order theirs, ours, ours, theirs",
+    "sampling": "greedy, streamed",
+    "reply": "63 streamed chunks per request",
+    "load1": "4.34 at start, 3.30 at end"
+  },
+  "instrument": "streamed chat completion, decode rate from the stream's own chunk times; 3 repeats in each of two arms, median of the six (range 39.20 to 39.32)",
+  "repeats": 6,
+  "timestamp": "2026-11-11T05:42:39Z",
+  "source": "~/lb/aval/clperf-f9-abba-20261111T054201Z/run.log on strixhalo (lane cl-perf's f9-npu-abba.sh, unmodified; engine#1787)",
+  "notes": "Side by side with FastFlowLM 1.0.12's own runtime on the same array and the same model.q4nx, same ticket, A B B A: theirs decoded at 45.57 tok/s (median of its six, range 45.08 to 45.93), ours at 39.25, so theirs is 1.16 times ours on this pack. This row is ours; theirs is quoted here because a row is one run of this engine. What would make it wrong: it is one ticket at load 3 to 4, one prompt, and the shipped c32 build rather than main."
+}

bench/npu-leaderboard/qwen3-17b-fused-first-token-flm-pair-20261111.json

@@ -0,0 +1,26 @@
+{
+  "id": "qwen3-17b-fused-first-token-flm-pair-20261111",
+  "host": "strixhalo",
+  "model": "Qwen3-1.7B-NPU2",
+  "route": "npu",
+  "quant": "Q4NX q4_1 tiles (FastFlowLM's published container, unmodified), sealed fused set 2048x6144x16x8",
+  "metric": {
+    "name": "first token (warm)",
+    "value": 0.568,
+    "unit": "s"
+  },
+  "settings": {
+    "device": "accel0",
+    "engine": "1bit 80aceca152a909d5 (lemonade-onebit-rc-2026-11-07-c32)",
+    "container": "FastFlowLM/Qwen3-1.7B-NPU2 model.q4nx, sha256 b1c1fe9d6c4a6d0c…, the same file both runtimes loaded",
+    "ticket": "one npu-hold ticket, 38 s, arms in the order theirs, ours, ours, theirs",
+    "sampling": "greedy, streamed",
+    "reply": "63 streamed chunks per request",
+    "load1": "4.34 at start, 3.30 at end"
+  },
+  "instrument": "time from request to the first streamed chunk; 3 repeats in each of two arms, median of the six (range 0.566 to 0.578)",
+  "repeats": 6,
+  "timestamp": "2026-11-11T05:42:39Z",
+  "source": "~/lb/aval/clperf-f9-abba-20261111T054201Z/run.log on strixhalo (lane cl-perf's f9-npu-abba.sh, unmodified; engine#1787)",
+  "notes": "Same ticket as qwen3-17b-fused-decode-flm-pair-20261111. FastFlowLM 1.0.12's runtime on the same model.q4nx: 0.591 s (median of its six). The two are level within the ticket's own spread; the first request of each of their arms was slower (0.658 and 0.653 s) and is inside their median."
+}

docs/arch-gaps.md

@@ -1041,7 +1041,10 @@ takes the reviewed set: such a pack prints `. FLM ... registry: none (reviewed,
 Still open, and alerting on purpose: `MiniCPMV4_7ForConditionalGeneration` (needs one converter registration, not an
 architecture), `MiniCPMOcrForConditionalGeneration` (own `minicpm-ocr` library), `smolvla-NPU2` (no config at all), and
 the `!! NPU FLM` families whose registry mapping is HRX-only (`hy_v3`, `dflash`, `gemma4`, `hunyuan-dense`, `gemma3`) —
-that last group is engine #588 (the family must be served on the NPU route), not a registry edit.
+that last group is **not a registry edit**: the family has to be served on the NPU route. It used to be recorded here as
+engine #588, which is **closed** (2026-10-27) and was about *watching FLM's releases*, not about serving a family on the
+route. The issue that does own this class is engine#1786 (every `FastFlowLM/*-NPU2` source serves on `--device npu`,
+opened 2026-11-10). See the last section of this page for the arrivals that are sitting in it.
 
 ## Census run 2026-11-09 (`engine #1667`)
 
@@ -1072,9 +1075,11 @@ Left as they are, on purpose, under the policy above: the six `. REMOTE-CODE` cl
 (`HajeriOperatorForCausalLM`, `MM200ForCausalLM`, `K2HorizonForCausalLM`, `KLMForCausalLM`,
 `KimiK31SuperExpertForCausalLM`, `ThalamusReasoningQwen3_5ForConditionalGeneration`); the `? UNVERIFIABLE` gated repos,
 which are retried on every run; and the `!! FLM` lines. Those keep the 2026-11-08 reasons: `MiniCPMV4_7`, `MiniCPMOcr`
-and `smolvla` as above, and `!! NPU FLM` is engine #588. This run's issue window also includes the `gpt-oss`
+and `smolvla` as above, and `!! NPU FLM` was recorded as engine #588 — which is **closed** (2026-10-27) and was
+about watching FLM's releases, not about serving a family on the NPU route; the class is owned by engine#1786
+(corrected 2026-11-11; see the section at the end of this page). This run's issue window also includes the `gpt-oss`
 (`GPT-OSS-20B`, `GPT-OSS-Safeguard-20b`) and `lfm2` (`LFM2-1.2B`, `LFM2-2.6B`) Q4NX packs. Their mapping is HRX-only
-too, so they belong to #588, not to a registry edit.
+too, so they are the same class — a route to serve, not a registry edit.
 
 ## 2026-11-10: `DeepseekV4FlashSuperExpertForCausalLM` — reviewed, and recorded here
 
@@ -1098,3 +1103,45 @@ the single undocumented entry out of 37.
   The other spelling, `OpenSciForCausalLM` (7 models, same `model_type` and the same `auto_map`), is a separate exact
   key: `census_watch.py` matches architecture classes exactly, so that spelling needs its own entry and is not covered
   by this one.
+
+## FLM 1.0.12's build on the box: one family root with nothing, three that are already somewhere (engine#1787)
+
+F9 of `TNPU-ALL` §9 asks for a row for *every family FastFlowLM's build serves and we do not*. The build is on the box,
+so the checklist was enumerated from it rather than from the release notes:
+
+```
+$ ls ~/flm-f9/xclbins | wc -l      -> 54     (per-family xclbin directories)
+$ ls ~/flm-f9/lib/*.so | wc -l     -> 34     (per-family shared objects)
+```
+
+A word on the name: #1787 calls this "FLM 1.0.12's HRX build", and the section used to. What is counted here is a
+directory of **xclbins, every one named `*-NPU2`**: they are NPU artifacts. A reader looking for GPU families will not
+find them in this list.
+
+Mapping each of the 54 by **family root** (not by model name: `Gemma3-1B` normalises to `gemma31b`, which appears nowhere,
+and matching that way reported 53 of 54 as unknown) against `registry/*.json`, `docs/registry.md` and this page gave 48
+with a trace and 6 xclbins over 4 roots without one. **That second number was wrong** (review of engine#2065): the root
+was compared as a literal against class names and `model_type`s that do not contain it (`nanbeige4.1` against
+`NanbeigeForCausalLM`, `medgemma` against `gemma3`). Each of the four, searched on `main` case-insensitively and
+restated 2026-11-11:
+
+| xclbin directory | family root | what `main` has | what is actually owed |
+|---|---|---|---|
+| `Translategemma-4B-Instruct-NPU2` | `translategemma` | nothing: `git grep -il translategemma` finds 0 files | everything. This is the one root with no trace. By its name it would be a `gemma3` arrival like `medgemma`; nobody has opened its config, so that is a guess and not a mapping |
+| `LFM2.5-1.2B-NPU2`, `LFM2.5-1.2B-Thinking-NPU2` | `lfm2.5` | the packs' own `config.json` says `Lfm2ForCausalLM` / `model_type: lfm2`, which the registry maps, so there is no "2.5 class" to add; `models/pins.json` pins the `LFM2.5-2.6B` and `LFM2.5-8B-A1B` GGUFs; and since engine#2170 both packs **serve on the NPU route** with measured rows in [npu.md](npu.md) ("FastFlowLM's own containers") | nothing in the registry. On the route: speed (per-op, about 4 tok/s against their 48 to 83), not coverage |
+| `Medgemma-4B-NPU2`, `Medgemma-1.5-4B-NPU2` | `medgemma` | a `gemma3` arrival, already routed: [npu.md](npu.md) has the row `gemma3` / `medgemma-1.5-4b-it-NPU2` / hrx / not served, and `tests/census_watch_test.py` pins that pack to `Gemma3ForCausalLM` | an NPU row. `Medgemma-4B-NPU2` is on the box. From its `config.json` (H 2560, 8 heads, 4 KV heads, head_dim 256, IM 10240) its four shapes are `QKV K2560 N4096`, `O K2048 N2560`, `GU K2560 N20480` and `D K10240 N2560`. In the unblocked kernel family the last three are built (for example `~/models/npu/qwen3-coder-kernels/`); in the blocked (`_b8`) family only `O` is. `QKV K2560 N4096` is built in neither, so it is one kernel shape away, not a model-type item (counted 2026-11-11 with `find ~ -name 'full_i8_<shape>*.elf'`, FastFlowLM's own files excluded) |
+| `Nanbeige4.1-3B-NPU2` | `nanbeige4.1` | mapped: `registry/architectures.json` has `NanbeigeForCausalLM` (gguf `nanbeige`), `registry/census.json` counts it, `registry/support_pins.json` lists its converter config, and `q4nx_semantics.h` and [tokenizers.md](tokenizers.md) name `Nanbeige4.1-3B` itself (11 files) | an NPU row. Nothing on [npu.md](npu.md) says whether it serves there, and it has not been tried |
+
+So: **one** family root with nothing (`translategemma`), and three that are mapped or served and owe a measured NPU row
+or nothing at all. The count of 48 "with a trace" was spot-checked, not re-derived, and after this correction it should
+be read as "at least 51".
+
+**The rows that are still owed are un-inspected, and say so on purpose.** Nothing above describes what `translategemma`
+or `nanbeige4.1` computes: an xclbin name is evidence that *their* build carries a compiled family for it, and that is
+all. §1's five conditions are what converts an entry from *listed* to *settled* — imports with zero gaps and serving on
+the device, a known-answer prompt under the model's own template with three identical answers, decode tok/s and TTFT
+under a ticket, and the kernel set and engine commit named. The two `lfm2.5` packs are the example of an entry that
+went from listed to settled that way.
+
+The `!! NPU FLM` class above is where the owed rows belong. Its owning issue is engine#1786 (every `*-NPU2` source
+serves), which is open.