[bong-water-water-bong]

Refs #1786 (backlog). Docs only: docs/npu.md gains one subsection, "FastFlowLM's own containers, served as published: measured rows", after the existing census tables. No code changes.

Why

#1786 asks that results be written where they can be read later, not only in comments. The existing section records which families the registry does not map; it has no place for the other question, which the last day answered for eleven packs: does FastFlowLM's own model.q4nx, with nothing converted, load and answer on --device npu.

What it records

• Fused route, shipped build, nothing edited: Qwen3-0.6B, Qwen3-1.7B, Qwen3-4B, Qwen3-4B-Instruct-2507, Qwen3-4B-Thinking-2507 and the text tower of Qwen3-VL-4B-Instruct, each with the sealed set it ran on, ready time, the known answer and a decode rate.
• Per-op route: Qwen3-1.7B, Qwen3.5-2B, Llama-3.2-1B, Llama-3.2-3B, Phi4-mini-Instruct, Qwen3-VL-4B-Instruct (text), with what that route needs (a blocked-family kernel set and one key in a copy of config.json) and the plain statement that its decode rate is that route's, with the same pack on both routes as the contrast (3.0 against 39.0 tok/s).
• D3-Nano-NPU2 on its decision route.
• The one same-pack pair against FastFlowLM's own runtime (Qwen3-1.7B: 45.7 against 39.3 tok/s, time to first token level), from F9: FLM 1.0.12's HRX-targeted build — side-by-side against ours on the same models, and a row for every family we do not serve #1787.
• The open fault on the fused route (npu: on the fused route, enable_thinking false on a thinking-only pack returns the reasoning and a literal </think> as content (Qwen3-4B-Thinking-2507-NPU2) #2135) and which two packs it affects.
• The six engine faults these rows found, all merged (forward-serve: the tokenizer's eos_token and the ChatML turn end are stop ids too — <|im_end|> was served as text on FLM's Qwen3.5-2B (Refs #1785, #1786) #2064, forward-serve: a streamed reply is sent as it is generated, not after — time to first token was the whole request (Refs #1785) #2067, forward-serve: reasoning the prompt opened is returned as reasoning — on Qwen3.5-2B's own template it all came back as content (Refs #1786) #2098, forward-serve: whitespace around a reasoning block is layout, not text — 'one word' came back as "\n\nParis" (Refs #1786) #2118, forward-serve: a text request to a decision-only pack is a 400 that names the decision route — it was a 500 from inside the forward (Refs #1786) #2121, forward-serve: the turn end is read off the container's own template — Phi4-mini ran on past 'Paris<|end|>' to the token budget (Refs #1786) #2144).
• What no row establishes: the importer, vision and audio towers, packs with no kernel set, and families the forward may not model.

Every number is from this box under npu-hold; each row's ticket and artifacts are in the #1786, #1785 and #1787 comments it summarises. Where two figures are not comparable (the two Qwen3-4B rates were taken in different tickets at different loads) the text says so instead of leaving the table to imply a model difference.

Checks

tools/copyright.py --check rc 0. Nothing is built from this file. I checked each figure in the new text against the run output it came from; the two I had to look twice at are the Qwen3.5-2B decode range (2.7 to 3.4 tok/s across the #2075 and row tickets) and "six faults" (the six PRs listed above).

🤖 Generated with Claude Code

[bong-water-water-bong]

ready for review at 72b0c08

[bong-water-water-bong]

VERDICT at 72b0c08: READY

Reviewed head 72b0c08 (refs #1786). Docs only: docs/npu.md +62, nothing else. This is the first verdict.

• git merge-tree onto main: no conflicts. tools/copyright.py --check: rc 0. CI build succeeded (waited for it before posting).
• The six faults it names are all merged: #2064, #2067, #2098, #2118, #2121 and #2144 (merged=true each).
• Spot-checked figures against the comments they summarise:
  • FastFlowLM's own runtime vs ours on Qwen3-1.7B-NPU2, "45.7 tok/s against 39.3 … (0.59 s and 0.57 s)", matches #1787: "45.7 against 39.3 tok/s … ours slightly ahead (0.57 s against 0.59 s)". The pair is in the sentence's own order, theirs then ours.
  • The per-op contrast "3.0 tok/s here and 39.0 on the fused route" matches #1786's row #19 correction ("My 3.0 tok/s above was the slow route") and that row's 39.0.
• The #2135 paragraph describes that issue as it stands. I have it claimed, with a fix in progress, so the sentence will want "fixed by #…" once that lands.

Nit: the fused Qwen3-1.7B-NPU2 row gives 39.0 tok/s. The same row in #1786 also lists 44.5 / 43.5 / 43.7, and its prose says "39–44 tok/s". Choosing the low end is the conservative reading, but the table does not say which of those figures its single number is (a cold run, a loaded ticket?). A word in the column header or a footnote would stop the next reader re-deriving it.

[github-actions]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶 1786 - Partially compliant Compliant requirements: Every FastFlowLM/*-NPU2 model source imports and serves on --device npu Each repo must import with zero gaps and pass self-check Each repo must serve on --device npu Each repo must answer a known-answer prompt with the model's own chat template, 3 identical requests returning identical text Decode tok/s and time-to-first-token must be measured under npu-hold Non-compliant requirements: The row names the kernel set and the engine commit that produced it (this is not explicitly verified in the PR, but implied by the data) Requires further human verification: Verification that the kernel set and engine commit are correctly identified for each model (this requires checking the actual hardware logs or build artifacts) 1787 - Partially compliant Compliant requirements: Side-by-side measurement of FastFlowLM 1.0.12's HRX-targeted build against ours on the same models Interleaved A B B A pattern with their build and ours Decode tok/s and time to first token for each Every family their build serves that we do not becomes a row Non-compliant requirements: The result is written where the scoreboard is recomputed from, not in a comment (this is not directly addressed in the PR) Requires further human verification: Verification that the scoreboard is recomputed from the correct location (this requires checking the actual scoreboard setup) 2135 - Partially compliant Compliant requirements: The PR documents the issue and its impact on specific packs Non-compliant requirements: The actual fix is not implemented in this PR (as noted in the PR description) Requires further human verification: Verification that the fix is actually implemented in a subsequent PR (this is not part of this PR)

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Documentation Clarity The documentation does not explicitly state how the kernel set and engine commit are identified for each model, even though it's implied by the data. This could lead to confusion for readers trying to reproduce the measurements. Serving these packs as published is what found six engine faults, each fixed with the pack as its evidence: Missing Verification of Engine Commit While the documentation mentions that each row names the kernel set and engine commit, there is no verification that this information is accurate or complete in the PR. This could be a problem for reproducibility. Serving these packs as published is what found six engine faults, each fixed with the pack as its evidence: