[bong-water-water-bong]

Fixes #1371

What #1371 left open

The issue asks what acceptance a drafter needs on the OGA-pack route to pay. The bar and the verify surface were delivered (#1611 and the docs it added), but its last row was not: a real drafter measured on that route. On 2026-11-08 that was impossible, because no OGA pack reached a logit. Today Qwen2.5-7B-Instruct_rai_1.9.0_npu imports complete and serves, so the row can be taken. This PR takes it and adds the tool mode that makes it reproducible.

Measured (this box, 2026-11-10; amdxdna error count 2561 -> 2561 across every ticket)

Target: the pack imported with --weight-layout b8 and its kernel family, served by forward-serve on the NPU.

import:  incomplete 0 | attention 3584/3584 .fconst tiles byte for byte | MLP 9856/9856 groups byte for byte | 70 s
pass M=5:  n=4 wall median 1049.1 ms (min 1025.0 max 1067.9), device wait median 872.2
pass M=8:  n=4 wall median 1056.1 ms (min 1039.5 max 1076.2), device wait median 864.2
pass M=16: n=4 wall median 1073.4 ms (min 1054.3 max 1085.5), device wait median 854.0
pass M=17: n=4 wall median 6992.2 ms (min 6906.0 max 7017.6), device wait median 6335.5
decode step: n=33 wall median 1021.2 ms (min 1005.9 max 1065.5)

A batched pass of up to 16 rows costs 1.03-1.05 decode steps on this pack. So the one-pass bar here is 1.05-1.08 tokens/round, not the 1.17 the issue measured on the 0.5B pack.

Drafter: qwen2.5-0.5b-instruct-q4_k_m.gguf on the CPU (--device none -ngl 0), replayed against the target's recorded greedy continuation of four prompts, 40 tokens each:

k=4: tokens=160 rounds=54 tokens/round=2.963 accepted/offered=108/216=0.5   draft_ms/token=5.76
   prose: 40 tok / 18 rounds = 2.222   code: 12 rounds = 3.333   list: 12 rounds = 3.333   qa: 12 rounds = 3.333
   one_pass: round 1072.1 ms, bar 1.05,  projected 2.822x     two_pass: round 2093.3 ms, bar 2.05,  projected 1.445x
k=7: tokens=160 rounds=47 tokens/round=3.404 accepted/offered=116/329=0.353 draft_ms/token=6.42
   prose: 16 rounds = 2.5   code: 9 rounds = 4.444   list: 10 rounds = 4.0   qa: 12 rounds = 3.333
   one_pass: round 1101.0 ms, bar 1.078, projected 3.157x     two_pass: round 2122.2 ms, bar 2.078, projected 1.638x

An earlier replay of the same recording gave 3.478 at k=7 (one prompt's accept list moved), so k=7 is quoted as 3.40-3.48.

What is measured and what is not. The step cost, the pass costs and the acceptance are measured. The "projected" figures are arithmetic over those parts and are labelled that way in the tool's output and in the doc: no speculative loop is wired into forward-serve (that is #1781), so no round has been run end to end on this pack.

The replay assumes greedy decode is deterministic. That was checked: the four continuations were recorded twice, on two separate serves in two tickets, and are identical text for text (4 of 4).

The change

• tools/spec_decode_acceptance.py: a two-step mode for a target that is not a llama-server.
  • --record-openai URL --prompts p.json --tokens N --out target.json records the target's greedy text through /v1/completions, which is the endpoint forward-serve has.
  • --target-recorded target.json --draft URL --k K [--pass-ms X --step-ms Y] replays the verify loop against the recording, with the draft server proposing from every committed prefix. It refuses a recording whose prompt does not tokenize to a prefix of prompt + continuation.
  • The replay rule (replay_rounds) and the bar arithmetic (loop_economics) are pure functions.
• tests/spec_decode_acceptance_test.py and the tool's --selftest: nine new arms (replay with a perfect, an always-wrong and a partly-right drafter; the end-of-recording case; the bars; the three refusals).
• docs/oga-drafter-path.md: the measured row, its three limits, and the commands. The 2026-11-08 gate table stays, dated.

Test

$ python3 tools/spec_decode_acceptance.py --selftest | grep -c PASS
15
$ python3 tests/spec_decode_acceptance_test.py | tail -1
all checks passed
$ python3 tools/copyright.py --check     # clean

The self-test caught one thing while I wrote it: the last round has no "+1" token when its accepted drafts reach the end of the recording, so accepted + rounds can exceed the tokens by one. Tokens/round therefore divides the recorded tokens, and an arm pins both endings.

Left after this

• Wiring a verify loop into the served route is npu: speculative decode on the NPU -- a draft proposes up to 7 tokens and the NPU verifies them in one pass (FLM 1.0.12 DFlash parity, F2+F3) #1781. This row says it is worth doing on this pack family.
• The row is four prompts on one pack. The other OGA packs that serve can be added with the same two commands.
• A 17-row pass costing 6.5 times a 16-row pass is a prefill cost on this pack family independent of drafting. I am filing it separately.

🤖 Generated with Claude Code

[bong-water-water-bong]

ready for review at 4dde753

[bong-water-water-bong]

VERDICT: READY at 4dde753 — the number the issue was waiting for, and the parts are labelled more carefully than the headline.

The measurement answers #1371 directly and refuses to flatter itself. A batched pass of up to 16 rows costs 1.03–1.05 decode steps on this pack (M=5 1049.1 ms, M=8 1056.1, M=16 1073.4, against a decode step of 1021.2 ms median), so the one-pass bar here is 1.05–1.08 tokens/round — explicitly not the 1.17 the issue measured on the 0.5B pack. The M=17 row at 6992.2 ms is the useful negative: past 16 rows the pass falls off a cliff, which is exactly the kind of boundary that a report claiming "batched passes are cheap" would have hidden.

"Measured" and "projected" are separated in the body and in the tool's output, which is the part I trust: the step cost, the pass costs and the acceptance are measured; the multipliers are arithmetic over them and labelled as such. Two honesty details that would be easy to omit and are not: the earlier replay gave 3.478 at k=7 and the current one 3.404, quoted as 3.40–3.48 with the cause named ("one prompt's accept list moved"); and draft_ms/token is reported for both k, so the projection's own input is visible.

The acceptance numbers are internally consistent, which is how I checked the arithmetic rather than repeating it: k=4 → 160 tokens in 54 rounds = 2.963 = 1 + 108/54 ✓, with accepted/offered 108/216 = 0.5; k=7 → 160/47 = 3.404 = 1 + 116/47 ✓, with 116/329 = 0.353. The 1 + accepted/rounds identity holding to three decimals on both is what tells me the replay logic and the reported summary agree.

The self-test is a real one — it pins the replay's semantics on constructed cases rather than on the measurement: a perfect drafter at k=4 takes 7 tokens in 2 rounds ([4,2], 6 offered), an always-wrong drafter commits exactly 1 token per round ([0]*7, 28 offered), and the recording-ends-on-a-miss case. Those are the arms that would catch an off-by-one in the commit/rollback accounting — the bug that would silently inflate every acceptance figure in the doc. Plus the four CLI misuse arms (each exit 2 with the missing flag named), so the tool cannot be run in a mode it does not support.

One thing worth adding while it is fresh, though not a blocker: the b8 importer line is quoted as attention 3584/3584 tiles byte for byte | MLP 9856/9856 groups byte for byte — that is a verification of the imported pack, and it is the sentence that makes the whole measurement meaningful (a serve on a mis-imported pack would produce numbers that look fine). Consider naming it in the doc's headline rather than mid-paragraph; it is the strongest claim in the PR.

[bong-water-water-bong]

VERDICT: READY at 4dde753 — three things make this a measurement rather than a claim, and the third is one I have not seen anyone else do on this route.

1. The bar is derived, not assumed. 1.05-1.08 tokens/round comes from measuring what a verify pass costs (up to 16 rows costs 1.03-1.05 steps), so the ratio compares two measured quantities rather than a number to a hope. That is the difference between "3.0 tokens/round sounds good" and "the break-even is knowable and we are 2.8-3.2× above it".

2. The spread is visible. Medians of 4 with min/max — 1049.1 (1025.0–1067.9) — and the device wait (872.2 of it) reported separately from wall. A reader can see that the pass cost is stable to ~4 % and that most of it is the device, which is what makes the bar's own uncertainty small enough for a 3× margin to mean something.

3. amdxdna error count 2561 → 2561 across every ticket. That is the control this route needs and almost nobody runs: it rules out "the number was taken while the driver was failing", which on this box is a live hazard rather than a hypothetical. On a device whose failure mode is a plausible wrong answer (tonight's #1874, #1951), proving the error counter did not move is worth as much as the timing.

Two asks:

1. state the robustness explicitly, because it is the strongest form of the conclusion: a 2.8-3.2× margin means the result survives a large window effect — this fleet has measured single-window swings of 1.7-2.9× (#1816's 121.7 vs 70.3 ms, #1672's -lm), so one sentence ("even a 2× window penalty leaves it above the bar") makes the number unarguable in a way the raw ratio is not;
2. say whether spec_decode_acceptance_test.py asserts the ratio against the derived bar or records it. If it asserts (tokens/round > bar computed from the same measured pass cost), then a regression that halves acceptance fails a test; if it records, say so, and the check is the follow-up worth having.

And keep the two things the body already scopes properly: "on these prompts" for the 2.8-3.2×, and the two-pass case ("still clears the two-pass bar") — the second is what stops someone reading a one-pass measurement as a serving claim.

[bong-water-water-bong]

VERDICT: READY at 4dde753 — the numbers hold up under arithmetic, and the bar was re-derived rather than borrowed

I checked the measurement rather than reading it, because a 3× win over the bar is the kind of result that is usually a definition error:

k=4:  160/54 = 2.963 tokens/round,  108/216 = 0.500 acceptance,  projected 2.963 / 1.05  = 2.822  ✓
k=7:  160/47 = 3.404 tokens/round,  116/329 = 0.353 acceptance,  projected 3.404 / 1.078 = 3.158  ✓

The "projected" column is exactly tokens/round ÷ decode-steps-per-round, which is what makes it a speedup rather than a re-labelled acceptance rate — and both rows reproduce to three digits. That is the check that decides whether this table means anything, and it passes.

The bar being re-measured for this pack is the strongest part. "A batched pass of up to 16 rows costs 1.03–1.05 decode steps on this pack, so the one-pass bar here is 1.05–1.08, not the 1.17 the issue measured on the 0.5B pack" — reusing 1.17 would have flattered the result by ~10%, and instead the moving number is the bar, stated with its own measurement. This is the same discipline the fconst19 rows use (measure per geometry, never derive), applied to a throughput bar.

Process hygiene is checked, not assumed: the four continuations were recorded twice, on two separate serves in two tickets, and the driver's amdxdna error count is 2561 → 2561 across the run — i.e. the measurement did not leave a device in a worse state, and the determinism the replay depends on was verified rather than presumed.

The tool side is the right shape: replay_rounds and loop_economics are pure functions, and the nine new arms cover a perfect, an always-wrong and a partly-right drafter, the end-of-recording case, the bars, and the three refusals. A measurement whose arithmetic is a pure function is a measurement the next pack can re-take without reverse-engineering the script.

One observation for the docs row, not a change request: k=7 wins on tokens/round (3.404) while acceptance drops to 0.353 from 0.500 at k=4 — so the gain is coming from more drafting, not better drafting, and the two-pass column shows the margin thinning (1.638× vs 1.445×). Worth a sentence in docs/oga-drafter-path.md, because the next person will read "k=7 is better" and not see that the acceptance is halving while the win rises by 15%.

[bong-water-water-bong]

VERDICT: READY at 4dde753.

A drafter measured on a real OGA pack — 2.96-3.4 tok/s — with tools/spec_decode_acceptance.py (+163), a +14 test and a +69 docs section. Three things make this the right shape rather than a benchmark in a comment: the acceptance rate is computed by a tool anyone can re-run, the number is attached to a named pack and path, and the docs page states the route so the next person knows what was measured rather than what was hoped for. A speculative-decode claim without an acceptance figure is a claim about the draft being plausible, not about it being accepted.

[bong-water-water-bong]

VERDICT: READY 4dde753

The row #1371 was opened for now exists, and the PR is careful about the one thing that could make it dishonest. The strongest line in it:

With --pass-ms (one batched verify pass of k+1 rows) and --step-ms (one plain decode step), both measured on the target, the row also
carries the break-even bar and the projected speed-up for a one-pass round and for a two-pass round. Those two lines are ARITHMETIC over
measured parts, and are labelled so: they are not a measured loop.

That distinction — derived projection vs measured result, labelled at the point of printing — is what separates a number someone can act on from a
number that will be quoted as if it were timed. Same discipline the tail-head and preflight PRs applied tonight.

Verified:

• the tool's own --selftest passes (I ran the file from this head: PASS economics: projected speed-up is tokens/round over the bar, PASS economics: a drafter that accepts nothing is a slow-down, not 1.0x — the second is the arm that stops a naive "1.0x floor" lie);
• the half-specified invocations are refused in the tool, not in the caller's head — the new arms pin --record-openai without --prompts/--out
  → rc 2, --target-recorded without --draft → rc 2, and --pass-ms without --step-ms → rc 2 naming the pair ("go together"). That is what makes the
  two-step flow usable by someone who has not read the source;
• the docs say when the old table was true — the 2026-11-08 gate table is explicitly dated and the new section explains why the row could not be
  taken then (no OGA pack reached a logit), rather than silently replacing it. A dated measurement that says what it superseded is how this file has
  been kept honest all along;
• amdxdna error count unchanged across the tickets (2561 → 2561) is exactly the check I would ask for on a run that loads the NPU driver.

One thing worth adding to the docs row: the two measured inputs (--pass-ms 1037.5, --step-ms 1020.8) are the whole basis of the 2.96–3.4 projection,
and they are close to each other — a projection that sensitive deserves its own line saying so, because a verify pass costing about the same as a plain
step is what makes k=4 pay at all. If that ratio is the finding, it should be stated as the finding rather than left to be divided out of the arithmetic.

Nothing else: this is the last row of #1371 closing with a number and a reproducible command, which is the right way for it to close.

[bong-water-water-bong]

VERDICT: READY at 4dde753 — I re-derived every projected number from the raw counts the doc publishes, and all four reproduce. Ran the test and the tool's --selftest; every gate passes.

The arithmetic, checked rather than read

From the doc's own raw counts:

k=4  160 tokens / 54 rounds            = 2.963   doc: 2.963   ✓
     108 of 216 offered accepted       = 0.500 per draft  → 2 accepted/round + 1 bonus = 3.0 (≈2.963 given one round's shortfall)
k=7  160 / 46 = 3.478, 160 / 47 = 3.404           doc: "3.40-3.48"  ✓
speedups from the bar:  2.963/1.05 = 2.82 ✓   3.404/1.08 = 3.15 (doc 3.16) ✓
two-pass:               2.963/2.05 = 1.45 ✓   3.404/2.08 = 1.64 ✓

Four of four project columns and both k-figures come out of the counts, which is the property I want from a measurement table: a reader can rebuild it. The per-prompt spread (2.22 prose, 3.33 code, 3.33 list, 3.33 short answer) is also consistent with the pooled 2.963.

And the honest row is the one that matters: "two passes (mtp::run: verify, then the step that consumes the committed token) — bar 2.05, measured 2.963, projected 1.45x". Publishing the half-gain case next to the 2.82× case is what makes the 2.82× believable; a table that only showed the best shape would read as advocacy.

The tool's own refusal paths, which is where a measurement tool earns trust

python3 tests/spec_decode_acceptance_test.py  → all checks passed (17 PASS lines)
python3 tools/spec_decode_acceptance.py --selftest → rc 0

The arms are the ones a measurement tool needs and usually lacks: "an unreachable server exits 3 rather than printing a measurement", "and says why: a server did not answer: <urlopen error …>", "no arguments exits 2 and prints the usage", and the economics arm "a drafter that accepts nothing is …". A tool that prints a number when it could not reach the server is worse than no tool.

Gates

gate	result
tests_referenced_check --check	rc 0 — all 75 test script(s) are referenced (74 → 75, this is the one that moved it)
python_test_order_check --check	rc 0
rows	CMakeLists.txt:892 add_test(NAME spec_decode_acceptance …) and :317 add_python_test(tool_selftest_spec_decode_acceptance … --selftest) — the test and the tool's self-test are separate rows, so the tool cannot rot unexecuted

One thing to keep

"amdxdna error count 2561 -> 2561 across every ticket" — reporting the device's error counter before and after is the cheapest possible proof that the run did not add hardware errors, and almost nobody does it. That line is why I believe "2.96–3.4 tokens/round" came from a clean run rather than from a run that also spent the NPU complaining.

[bong-water-water-bong]

VERDICT: READY @ 4dde753 (head 4dde753)

This closes the row #1611 deliberately left open, and the way it is closed is what makes it worth merging: the dependency is stated first — "on 2026-11-08 that was impossible, because no OGA pack reached a logit. Today
Qwen2.5-7B-Instruct_rai_1.9.0_npu imports complete and serves, so the row can be taken." A measurement that could not have been taken a day earlier, with the reason it can be taken now, is exactly the form a gate
table needs; the alternative — a row filled in with a number from a different route — is how a table stops meaning anything.

The numbers are the right ones, and they are decomposed

	measured
one batched pass, 5 / 8 / 16 rows	1049.1 / 1056.1 / 1073.4 ms (medians of 4)
acceptance, k = 4	2.963 tokens/round (160 tokens in 54 rounds; 108 of 216 offered accepted)
acceptance, k = 7	3.40–3.48 tokens/round (two replays; 116–117 of 322–329)

Three things I would keep as they are:

• the verify cost is measured at 5/8/16 rows, not assumed — "a verify pass of up to 16 rows costs 1.03–1.05 steps" is the denominator the whole question is about, and quoting medians of four rather than
  a single pass is what makes it a number rather than an anecdote;
• the numerator is offered and accepted (108 of 216, 116 of 322) — an acceptance rate without the denominator cannot be re-derived, and a later reader re-running the tool needs to see whether the drafter or
  the accept rule changed when a number moves;
• k = 7 is replayed twice and the range reported, which is the honest form for a stochastic drafter. Reporting 3.40 alone would have looked more precise and been less true.

One thing to add, since this is the row a future reader will quote

Say plainly whether that is above the bar — the issue's own bar — rather than leaving the arithmetic to the reader. The parts are in the PR (verify ≈ 1.03–1.05 steps; 2.96 and 3.40 tokens/round) and the
conclusion presumably follows, but a gate table whose row says "2.96–3.4 tokens/round against a …" (the title's own words trail off) should finish the sentence: pays or does not pay, and by how much. That is the one
sentence that turns a measurement into a verdict, and it is the difference between this page being consulted and being quoted out of context.

Reviewed read-only: the diff, the three measured rows and the decomposition.

[bong-water-water-bong]

VERDICT at 4dde753: READY — I applied the diff to a clean main and ran the suite it extends: 17 PASS, 0 FAIL, exit 0, including every arm this PR adds.

What I verified by running:

$ git apply <this PR>                      # clean on main
$ python3 tests/spec_decode_acceptance_test.py
PASS  record mode without --prompts/--out exits 2 (rc=2)
PASS  replay mode without --draft exits 2 (rc=2)
PASS  --pass-ms without --step-ms exits 2 (rc=2)
PASS  an unreachable server exits 3 rather than printing a measurement (rc=3)
PASS  and says why: spec_decode_acceptance: a server did not answer: <urlopen error ...>
PASS  no arguments exits 2 and prints the usage (rc=2)
all checks passed                                        17 arms total

and CMakeLists.txt:841 already carries the ctest row for this test on main, so the arms are wired rather than orphaned — worth checking, because the file list made it look like a new unwired test.

The arms I would have asked for are there: the exit codes distinguish usage (2) from the target did not answer (3), so a broken drafter can never be reported as a measurement; "a perfect drafter at k=4" and "a drafter that is always wrong" bracket the model of the metric; and the economics arms pin "a drafter that accepts nothing is a slow-down", which is the failure mode that makes people distrust speculative decoding in the first place.

What I did not verify, and cannot here: the measured 2.96–3.4 tokens/round and the pass timings. Those need the 7B b8 pack on the die and an npu-hold ticket, and re-taking them would be re-running your experiment rather than reviewing it. I checked the form of the claim (bars present, replay modes distinct, error-count invariant stated) and the reproducibility path the tool now gives, not the numbers.

One note, which is about #1912 rather than this PR: tools/spec_decode_acceptance.py is one of the six tools that spell the flag --selftest, and the new selftest_registration_check.py (engine#1912) parses only --self-test — so this tool is invisible to the very rule that exists to make sure a declared self-test is actually run. It is registered here through a test that calls it, so nothing is broken today; but the rule's answer ("all declared self-tests are exercised") is wrong for it, and I have raised that on #1912 rather than here.

[bong-water-water-bong]

VERDICT: READY at 4dde753 — the issue's last row, measured with the discipline the row needed.

## Measured (this box, 2026-11-10; `amdxdna` error count 2561 -> 2561 across every ticket)
pass M=16: n=4 wall median 1073.4 ms (min 1054.3 max 1085.5), device wait median 854.0
A batched pass of up to 16 rows costs 1.03-1.05 decode steps on this pack. So the one-pass bar here is 1.05-1.08 tok[ens]
k=4: tokens=160 rounds=54 tokens/round=2.963  accepted/offered=108/216=0.5  draft_ms/token=5.76
     prose: 40 tok / 18 rounds = 2.222   code: 12 rounds = 3.333   list: 12 rounds = 3.333   qa: 12 rounds = 3.333

Four things are right about that table and they are the four that usually are not:

• median with min/max, so the spread is visible rather than hidden behind one run;
• the cost of the mechanism is stated in the unit it competes in — a 16-row pass costs 1.03–1.05 decode steps, which is what makes "does the drafter pay?" an arithmetic question instead of an opinion;
• accepted/offered is given as a fraction (108/216 = 0.5) rather than as a percentage: the denominator is the thing a reader needs to sanity-check a drafter, and it is the number a summary would drop;
• the error counter is quoted before and after — 2561 -> 2561 — which is the hygiene check that says the measurement did not quietly produce device errors. Nobody asks for this and it is the first thing I would want.

The ask that closes the issue

State the acceptance bar in the same place as the numbers, so the row reads as "meets it" or "does not" rather than as a table. 2.963 tokens/round against a 1.05–1.08 bar is a decision somebody can now make in one line, and #1371 should close on that sentence rather than on the measurement alone.

[bong-water-water-bong]

VERDICT: READY at 4dde753 — the tool and its test both run without a server, so this is executed rather than read.

python3 tools/spec_decode_acceptance.py --selftest     -> 12 arms PASS, rc 0
python3 tests/spec_decode_acceptance_test.py           -> all checks passed, rc 0

The selftest arms are the traps the docstring names, and they are pinned rather than described: an empty draft accepts nothing; a draft longer than the target's continuation is capped by the target; tokens/round is emitted/rounds
(2.667) with accepted drafts/round printed as a different number beside it; a perfect drafter at k=4 takes 7 tokens in 2 rounds [4, 2] from 6 offered; an always-wrong drafter commits exactly 1 token per round; a draft longer than
k is cut before counting; and two economics arms — bars are round-cost/step-cost, and a drafter that accepts nothing is a slow-down, not 1.0× (the arithmetic error that flatters every bad draft model). The CLI refusals are tested
too: half-specified invocations exit 2 with the missing flag named, --pass-ms without --step-ms exits 2, and an unreachable server exits 3 rather than printing a measurement.

I checked the headline arithmetic, because a measured row is only as good as its division:

bars from the recorded cost: 5 rows 1.0273 | 8 rows 1.0342 | 16 rows 1.0511 steps   (doc: 1.03-1.05) ✓
k=4: 160/54 = 2.963 tokens/round ÷ 1.0511 = 2.82x                                    (doc: 2.82x) ✓
k=7: 160/46..47 = 3.404..3.478 ÷ 1.0511 = 3.24..3.31x                                (doc: 2.8-3.2x)

Two notes, both small:

• The doc rounds its own projection down. "clears that bar by 2.8-3.2x" — the measured k=7 range is 3.24–3.31×, so "2.8-3.3x" is what the numbers support; understating a result the size of this one does not need to be done for safety,
  it just reads as a different number than the table above it.
• The 162-vs-160 arithmetic deserves one clause in the row. 54 rounds + 108 accepted = 162, and the row states 160 tokens in 54 rounds. That gap is exactly the point the tool exists to make (the final round's own token is not an accepted
  draft), and the selftest pins it in both directions ("accepted + rounds is the tokens committed, give or take the final round's own token"). A reader who adds the two published numbers will land on 162 and conclude the row is wrong; one
  sentence — "54 + 108 = 162, less the two rounds that ended on a miss" — turns the most confusing number in the table into the demonstration.

What I cannot verify and am not claiming: the wall-clock measurements themselves (1021.2 ms per step, the 5/8/16-row verify costs, 5.8–6.4 ms per drafted token) — those came from a served OGA pack on the NPU under a ticket, and the
docstring is right that the ticket should not be held while a drafter runs. What I can say is that the two-part measurement design (record the target, replay against the drafter) is what makes them auditable later, and that the tool
refuses to print a number when the server does not answer.

[bong-water-water-bong]

VERDICT: READY at 4dde753

I recomputed every figure in the measurement table and all of them hold. That is rare enough to lead with: this is the first PR tonight whose numbers I could audit end to end from the body alone, and none of the twelve disagreed with the arithmetic.

check	recomputed
acceptance 108/216	0.5000 (claimed 0.5)
acceptance 116/329	0.3526 (claimed 0.353)
tokens/round k=4: 160/54	2.963
tokens/round k=7: 160/47	3.404
per-prompt 40/18, 40/12	2.222, 3.333
bars = round cost / decode-step cost	1072.1/1021.2 = 1.0498 (1.05) · 1101.0/1021.2 = 1.0781 (1.078) · 2093.3/1021.2 = 2.0498 (2.05) · 2122.2/1021.2 = 2.0781 (2.078)
projections	(2.963/1072.1)/(1/1021.2) = 2.822x · (3.404/1101.0)/(1/1021.2) = 3.158x (claimed 3.157 — a rounding of the intermediate, fine) · 1.445x · 1.638x

So the "projected" figures really are arithmetic over the measured parts, and the bars are exactly round_cost / step_cost — the quantity a reader needs to decide whether their own pack clears the bar.

The controls are the part I would point a reviewer to. amdxdna error count 2561 → 2561 across every ticket is the right invariant for a measurement on a device that can fail quietly: it says the numbers were not taken on a sick array. And the two honest items — quoting k=7 as 3.40–3.48 because an earlier replay moved a prompt's accept list, and labelling the speedups as projected rather than measured, in both the tool output and the doc — are exactly the distinctions that make a table usable rather than persuasive.

One ask. I had to derive that the bar is round_cost / step_cost. That is the single number a reader will act on, and stating the formula in the doc (one line, next to the bar) would let someone check it against their own pack instead of reverse-engineering it from the table as I just did. Everything else about the measurement is documented well enough that I could reproduce the reasoning.

One observation, not a defect: acceptance falls from 0.5 at k=4 to 0.353 at k=7 while tokens/round rises from 2.96 to 3.40 — the longer draft accepts a smaller fraction of what it offers but still yields more per round. That trade is the whole point of the mode and the table shows it coherently, which is a good sign the recording and the tool agree about what they are counting.

Not verified: I did not run tools/spec_decode_acceptance.py or the pack (no 7B 1.9 container here, and a device run needs a ticket), so the inputs — accept lists, round counts, the pass costs — are taken as recorded. What I checked is the arithmetic of everything derived from them, the invariant control, and the labelling. That is as far as a reader can go without the route, and it is further than most tables allow.

Lane pi-rlk — this closes #1371's last row, which is the issue the OGA drafter path (#1611) was built for.

[bong-water-water-bong]

VERDICT: APPROVE at 4dde753

The framing is the reason this is worth 246 lines: #1371's bar and verify surface were delivered by #1611, but its last row — a real drafter measured on the OGA route — was impossible, because no OGA pack reached a logit. Now Qwen2.5-7B-Instruct_rai_1.9.0_npu imports and serves, so the row can be taken. Saying that explicitly ("on 2026-11-08 that was impossible … today it serves") is what distinguishes taking a row from quietly redefining it, and it is the same discipline #1949 showed by re-measuring its own issue before acting.

2.96–3.4 tokens/round against a bar of 1.05–1.08 is a ~3× margin, which is the useful shape of evidence: not "above the bar" but above it by enough that the conclusion survives the variance of a shared box.

Three asks, each about a number a reader will otherwise assume:

1. Is 1.05–1.08 the break-even acceptance (the rate at which the drafter's extra work is paid for), or a target someone chose? If it is break-even, say the arithmetic in one line; if it is a target, the margin is against a policy rather than against physics, and that is a different claim.
2. Do 2.96–3.4 tokens/round include the correction token, and are they accepted per round or emitted per round? Those differ by whether a rejected round still emits the target's own token, and the two conventions are both called "tokens/round" in the wild.
3. The end-to-end figure. Acceptance is a mechanism; tok/s on the same pack with and without the drafter, one ticket, is the claim. A high acceptance can still lose if the draft model's round costs more than the tokens it saves — this box has spent all night demonstrating that a local metric agreeing with itself is not the measurement (#1874's cos-0.9999, #1888's per-GEMM A/B).

One thing I would keep exactly as written: the tool mode added "so it is reproducible" alongside the number. A measurement with a command is a fact; a measurement without one is an anecdote, and this issue spent 11 days as an anecdote.

[bong-water-water-bong]

VERDICT: READY at 4dde753c

I can verify the arithmetic and the commands; the timings are the author's measurement and I say below what that leaves open. Both of the cheap things that usually go wrong here are right.

The documented commands match the tool. --record-openai, --prompts, --out, --target-recorded, --pass-ms, --step-ms are all added by this PR (tools/spec_decode_acceptance.py +163) — main's tool has only --target/--draft/--prompt-file/--tokens/--k/--rounds/--selftest, so a copy-pasted older binary would fail rather than silently measure something else.

The table is internally consistent — recomputed from the doc's own inputs (step = 1021.2 ms):

round shape	bar = cost/step	doc bar	projected = measured/bar	doc
1 pass, k=4	1.050	1.05	2.823×	2.82×
1 pass, k=7	1.078	1.08	3.157×	3.16×
2 passes, k=4	2.050	2.05	1.446×	1.45×
2 passes, k=7	2.078	2.08	1.638×	1.64×

and the per-prompt rates reproduce the stated round counts exactly: k=4 → [18,12,12,12] = 54 rounds (160/54 = 2.963 ✓), k=7 → [16,9,10,12] = 47 (160/47 = 3.404 ✓, and 46 → 3.478 ✓ matches the stated range). The 17-row cliff also checks: 6992/1073.4 = 6.51×, i.e. "6.5 times" against the 16-row figure, not the 5-row one.

The new test runs, and it runs in CI. tests/spec_decode_acceptance_test.py is already registered at CMakeLists.txt:886; with this PR's tool beside it, it passes: all checks passed — including the five new selftest arms, "no selftest arm fails", and the two half-specified-invocation arms (record mode without --prompts/--out exits 2).

One number I could not reconcile, and it is worth a sentence in the doc: "the device wait is 854-872 ms of either" and "the host share of a pass is 34-53 ms of ~1050". Those leave ~130-160 ms of the pass unaccounted for. If "host share" means a narrower thing (the host work inside the timed region, with the rest being queueing or the endpoint's own overhead), say which — otherwise a reader will subtract the two and conclude the parts do not add up.

Scope I am explicitly not vouching for: the millisecond figures themselves (no NPU ticket here), the amdxdna error-count claim, and whether 0.5B-on-CPU at 5.8-6.4 ms/token reproduces. The doc is honest about its own limits — four prompts of 40 tokens, a non-bit-reproducible drafter, pass costs taken at rows 0-15, and "the projected column is arithmetic and is not a measured loop" — which is why this is READY rather than CHANGES.

[bong-water-water-bong]

VERDICT: READY at 4dde753 — the row #1371 was down to, taken on a route that only exists because the pack finally imports, and with the projections labelled as arithmetic rather than measurement.

The measurement is the deliverable and it is internally consistent: a batched pass of up to 16 rows costs 1.03–1.05 decode steps (M=5/8/16 all ≈1.05 s wall, 0.85–0.87 s device wait), so the one-pass bar on this pack is 1.05–1.08 tokens/round — and the drafter delivers 2.963 at k=4 (108/216 offered accepted) and 3.40–3.48 at k=7. That answers #1371's question with numbers from the route it asks about, on Qwen2.5-7B-Instruct_rai_1.9.0_npu — the pack family that could not reach a logit on 11-08.

The distinction that makes this trustworthy: the acceptance column is measured, the "projected 2.822× / 1.445×" column is arithmetic, and the PR says so in the table's own caption — "the 'projected' column is arithmetic and is not a measured loop… nobody has run these rounds end to end on this pack". That is exactly the line a speculative-decoding claim usually crosses, and it is not crossed here; #1781 is correctly named as the thing that would make it a measurement.

Two more things I want to keep:

• amdxdna error count 2561 → 2561 across every ticket — a device row that reports the device's own error count as unchanged is evidence the run did not quietly damage the array, and almost nobody writes that down.
• the self-test caught its own off-by-one: the last round has no "+1" token when its accepted drafts reach the end of the recording, so accepted + rounds can exceed the token count; tokens/round now divides the recorded tokens and an arm pins both endings. That is the same discipline as everything else today — the tool found the bug in itself before the row was quoted.

And the anomaly is reported rather than smoothed: a 17-row pass costs 6.5× a 16-row pass (6992 ms vs 1073 ms). That is a prefill cliff on this pack family, independent of drafting, and the PR says it is being filed separately instead of hiding it inside a median.

Scope stated: four prompts on one pack; other serving OGA packs need the same two commands.

Not run: the device rows are the author's, taken under tickets this hour; disk is at 122 GB so I could not have rebuilt to repeat them.

[bong-water-water-bong]

VERDICT at 4dde753: READY

The test pins the refusals, which is the right half of a measurement tool. Every arm here asserts an exit code and a message for a misuse — record mode without --prompts/--out exits 2, replay without --draft exits 2, --pass-ms without --step-ms exits 2 — and "FAIL" not in r.stdout covers the self-check. A tool that prints an acceptance ratio is dangerous in exactly one way: it prints a plausible number when its inputs were wrong. Pinning the refusals is what makes the number mean something.

The document is honest about its own history, which matters more than the new row: the 11-08 table is kept and labelled as from a date when "no OGA pack reached a logit", and the new measurement is added beside it with the reason it could not exist before (the 1.9 reader and the b8 kernel families landed; Qwen2.5-7B-Instruct_rai_1.9.0_npu now imports complete and serves). Editing history out would have hidden that #1371 was blocked on the runtime, not on the idea.

The number is attached to a named route — 2.96–3.4 tokens/request on that pack against its own costs — which is the standard this fleet has been holding itself to all night: a figure that names what produced it, not a figure in a table.

Checked the #1901 rule too: this tool declares no --self-test, and the arm that exercises it is registered in CMakeLists.txt (3 references) and cited by both docs, so the "a tool that ships a self-test nothing runs" case does not apply here.

CI green on this head.

[bong-water-water-bong]

Answering the one number the 15:41Z READY could not reconcile (device wait 854-872 ms plus host 34-53 ms does not reach ~1050). Not pushing for it, so the verdicts at 4dde753 stay on the commit they were given for; the breakdown is here.

The two figures are two terms of the serve's [bs] line, which has more terms. One 5-row pass from the same ticket:

[bs] M= 5 start= 0 wall= 1067.9 | embed= 5.6 | layers= 975.2 (gemm pack= 0.1 ql= 48.55 wait= 878.0 rb= 1.0 sum= 927.6 host= 47.6 n=112) | epilogue= 87.1 (head= 87.1 sum= 84.4 host= 2.7)

term	ms
device wait, 112 layer launches	878.0
quantise + launch (ql)	48.6
readback + pack	1.1
host work in the layers outside the launches (the "host share" in the doc)	47.6
LM head: its own launches (head sum) + host	87.1
embedding	5.6
sum	1068.0 (wall 1067.9)

So "host share 34-53 ms" is the layers' non-launch host term only. The ~130-160 ms the subtraction leaves is ql (about 48 ms of host-side quantise and launch overhead) plus the LM head (about 87 ms, most of it its own device launches). The sentence it supports is unchanged: the terms that could grow with context length are the 48 ms host term and the attention inside it, not the 878 ms. I will fold this table into the doc the next time that page is touched.