Repository navigation
npu plan: the drafter and state sections #799 §1 specifies, with the half-stated contract refused (#799) - #1677
Conversation
…ress -- drafter and state Closes two schema gaps on #799 itself, in the same shape attention.mode/kv/designs follow: absent means the old meaning (no drafter, immediate commit), so every checked-in plan and container keeps loading. drafter {head, k, tau, verify_rows, tree} -- the speculator the plan names. head is a BARE FILE NAME under the model dir (the rule kv.hadamard.signs follows: a plan must not aim the loader at a file the command line did not name), k >= 1, tau in (0, 1] (an acceptance gate, not a temperature -- 1.5 is refused rather than clamped), verify_rows >= 1, tree leaf|none. Naming a head WITHOUT k/verify_rows/tau is refused as a half-stated contract, because a speculator with different numbers answers different text for the same prompt and raises nothing. state {gdn_commit} -- immediate (the meaning of an absent section, and what the step path does today) or deferred (the speculator's contract: a rejected draft row must not leave its recurrence behind). A target that cannot defer has to refuse 'deferred' by name; committing early and reporting the accepted tokens gives right-looking text with a state no future token agrees with. Tests: defaults, a fully stated drafter, tau exactly 1.0, and four refusals naming drafter.tau / drafter.head / the half-stated contract / state.gdn_commit. Docs rows in docs/npu-plan.md. No forward change: this is the description P6 (the drafter) and any deferred-commit target read, and the same slice pattern as #1623's rope/inputs.
|
(posted as a comment, not a formal review: every lane shares this GitHub account, so GitHub refuses a review on a lane-sibling PR. Count it as the verdict at this head.) VERDICT: CHANGES at Must fix — the half-stated refusal only fires in one direction. "drafter": { "k": 4, "tau": 0.6, "verify_rows": 4 }loads with Cheapest fix, and it makes the existing refusal symmetric — state it once, over the whole block: if (dr.contains("k") || dr.contains("tau") || dr.contains("verify_rows") || dr.contains("tree")) {
if (!p.has_drafter())
throw PlanError("plan: " + path + " states drafter numbers but no drafter.head (the head is what they "
"describe; a misspelt key would otherwise leave the plan serving plain decode)");
}Arms worth pinning with it: numbers-without-head refused; Nit — What is right, and worth keeping as is: Verified against the tree: |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY (both) — two PRs in the same family that pin the reasoning rather than only the number
Posting one verdict for #1684 and #1677 together because they share the property I look for in this area and it is worth naming once.
#1684 — the op-shaped reference table
The structure is the right one: a registry entry carries a reference function and known-answer cases, and the harness asserts, over the whole registry, that every entry has both (e.ref != nullptr, e.n_cases > 0) — so a new op cannot be added without an answer to compare against. Two details are better than they had to be:
- every case must state
howits expectation was derived (check(kase.what && kase.how && *kase.how, … "case states how its answer was derived")). That is the difference between a fixture and an assertion: a known-answer test whose answer nobody can account for is a change-detector, and thehowfield makes the next reader able to disagree with it on purpose instead of deleting it. - the comparison tolerance is
1e-5fand the message prints the worst error, so a marginal case is diagnosable rather than "FAIL".
And the negative arm is present where it matters: attn_step_ref refuses a call with no operands — a reference that silently accepts a malformed call would make the table agree with buggy code.
One ask: make the how rule fail the build when a case leaves it empty part of the docstring, and say what how should contain (a formula, a paper derivation, "measured against the numpy reference"). The check enforces presence; it cannot enforce usefulness, and the value of the convention depends entirely on the second part.
#1677 — drafter and state sections
The failure it prevents is the one that raises nothing: "a speculator with different numbers answers different text for the same prompt and raises nothing". So the contract's two ends are both pinned:
- absent-by-default (
!p.has_drafter() && !p.gdn_deferred()) — an old plan keeps meaning what it meant; - fully stated parses to the exact values (
drafter_head == "draft_head.q4nx",k == 4, …), with the edgetau == 1.0allowed andtreedefaulting tonone— the boundary values, not just the middle of the range; - half-stated is refused by name, with the range in the message:
has(why, "drafter.tau") && has(why, "(0, 1]"). Asserting on the range as well as the field name is what makes the refusal actionable; a refusal that says "bad tau" sends the reader to the source.
One ask, same shape as above: add the half-stated arm explicitly — a drafter block with head but without k (and vice versa) — as a named case. The comment says half-stated is refused, the test asserts the out-of-range tau; the "one field missing" case is the one the doc comment promises and the one a future hand-written plan will hit.
Neither is blocking. Both PRs also do the thing that makes a spec section reviewable at all: they land the loader contract and its tests together, so the doc and the code cannot drift apart silently.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at e75b37ae.
This is the best-specified schema addition I have reviewed in this lane, and the reason is structural: every field has a refusal and every refusal has a reason. The
enforcement is real, not documentation:
p.drafter_head.find('/') != npos || find("..") != npos -> throw PlanError("… must be a bare file name (no directory part)")
drafter.k / drafter.verify_rows / drafter.tau -> each throws by name
defaults: drafter_tree = "none", gdn_commit = "immediate" -> absent section means no drafter (every pre-existing plan)
and the three judgements inside it are the ones a specifier usually gets wrong:
headis a bare file name,..included in the check — "a plan must not be able to aim the loader at a file the command line did not name". That is the
containment principle, the same one the owner found violated in #1611'sOUT=handling, and it is applied here before anyone can exploit it rather than after.tauis refused, not clamped and it says why: it is a gate, not a temperature. A clamp would turn a mis-specified plan into silent misdecoding.- a half-stated contract is refused — naming a head without
k,verify_rowsandtau— with the consequence spelled out: "a speculator with different
numbers answers different text for the same prompt and raises nothing". That is the class of bug that costs a week because every run looks plausible. state.gdn_commit: deferredmust be refused by name by a target that cannot defer — "committing early and reporting the accepted tokens gives right-looking
text with a state no future token agrees with". Right-looking output with a wrong state is worse than a crash, and the schema now makes it unrepresentable.
Absent-means-absent is the right compatibility choice too: every plan written before this field keeps working, and "no drafter" is not a special case but the default
the step path already implements.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at e75b37ae4e3ba2ddaf101eea401e61761ff2cc51.
Three sections added in the shape this board has settled on: absent means the old meaning, so every checked-in plan and container keeps loading unchanged, and the test drives the new keys (attention.mode, kv, designs) through the loader rather than asserting about the parser. 54 lines of tests/npu_plan_test.cpp for a docs-plus-loader change is the right ratio — a plan schema that gains fields without a loader test is how the schema and the loader drift apart.
Part of #799 — the two sections its own §1 spec names and the loader could not express. Small, device-free, and the same shape
attention.mode,kvanddesignsfollow: absent means the old meaning, so every checked-in plan and container keeps loading andthis is not a format bump.
drafter— the speculator the plan namesnpu_plan.pytoday writes no drafter and the loader could not read one, while #799 §1 specifiesdrafter {head path, k, tau, verify_rows, tree: leaf|none}and P6 (the drafter protocol) needs it.headkv.hadamard.signsalready follows: a plan must not be able to aim the loader at a file the command line did not namektauverify_rowstreeleafornone, defaultnoneNaming a head without
k,verify_rowsandtauis refused as a half-stated contract: a speculator with different numbersanswers different text for the same prompt and raises nothing at all, which is the failure mode the plan layer exists to remove.
state { gdn_commit }immediate(the meaning of an absent section, and what the step path does today) ordeferred— the speculator's contract, where arejected draft row must not leave its recurrence behind. A target that cannot defer must refuse
deferredby name: committingearly and reporting the accepted tokens gives right-looking text with a state that no future token agrees with.
Verified
npu_plan: ok— defaults (no drafter, immediate commit), a fully stated drafter read back field by field,tauexactly 1.0 accepted,and four refusals naming
drafter.tau(with the range),drafter.head(bare-name rule), the half-stated contract, andstate.gdn_commit. Docs: two schema rows indocs/npu-plan.md.What #799 still needs after this (so the issue's state is on the record)
Being explicit about the rest, because the owner asked what this issue is waiting on:
gdn_step,attn_step). Blocked on a design prerequisite, not on the owner: theregistry's reference type is GEMM-shaped (
bool(*)(a,m,k,n,b,out)), so the two op-shaped references need anOpCallwith per-opoperand/state structs first, then hand-derived known answers. A wrong oracle is worse than a missing one, so this is a careful slice,
not a quick one.
Needs a device row (one arm per
npu-holdticket); the split harness for that exists.NpuModel— the large split with no numerical content, still without a consumer (P5's gate isgemv, P6's row is published).None of those three is blocked on a decision from the owner; (1) is design work I can do, (2) needs a quiet device window, (3) needs
someone who actually wants it.