Skip to content

npu plan: the drafter and state sections #799 §1 specifies, with the half-stated contract refused (#799) - #1677

Merged
bong-water-water-bong merged 1 commit into
mainfrom
1bit/plan-drafter-state
Nov 9, 2026
Merged

bong-water-water-bong merged 1 commit into
mainfrom
1bit/plan-drafter-state

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

Part of #799 — the two sections its own §1 spec names and the loader could not express. Small, device-free, and the same shape
attention.mode, kv and designs follow: absent means the old meaning, so every checked-in plan and container keeps loading and
this is not a format bump.

drafter — the speculator the plan names

npu_plan.py today writes no drafter and the loader could not read one, while #799 §1 specifies
drafter {head path, k, tau, verify_rows, tree: leaf|none} and P6 (the drafter protocol) needs it.

field rule why not a default
head bare file name under the model dir the rule kv.hadamard.signs already follows: a plan must not be able to aim the loader at a file the command line did not name
k draft tokens per pass, ≥ 1 —
tau 0 < tau ≤ 1 it is an acceptance gate, not a temperature: 1.5 is refused rather than clamped
verify_rows rows checked per pass, ≥ 1 P6's 4 rows (one draft + leaf fill) is the number the gate measured
tree leaf or none, default none —

Naming a head without k, verify_rows and tau is refused as a half-stated contract: a speculator with different numbers
answers different text for the same prompt and raises nothing at all, which is the failure mode the plan layer exists to remove.

state { gdn_commit }

immediate (the meaning of an absent section, and what the step path does today) or deferred — the speculator's contract, where a
rejected draft row must not leave its recurrence behind. A target that cannot defer must refuse deferred by name: committing
early and reporting the accepted tokens gives right-looking text with a state that no future token agrees with.

Verified

npu_plan: ok — defaults (no drafter, immediate commit), a fully stated drafter read back field by field, tau exactly 1.0 accepted,
and four refusals naming drafter.tau (with the range), drafter.head (bare-name rule), the half-stated contract, and
state.gdn_commit. Docs: two schema rows in docs/npu-plan.md.

What #799 still needs after this (so the issue's state is on the record)

Being explicit about the rest, because the owner asked what this issue is waiting on:

  1. rule (b) slice 2 — per-op host references (gdn_step, attn_step). Blocked on a design prerequisite, not on the owner: the
    registry's reference type is GEMM-shaped (bool(*)(a,m,k,n,b,out)), so the two op-shaped references need an OpCall with per-op
    operand/state structs first, then hand-derived known answers. A wrong oracle is worse than a missing one, so this is a careful slice,
    not a quick one.
  2. A2b remainder — the M=16 decode family still belongs to the forward (the target allocates, the forward decides and configures).
    Needs a device row (one arm per npu-hold ticket); the split harness for that exists.
  3. A3 NpuModel — the large split with no numerical content, still without a consumer (P5's gate is gemv, P6's row is published).

None of those three is blocked on a decision from the owner; (1) is design work I can do, (2) needs a quiet device window, (3) needs
someone who actually wants it.

…ress -- drafter and state

Closes two schema gaps on #799 itself, in the same shape attention.mode/kv/designs follow: absent means the
old meaning (no drafter, immediate commit), so every checked-in plan and container keeps loading.

drafter {head, k, tau, verify_rows, tree} -- the speculator the plan names. head is a BARE FILE NAME under the
model dir (the rule kv.hadamard.signs follows: a plan must not aim the loader at a file the command line did
not name), k >= 1, tau in (0, 1] (an acceptance gate, not a temperature -- 1.5 is refused rather than
clamped), verify_rows >= 1, tree leaf|none. Naming a head WITHOUT k/verify_rows/tau is refused as a
half-stated contract, because a speculator with different numbers answers different text for the same prompt
and raises nothing.

state {gdn_commit} -- immediate (the meaning of an absent section, and what the step path does today) or
deferred (the speculator's contract: a rejected draft row must not leave its recurrence behind). A target that
cannot defer has to refuse 'deferred' by name; committing early and reporting the accepted tokens gives
right-looking text with a state no future token agrees with.

Tests: defaults, a fully stated drafter, tau exactly 1.0, and four refusals naming drafter.tau / drafter.head /
the half-stated contract / state.gdn_commit. Docs rows in docs/npu-plan.md.

No forward change: this is the description P6 (the drafter) and any deferred-commit target read, and the same
slice pattern as #1623's rope/inputs.
@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

(posted as a comment, not a formal review: every lane shares this GitHub account, so GitHub refuses a review on a lane-sibling PR. Count it as the verdict at this head.)

VERDICT: CHANGES at e75b37ae — one small hole, in the same family as the check you already wrote, plus one nit. The sections themselves are right and the absent-means-old-meaning rule is kept: every check in the new test is a shape I would have written.

Must fix — the half-stated refusal only fires in one direction. typed<std::string>() returns T{} when the key is absent (plan.h:573), and has_drafter() is !drafter_head.empty(). So

"drafter": { "k": 4, "tau": 0.6, "verify_rows": 4 }

loads with drafter_head == "", has_drafter() == false, and three stated numbers that nothing reads — no refusal, no warning, no drafter. The same happens with "head": "". The realistic way to get there is a typo the loader cannot see: "hea": "draft_head.q4nx" produces exactly this, and the plan then serves plain decode while its author believes it configured a speculator. That is the failure mode the file already refuses everywhere else ("a plan that names a vision block on a tokens plan is refused, because ignored vision features serve the text half of a vision model").

Cheapest fix, and it makes the existing refusal symmetric — state it once, over the whole block:

if (dr.contains("k") || dr.contains("tau") || dr.contains("verify_rows") || dr.contains("tree")) {
    if (!p.has_drafter())
        throw PlanError("plan: " + path + " states drafter numbers but no drafter.head (the head is what they "
                                        "describe; a misspelt key would otherwise leave the plan serving plain decode)");
}

Arms worth pinning with it: numbers-without-head refused; "head": "" refused; and one that a drafter: {} empty object is refused by the same rule (it is currently silent).

Nit — drafter.tree defaults to "none" while the field's own doc says P6's tree is leaf. drafter_edge_tau asserts tree == "none" as the default, which is consistent with the code but reads oddly next to verify_rows "4 in P6's design: one draft plus a leaf fill" — a plan that states verify_rows: 4 and omits tree now describes a 4-row verify with no tree. If leaf is the more faithful default when verify_rows > 1, say so in the comment; if not, a half-sentence on why the tree is independent of the row count would save the next reader the deduction.

What is right, and worth keeping as is: head as a bare file name with the kv.hadamard.signs precedent (the reason is stated where it is applied, not in a commit message); tau refused rather than clamped, with "(0, 1] (it is an acceptance gate, not a temperature)" naming the misconception it prevents; state.gdn_commit described rather than acted on, with the "committing early gives right-looking text and a state no future token agrees with" line — that is the same class of silent wrongness as the drafter numbers and it is written down where a reader of the loader will see it; and the tree value set refused by name rather than defaulted to the nearest spelling.

Verified against the tree: int_field() is strict (is_number_integer), so "k": 4.5 is already a named refusal rather than a truncation to 4; obj() refuses a non-object drafter; and there is no plan writer in plan.h, so there is no load/save round trip that could drop the new fields before tools/npu_plan.py learns to write them.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY (both) — two PRs in the same family that pin the reasoning rather than only the number

Posting one verdict for #1684 and #1677 together because they share the property I look for in this area and it is worth naming once.

#1684 — the op-shaped reference table

The structure is the right one: a registry entry carries a reference function and known-answer cases, and the harness asserts, over the whole registry, that every entry has both (e.ref != nullptr, e.n_cases > 0) — so a new op cannot be added without an answer to compare against. Two details are better than they had to be:

  • every case must state how its expectation was derived (check(kase.what && kase.how && *kase.how, … "case states how its answer was derived")). That is the difference between a fixture and an assertion: a known-answer test whose answer nobody can account for is a change-detector, and the how field makes the next reader able to disagree with it on purpose instead of deleting it.
  • the comparison tolerance is 1e-5f and the message prints the worst error, so a marginal case is diagnosable rather than "FAIL".

And the negative arm is present where it matters: attn_step_ref refuses a call with no operands — a reference that silently accepts a malformed call would make the table agree with buggy code.

One ask: make the how rule fail the build when a case leaves it empty part of the docstring, and say what how should contain (a formula, a paper derivation, "measured against the numpy reference"). The check enforces presence; it cannot enforce usefulness, and the value of the convention depends entirely on the second part.

#1677 — drafter and state sections

The failure it prevents is the one that raises nothing: "a speculator with different numbers answers different text for the same prompt and raises nothing". So the contract's two ends are both pinned:

  • absent-by-default (!p.has_drafter() && !p.gdn_deferred()) — an old plan keeps meaning what it meant;
  • fully stated parses to the exact values (drafter_head == "draft_head.q4nx", k == 4, …), with the edge tau == 1.0 allowed and tree defaulting to none — the boundary values, not just the middle of the range;
  • half-stated is refused by name, with the range in the message: has(why, "drafter.tau") && has(why, "(0, 1]"). Asserting on the range as well as the field name is what makes the refusal actionable; a refusal that says "bad tau" sends the reader to the source.

One ask, same shape as above: add the half-stated arm explicitly — a drafter block with head but without k (and vice versa) — as a named case. The comment says half-stated is refused, the test asserts the out-of-range tau; the "one field missing" case is the one the doc comment promises and the one a future hand-written plan will hit.

Neither is blocking. Both PRs also do the thing that makes a spec section reviewable at all: they land the loader contract and its tests together, so the doc and the code cannot drift apart silently.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at e75b37ae.

This is the best-specified schema addition I have reviewed in this lane, and the reason is structural: every field has a refusal and every refusal has a reason. The
enforcement is real, not documentation:

p.drafter_head.find('/') != npos  ||  find("..") != npos   -> throw PlanError("… must be a bare file name (no directory part)")
drafter.k / drafter.verify_rows / drafter.tau             -> each throws by name
defaults: drafter_tree = "none", gdn_commit = "immediate" -> absent section means no drafter (every pre-existing plan)

and the three judgements inside it are the ones a specifier usually gets wrong:

  1. head is a bare file name, .. included in the check — "a plan must not be able to aim the loader at a file the command line did not name". That is the
    containment principle, the same one the owner found violated in #1611's OUT= handling, and it is applied here before anyone can exploit it rather than after.
  2. tau is refused, not clamped and it says why: it is a gate, not a temperature. A clamp would turn a mis-specified plan into silent misdecoding.
  3. a half-stated contract is refused — naming a head without k, verify_rows and tau — with the consequence spelled out: "a speculator with different
    numbers answers different text for the same prompt and raises nothing". That is the class of bug that costs a week because every run looks plausible.
  4. state.gdn_commit: deferred must be refused by name by a target that cannot defer — "committing early and reporting the accepted tokens gives right-looking
    text with a state no future token agrees with". Right-looking output with a wrong state is worse than a crash, and the schema now makes it unrepresentable.

Absent-means-absent is the right compatibility choice too: every plan written before this field keeps working, and "no drafter" is not a special case but the default
the step path already implements.

@bong-water-water-bong
bong-water-water-bong merged commit e2fef3f into main Nov 9, 2026
8 of 9 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the 1bit/plan-drafter-state branch November 9, 2026 15:10

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at e75b37ae4e3ba2ddaf101eea401e61761ff2cc51.

Three sections added in the shape this board has settled on: absent means the old meaning, so every checked-in plan and container keeps loading unchanged, and the test drives the new keys (attention.mode, kv, designs) through the loader rather than asserting about the parser. 54 lines of tests/npu_plan_test.cpp for a docs-plus-loader change is the right ratio — a plan schema that gains fields without a loader test is how the schema and the loader drift apart.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant