npu plan: the drafter and state sections #799 §1 specifies, with the half-stated contract refused (#799)

docs/npu-plan.md

@@ -68,6 +68,8 @@ tools/npu_plan.py <model dir> [--kernels DIR] [--out PATH] [--check] [--print]
 | `designs[]` | **optional**: the per-role kernel set — `role` (`QKV`, `QKVLIN`, `O`, `GU`, `G`, `U`, `D`, the MoE `GB`/`UB`, the Gemma4 second attention type, the MLA and GDN roles), `M`, `K`, `N` and the artifact `elf` (a **bare file name**). When present, `Plan::designs_agree()` requires it to be exactly the set the container derives — same roles, same geometry, same artifact, **both directions** — and refuses by naming the role and both sides; a plan without the section keeps the engine's own derivation, which is what every plan written before the field means. Written by `tools/npu_plan.py` only for the families it can state completely (a partial section would be refused at load, so MLA, GDN mixes and Gemma4's second attention type / per-layer embeddings / double-wide MLP get none). `Plan::check_artifacts()` then refuses at load, before any context is opened, when a named artifact is not present and non-empty in the kernel directory — and `npu_plan.py --check` asks the same question with the same rule at import time |
 | `rope.kind`, `rope.planes`, `rope.sections` | **optional**: which rotary the decoder applies. `plain` (the meaning of an absent block, and of every plan written before the field) is one position index with one plane. `mrope` is the vision-conditioned form — a **3 x N block of position ids** whose planes (t, h, w) advance differently across an image, so a text row with all three planes equal is bit-identical to plain RoPE while an image row is not. `sections` is the pack's own split of the rotary width over the planes (Qwen3-VL: `[24, 20, 20]`) and is **required** for `mrope`: without it a caller has no declared width per plane and would rotate a different model than the pack was trained as. `planes` defaults to the number of sections and must agree with it. A `plain` rope that carries `sections`, or `planes` other than 1, is refused rather than read as if the planes existed. This is a description like `kv`: a caller that cannot build the planes must refuse `mrope` **by name** — falling back to one index predicts fluently, and text after a 448×448 image resumes at position **19** where a token-count advance would give 201 (engine#799) |
 | `inputs.kind`, `inputs.vision` | **optional**: what the decoder is fed. `tokens` (the meaning of an absent block) is one id per row, looked up by the decoder. `embeds` is rows × hidden of precomputed embeddings already merged with any vision features — the split AMD's reference makes with a separate `embedding.onnx` feeding `text.onnx`. `inputs.vision.merge` says who merged (`external` or `none`) and `inputs.vision.feature_dim` how wide the merged features are; `external` requires a positive `feature_dim`, `none` requires 0, and a `vision` block on a `tokens` plan is refused, because ignored vision features serve the text half of a vision model and answer confidently about an image it never saw. Descriptive only: a plan that says `embeds` and a caller that feeds ids is the caller's refusal to make |
+| `drafter.head`, `drafter.k`, `drafter.tau`, `drafter.verify_rows`, `drafter.tree` | **optional**: the speculator the plan names (engine#799 §1). Absent means no drafter, which is what every plan written before the section means. `head` is a **bare file name** under the model dir (a plan must not be able to aim the loader at a file the command line did not name, the rule `kv.hadamard.signs` follows), `k` is draft tokens per pass (≥ 1), `tau` the acceptance gate (**0 < tau ≤ 1** — it is a gate, not a temperature, and a tau of 1.5 is refused rather than clamped), `verify_rows` the rows checked per pass (≥ 1), `tree` `leaf` or `none`. Naming a head without stating k, verify_rows and tau is refused as a **half-stated contract**: a speculator with different numbers answers different text for the same prompt and raises nothing |
+| `state.gdn_commit` | **optional**: `immediate` (the meaning of an absent section, and what the step path does today) or `deferred` — the speculator's contract, where a rejected draft row must not leave its recurrence behind. A target that cannot defer must refuse `deferred` by name: committing early and reporting the accepted tokens gives right-looking text with a state no future token agrees with |
 | `companion` | **refused by name, with a reason**: a plan describes **one** decoder stream over one token sequence. A second graph (a vision encoder, a projector) is its own model directory that the serving layer merges into `inputs.embeds`; a companion list would put two graphs and their scheduling in a file whose only consumer can neither open nor order them (engine#799) |
 | `layers[]` | one entry per layer, in order: `index`, `role` (`attn`/`gdn`/`moe`/`mla`), and projections `qkv {K, N, fused}`, `attn_gate {K, N}`, `gate_up {K, N}`, `down {K, N}`. The count must equal `model.layers`. The projection shapes are recorded for P4’s `NpuModel`; the forward reads the count (for its [plan] log line), and takes the design set from `designs[]` when the plan names it, deriving it from the container otherwise |
 

npu/plan/plan.h

@@ -162,6 +162,30 @@ struct Plan {
     std::string vision_merge = "none";    // "external" | "none"
     int vision_feature_dim = 0;
     bool takes_embeds() const { return inputs_kind == "embeds"; }
+    // The drafter the plan may name (engine#799 §1 `drafter {head path, k, tau, verify_rows, tree}`).  Absent means
+    // no drafter, which is what every plan written before this field means.  Descriptive, like `rope` and `kv`: the
+    // speculator (P6) reads it, and a plan that names a drafter this engine cannot drive is refused at the point that
+    // tries to drive it -- never silently served as plain decode, because a speculator with the wrong tau or k answers
+    // with different text for the same prompt and no error at all.
+    //
+    //   head        the draft head artifact, a BARE FILE NAME under the model dir (the same rule kv.hadamard.signs
+    //               follows: a plan must not be able to aim the loader at a file the command line did not name)
+    //   k           draft tokens per pass, >= 1
+    //   tau         the acceptance gate, 0 < tau <= 1
+    //   verify_rows rows the verifier checks per pass, >= 1 (4 in P6's design: one draft plus a leaf fill)
+    //   tree        "leaf" (P6's tree) or "none" (a plain chain)
+    std::string drafter_head;
+    int drafter_k = 0, drafter_verify_rows = 0;
+    double drafter_tau = 0.0;
+    std::string drafter_tree = "none";   // "leaf" | "none"
+    bool has_drafter() const { return !drafter_head.empty(); }
+    // When a GDN layer's recurrent state is committed (engine#799 §1 `state {gdn commit}`): "immediate" is every plan
+    // written before this field and is what the step path does today; "deferred" is the speculator's contract, where a
+    // rejected draft row must not leave its recurrence behind.  A target that cannot defer must refuse it by name:
+    // committing early and reporting the accepted tokens gives the right-looking text with a state that no future token
+    // agrees with.
+    std::string gdn_commit = "immediate";   // "immediate" | "deferred"
+    bool gdn_deferred() const { return gdn_commit == "deferred"; }
     // P7 (#806) part A: what the KV cache is stored as.  `f16` is every plan written before this field
     // existed, and stays the default for that reason.  `int8` stores that side as int8 with one fp32
     // scale per (head, token), taken AFTER a per-head Hadamard rotation -- the rotation is what makes one
@@ -386,6 +410,58 @@ struct Plan {
                                     " but inputs.vision.merge is none (no merge, no features)");
             }
         }
+        // The drafter (engine#799 §1).  Absent means no drafter; every stated field is checked, because a speculator
+        // with a plausible-but-wrong k/tau/rows answers different text for the same prompt with no error at all.
+        if (j.contains("drafter")) {
+            const auto& dr = obj(j, "drafter", path);
+            p.drafter_head = typed<std::string>(dr, "head", "drafter", path);
+            if (!p.drafter_head.empty() &&
+                (p.drafter_head.find('/') != std::string::npos || p.drafter_head.find("..") != std::string::npos))
+                throw PlanError("plan: " + path + " drafter.head \"" + p.drafter_head +
+                                "\" must be a bare file name (no directory part)");
+            if (dr.contains("k")) {
+                p.drafter_k = int_field(dr, "k", "drafter", path);
+                if (p.drafter_k < 1)
+                    throw PlanError("plan: " + path + " drafter.k " + std::to_string(p.drafter_k) +
+                                    " must be at least 1");
+            }
+            if (dr.contains("verify_rows")) {
+                p.drafter_verify_rows = int_field(dr, "verify_rows", "drafter", path);
+                if (p.drafter_verify_rows < 1)
+                    throw PlanError("plan: " + path + " drafter.verify_rows " +
+                                    std::to_string(p.drafter_verify_rows) + " must be at least 1");
+            }
+            if (dr.contains("tau")) {
+                if (!dr.at("tau").is_number())
+                    throw PlanError("plan: " + path + " drafter.tau must be a number");
+                p.drafter_tau = dr.at("tau").get<double>();
+                if (!(p.drafter_tau > 0.0 && p.drafter_tau <= 1.0))
+                    throw PlanError("plan: " + path + " drafter.tau " + std::to_string(p.drafter_tau) +
+                                    " is not in (0, 1] (it is an acceptance gate, not a temperature)");
+            }
+            if (dr.contains("tree")) {
+                p.drafter_tree = typed<std::string>(dr, "tree", "drafter", path);
+                if (p.drafter_tree != "leaf" && p.drafter_tree != "none")
+                    throw PlanError("plan: " + path + " drafter.tree \"" + p.drafter_tree +
+                                    "\" is not leaf or none");
+            }
+            // A named drafter with no k/tau/rows is a half-stated contract: the values exist to be checked against what
+            // the speculator actually does, so a plan that omits them cannot be honoured either.
+            if (p.has_drafter() && (p.drafter_k == 0 || p.drafter_verify_rows == 0 || p.drafter_tau == 0.0))
+                throw PlanError("plan: " + path +
+                                " names drafter.head but not k, verify_rows and tau (a drafter's numbers are part of "
+                                "its contract: the same prompt answers different text at a different tau)");
+        }
+        // When GDN state is committed (engine#799 §1).  Absent means immediate, which is what the step path does today.
+        if (j.contains("state")) {
+            const auto& st = obj(j, "state", path);
+            if (st.contains("gdn_commit")) {
+                p.gdn_commit = typed<std::string>(st, "gdn_commit", "state", path);
+                if (p.gdn_commit != "immediate" && p.gdn_commit != "deferred")
+                    throw PlanError("plan: " + path + " state.gdn_commit \"" + p.gdn_commit +
+                                    "\" is not immediate or deferred");
+            }
+        }
         // A second graph (a vision encoder, a projector) is deliberately NOT a plan field: one plan is one
         // decoder stream over one token sequence.  A hybrid is two model directories and a merge above them,
         // which is the serving layer's business -- a companion list would put two graphs and their scheduling in

tests/npu_plan_test.cpp

@@ -639,9 +639,63 @@ void test_rope_and_inputs() {
     }
 }
 
+// engine#799 §1: the two sections the spec names and the loader could not express -- `drafter {head, k, tau,
+// verify_rows, tree}` and `state {gdn_commit}`.  Both are absent-by-default (no drafter, immediate commit), and a
+// half-stated contract is refused by name, because a speculator with different numbers answers different text for the
+// same prompt and raises nothing.
+void test_drafter_and_state() {
+    {
+        const Plan p = Plan::load(write_plan("drafter_absent", base_plan()));
+        check(!p.has_drafter() && !p.gdn_deferred(), "no drafter and immediate GDN commit are the defaults");
+    }
+    {
+        auto j = base_plan();
+        j["drafter"] = {{"head", "draft_head.q4nx"}, {"k", 4}, {"tau", 0.6}, {"verify_rows", 4}, {"tree", "leaf"}};
+        j["state"] = {{"gdn_commit", "deferred"}};
+        const Plan p = Plan::load(write_plan("drafter_full", j));
+        check(p.has_drafter() && p.drafter_head == "draft_head.q4nx" && p.drafter_k == 4 &&
+                  p.drafter_verify_rows == 4 && p.drafter_tau == 0.6 && p.drafter_tree == "leaf",
+              "a fully stated drafter loads as written");
+        check(p.gdn_deferred(), "and a deferred GDN commit is reported");
+    }
+    {
+        auto j = base_plan();
+        j["drafter"] = {{"head", "draft.q4nx"}, {"k", 2}, {"tau", 1.0}, {"verify_rows", 2}};
+        const Plan p = Plan::load(write_plan("drafter_edge_tau", j));
+        check(p.drafter_tau == 1.0 && p.drafter_tree == "none", "tau 1.0 is allowed and tree defaults to none");
+    }
+    {
+        auto j = base_plan();
+        j["drafter"] = {{"head", "draft.q4nx"}, {"k", 2}, {"tau", 1.5}, {"verify_rows", 2}};
+        const std::string why = refusal(write_plan("drafter_tau", j));
+        check(has(why, "drafter.tau") && has(why, "(0, 1]"), "a tau outside (0,1] is refused, naming the range: " + why);
+    }
+    {
+        auto j = base_plan();
+        j["drafter"] = {{"head", "sub/draft.q4nx"}, {"k", 2}, {"tau", 0.5}, {"verify_rows", 2}};
+        const std::string why = refusal(write_plan("drafter_path", j));
+        check(has(why, "drafter.head") && has(why, "bare file name"),
+              "a drafter head with a directory part is refused (a plan cannot aim the loader): " + why);
+    }
+    {
+        auto j = base_plan();
+        j["drafter"] = {{"head", "draft.q4nx"}, {"k", 2}, {"verify_rows", 2}};
+        const std::string why = refusal(write_plan("drafter_half", j));
+        check(has(why, "drafter.head") && has(why, "tau"),
+              "a drafter with no tau is refused as a half-stated contract: " + why);
+    }
+    {
+        auto j = base_plan();
+        j["state"] = {{"gdn_commit", "promptly"}};
+        const std::string why = refusal(write_plan("state_bad", j));
+        check(has(why, "state.gdn_commit"), "an unknown gdn_commit is refused by name: " + why);
+    }
+}
+
 int main() {
     test_valid();
     test_rope_and_inputs();
+    test_drafter_and_state();
     test_kv();
     test_tile_order();
     test_refusals();