A record flow re-splits after a handle resize; real-core lanes on core v0.71.13

docs/status.md

@@ -1,9 +1,9 @@
 # Status
 
 What `paged.data` ships and what it does not, read from the code on `main` at
-`@paged-media/data` 0.1.0-canary.16 (published 2026-10-13). The bundle declares
+`@paged-media/data` 0.1.0-canary.17 (2026-10-17). The bundle declares
 `@paged-media/plugin-api` and `@paged-media/plugin-sdk` `>=0.2.45-canary.0` as peers; its tests
-run against `@paged-media/canvas-wasm` 0.71.1 and the 0.2.45-canary.0 contract. How the parts
+run against `@paged-media/canvas-wasm` 0.71.13 and the 0.2.58-canary.0 contract. How the parts
 fit is in [`architecture.md`](architecture.md); the analysis behind the 2026-10-05 round of
 work, with file and line references and its outcome at the end, is
 [`design/analysis-2026-10-05.md`](design/analysis-2026-10-05.md). What each canary added is
@@ -125,6 +125,12 @@ under [Releases](#releases).
 - **Lowering**: a table binding lowers to a native table; a visibility binding sets
   `elementVisible`; a rule applies a named style to a story range, to one paragraph per
   fired record, or to table cells.
+- **A record flow over a live frame chain** (the session's `paginateChain` and
+  `subscribeChainReflow`): the chain and its frames' heights are read from the document, and
+  the flow is paginated again once a burst of resizes is quiet. A resize counts whether it is
+  a `resizeFrame` or a drag of a chain frame's handles; the second needs a host that reports
+  a gesture's reflow (core v0.71.13 through plugin-sdk 0.2.58). On an older host a handle
+  resize re-paginates nothing; a move never does. No panel calls these yet.
 - **Headless batch.** `paged-data-batch`, built from source, reads a JSON job with
   materialised query results and prints the paginated flow of each output document. A
   script evaluated in Boa can supply the locale, the parameters and the build.
@@ -376,6 +382,7 @@ publishes nothing.
 
 | Version | Published | Added |
 |---|---|---|
+| 0.1.0-canary.17 | not yet (a merge publishes it) | A record flow paginated over a live frame chain re-paginates when a chain frame is resized by its handles, not only by `resizeFrame`, on a host that reports a gesture's reflow (core v0.71.13, plugin-sdk 0.2.58). Data Merge reads a template's page-local bounds from the geometry on engines that report a page's origin (no hit-test probe) and no longer copies the knocked-out drop shadow core v0.71.13 reports on every frame. The real-core tests move to canvas-wasm 0.71.13 and the contract to 0.2.58-canary.0; peers stay `>=0.2.45-canary.0`. |
 | 0.1.0-canary.16 | 2026-10-13 | Record N is the Nth row in the order the query defines; the preview stepper and Data query's Preview work as soon as the query has run; merged frames keep the template frame's own formatting; Data Merge as the typed command `media.paged.data.merge`; a barcode re-lower or preview step draws the new symbol before removing the old one, in one batch (protocol 71 refused the old order). |
 | 0.1.0-canary.15 | 2026-10-11 | The data engine and DuckDB load their wasm through the host's module door, with integrity, and fall back to `bin/`; the published manifest carries each wasm's sha384. |
 | 0.1.0-canary.14 | 2026-10-11 | Messages a user sees say what happens in plain words instead of naming internal tracker or decision ids. |

packages/data-bundle/manifest.json

@@ -1,7 +1,7 @@
 {
   "id": "media.paged.data",
   "name": "paged.data",
-  "version": "0.1.0-canary.16",
+  "version": "0.1.0-canary.17",
   "apiVersion": "^0.2",
   "publisher": "paged.media",
   "capabilities": {

packages/data-bundle/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@paged-media/data",
   "private": false,
-  "version": "0.1.0-canary.16",
+  "version": "0.1.0-canary.17",
   "license": "AGPL-3.0-only OR LicenseRef-PMEL",
   "type": "module",
   "main": "./src/index.ts",
@@ -25,10 +25,10 @@
     "react": "^18.3.0"
   },
   "devDependencies": {
-    "@paged-media/canvas-wasm": "0.71.1",
+    "@paged-media/canvas-wasm": "0.71.13",
     "@paged-media/data-host-model": "workspace:*",
-    "@paged-media/plugin-api": "0.2.45-canary.0",
-    "@paged-media/plugin-sdk": "0.2.45-canary.0",
+    "@paged-media/plugin-api": "0.2.58-canary.0",
+    "@paged-media/plugin-sdk": "0.2.58-canary.0",
     "@types/node": "20.19.39",
     "@types/react": "^18.3.31",
     "apache-arrow": "^17.0.0",

packages/data-bundle/src/__tests__/merge.test.ts

@@ -148,6 +148,8 @@ describe("Data Merge writer, pure [data.lower.merge-writer]", () => {
     expect(isSetValue({ type: "text", value: "Drop" }, "frameDropShadowMode")).toBe(false);
     expect(isSetValue({ type: "length", value: 0 }, "frameGradientFillAngle")).toBe(false);
     expect(isSetValue({ type: "text", value: "None" }, "frameTextWrapMode")).toBe(false);
+    // Core v0.71.13 reads a knocked-out shadow on every frame, a fresh one too.
+    expect(isSetValue({ type: "bool", value: true }, "frameDropShadowKnockedOut")).toBe(false);
   });
 
   it("content: consuming a one-page merge removes the template frames in the same batch", () => {

packages/data-bundle/src/merge.ts

@@ -236,6 +236,9 @@ const FRESH_FRAME_DEFAULTS: Readonly<Record<string, readonly unknown[]>> = {
   frameGradientStrokeLength: [0],
   // Core reads "Drop" on every frame; only `frameDropShadow` turns it on.
   frameDropShadowMode: ["Drop"],
+  // Core reads a knocked-out shadow (InDesign's default) on every frame, a
+  // fresh one included: v0.71.13 does, v0.71.1 listed no such property.
+  frameDropShadowKnockedOut: [true],
   textFrameColumnCount: [1],
   textFrameVerticalJustification: ["TopAlign"],
   textFrameFirstBaseline: ["AscentOffset"],
@@ -363,13 +366,16 @@ export interface ReadTemplateOptions {
  * Read the record template off a page: its merge frames (page-local bounds,
  * text, formatting) and its margin box.
  *
- * Page-local bounds: `elementGeometry` answers raw bounds plus an item
- * transform in SPREAD space, and the spread origin of a page is not on any
- * read door. It is recovered from one text frame: `hitTest` (page-local in,
- * page-local `frameBounds` out) is asked at that frame's centre under the
- * usual page placements (single page centred on the spread, left or right of
- * a facing spread, spread origin), and the answer that names the frame back
- * fixes the offset for the whole page.
+ * Page-local bounds: an engine that reports where a page sits in its spread
+ * (`ElementGeometryItem.pageOrigin`) answers the item transform with the
+ * page's origin folded in, so the transformed bounds are page-local already.
+ * An older one answers raw bounds plus an item transform in SPREAD space, and
+ * the spread origin of a page is not on any read door. It is recovered from
+ * one text frame: `hitTest` (page-local in, page-local `frameBounds` out) is
+ * asked at that frame's centre under the usual page placements (single page
+ * centred on the spread, left or right of a facing spread, spread origin),
+ * and the answer that names the frame back fixes the offset for the whole
+ * page.
  */
 export async function readMergeTemplate(
   host: BundleHost,
@@ -403,6 +409,9 @@ export async function readMergeTemplate(
   // The text frames and their stories.
   type Raw = { element: ElementId; spread: Bounds; storyId: string | null; text: string | null };
   const raws: Raw[] = [];
+  // Every item of the page answers in page space when the engine names the
+  // page's origin.
+  const pageLocal = geoms.some((g) => (g as { pageOrigin?: unknown }).pageOrigin != null);
   for (const g of geoms) {
     const spread = transformBounds(g.bounds as Bounds, g.itemTransform as number[] | null);
     if (g.id.kind === "textFrame") {
@@ -420,12 +429,13 @@ export async function readMergeTemplate(
     return { template: null, diagnostics: ["merge: no text frame with a <<field>> on the template page"] };
   }
 
-  // Recover the page's spread origin from one confirmed hit.
-  let origin: [number, number] | null = null;
+  // Recover the page's spread origin from one confirmed hit (an older
+  // engine; a newer one answers in page space).
+  let origin: [number, number] | null = pageLocal ? [0, 0] : null;
   const s = firstText.spread;
   const cx = (s[1] + s[3]) / 2;
   const cy = (s[0] + s[2]) / 2;
-  for (const [ox, oy] of [
+  for (const [ox, oy] of origin ? [] : [
     [-pw / 2, -ph / 2],
     [-pw, -ph / 2],
     [0, -ph / 2],

packages/data-bundle/src/session.ts

@@ -739,7 +739,8 @@ export interface DataSourceSession extends ReviewSession, PropertyMethods {
    *  caller-supplied. */
   paginateChain(bindingId: string, storyId: string): Promise<unknown>;
   /** D-12: subscribe to content-box reflow so a catalog flow re-paginates when
-   *  its chain's frames resize. Returns a disposable; the callback fires with
+   *  its chain's frames resize (by a `resizeFrame`, or by their handles where
+   *  the host reports a gesture's reflow). Returns a disposable; the callback fires with
    *  the fresh paginated flow once per BURST of relevant reflows (a drag-resize
    *  streams one per step; only the settled chain is paginated). `options`
    *  sets the quiet period and injects the timers (tests). */
@@ -3463,7 +3464,12 @@ export function createSession(rawHost: BundleHost, today: number): DataSourceSes
     subscribeChainReflow(bindingId, storyId, onRepaginate, options) {
       // D-12: re-paginate when the chain's content boxes resize. A reflow event
       // carries ONLY a resize (never a transform, §8.5), so a transform-only
-      // change is ignored — exactly the pagination consumer contract.
+      // change is ignored — exactly the pagination consumer contract. Whatever
+      // committed it: a `resizeFrame` (`mutationApplied`) or a frame resized
+      // by its handles (`gestureCommitted`, which carries the reflow from core
+      // v0.71.13 through plugin-sdk 0.2.58). Keyed off the field's presence:
+      // an older host reports a handle resize without it, and nothing is
+      // re-paginated, as before.
       //
       // Coalesced: a drag-resize streams one event per step, and each
       // re-pagination is a full resolve + sort of the flow, of which only the

packages/data-bundle/test/chain-gesture-reflow-real-core.spec.ts

@@ -0,0 +1,140 @@
+// A record flow paginated over a frame chain re-splits when a chain frame is
+// resized by its HANDLES: a canvas gesture (begin, update, commit), which the
+// engine commits as a `gestureCommitted`, not a `resizeFrame` mutation. Core
+// v0.71.13 reports that gesture's reflow on the reply (core ADR 159),
+// plugin-sdk 0.2.58 passes it through `document.onDidChange`, and
+// `subscribeChainReflow` re-paginates on any change that carries one. Before,
+// only a `resizeFrame` carried it, and a handle resize left the split where
+// it was. A move by the same gesture kit carries none and re-paginates
+// nothing (display geometry only, §8.5).
+//
+// The REAL stack: real core (headless canvas-wasm), the real data-js wasm,
+// real DuckDB. Gate: skips without the pieces, EXCEPT under
+// REQUIRE_REAL_CORE=1 / REQUIRE_REAL_ENGINE=1 / REQUIRE_REAL_DUCKDB=1, where
+// a missing piece fails.
+
+import { afterAll, afterEach, describe, expect, it, vi } from "vitest";
+
+import type { HeadlessHost } from "@paged-media/plugin-sdk";
+
+import {
+  BUDGET_TIMEOUT_MS,
+  bootCountedDuck,
+  bootCountedEngine,
+  openDataHost,
+  productCsv,
+  RUN_BUDGETS,
+  sessionOver,
+  type CountedDuck,
+} from "./perf/harness";
+
+vi.setConfig({ testTimeout: BUDGET_TIMEOUT_MS });
+
+interface Flow {
+  frames: { frame: string; blocks: unknown[] }[];
+  placed: number;
+  total: number;
+  overflow: boolean;
+}
+
+type Reply = { kind: string; payload: Record<string, unknown> };
+
+/** The engine client under the headless host (the canvas's own channel). */
+function client(h: HeadlessHost): { send(m: unknown): Promise<Reply> } {
+  return (h.host as unknown as { editor: { client: { send(m: unknown): Promise<Reply> } } }).editor.client;
+}
+
+/** Run one gesture on `frameId` the way the canvas does: begun, moved by
+ *  `delta` (pt, snapping off), committed. Returns the commit's reply. */
+async function gesture(h: HeadlessHost, frameId: string, kind: unknown, delta: [number, number]): Promise<Reply> {
+  const c = client(h);
+  const begun = await c.send({ kind: "beginGesture", payload: { nodes: [{ kind: "textFrame", id: frameId }], gesture: kind } });
+  expect(begun.kind).toBe("gestureBegun");
+  const moved = await c.send({
+    kind: "updateGesture",
+    payload: { handle: begun.payload.handle, delta, modifiers: { shift: false, alt: false, disableSnap: true } },
+  });
+  expect(moved.kind).toBe("gestureUpdated");
+  const committed = await c.send({ kind: "commitGesture", payload: { handle: begun.payload.handle } });
+  expect(committed.kind).toBe("gestureCommitted");
+  return committed;
+}
+
+/** Let the subscription's debounce (16 ms) and the pass behind it run. */
+async function quiet(): Promise<void> {
+  await new Promise((r) => setTimeout(r, 60));
+  for (let i = 0; i < 20; i++) await new Promise((r) => setTimeout(r, 0));
+}
+
+describe.skipIf(!RUN_BUDGETS)("a record flow follows a handle resize, real core [data.lower.v43-consumers]", () => {
+  let h: HeadlessHost | null = null;
+  let duck: CountedDuck | null = null;
+  afterEach(() => {
+    h?.dispose();
+    h = null;
+  });
+  afterAll(async () => {
+    await duck?.handle.close();
+  });
+
+  /** A 40-record flow paginated over two threaded frames; the first frame's
+   *  id, the chain's story, the flows the subscription reported, and the
+   *  split before any resize. */
+  async function paginatedChain() {
+    h = await openDataHost();
+    const engine = await bootCountedEngine();
+    duck ??= await bootCountedDuck();
+    const s = await sessionOver(h.host, engine, duck);
+    const name = `p${Math.random().toString(36).slice(2, 8)}`;
+    await s.registerCsvSource(name, productCsv(40));
+    s.addQuery("q", `SELECT sku, name FROM ${name}`, "recordStream");
+    await s.refreshData();
+    s.defineRecordFlow("rf", "q", [{ expr: "sku" }], { lineHeightPt: 12 });
+
+    const frame = async (bounds: [number, number, number, number]) => {
+      const o = await h!.host.document.mutate({ op: "insertTextFrame", args: { pageId: "usp" as never, bounds } });
+      expect(o.applied).toBe(true);
+      return (o as { createdId: { id: string } }).createdId.id;
+    };
+    const fa = await frame([50, 50, 350, 250]);
+    const fb = await frame([50, 300, 700, 500]);
+    expect((await h.host.document.mutate({ op: "linkFrames", args: { from: fa, to: fb } })).applied).toBe(true);
+    const story = (await h.host.document.hitTest("usp" as never, [150, 200]))!.storyId as string;
+    expect(await h.host.document.frameChain(story)).toHaveLength(2);
+
+    const split = (f: Flow) => f.frames.map((x) => x.blocks.length);
+    const first = (await s.paginateChain("rf", story)) as Flow;
+    expect(first.placed).toBe(40);
+    const flows: Flow[] = [];
+    const sub = s.subscribeChainReflow("rf", story, (f) => flows.push(f as Flow));
+    return { fa, story, flows, sub, split, before: split(first) };
+  }
+
+  it("records move into the second frame when the first is shortened by its handle", async () => {
+    const { fa, flows, sub, split, before } = await paginatedChain();
+    expect(before).toHaveLength(2);
+    expect(before[1]).toBeGreaterThan(0);
+
+    // The first frame's bottom edge up by 150 pt, dragged by its handle.
+    await gesture(h!, fa, { kind: "resize", handle: "south" }, [0, -150]);
+    await quiet();
+    sub.dispose();
+
+    expect(flows.length, "the handle resize re-paginated the flow").toBeGreaterThan(0);
+    const last = flows[flows.length - 1];
+    const after = split(last);
+    expect(last.placed).toBe(40);
+    expect(after[0], "the first frame holds fewer records").toBeLessThan(before[0]);
+    expect(after[1], "the records it lost moved into the second").toBeGreaterThan(before[1]);
+    expect(after[0] + after[1]).toBe(before[0] + before[1]);
+  });
+
+  it("a move by the gesture kit re-paginates nothing", async () => {
+    const { fa, flows, sub } = await paginatedChain();
+    const committed = await gesture(h!, fa, { kind: "translate" }, [0, 40]);
+    expect(committed.payload.reflow ?? null, "a move reports no reflow").toBeNull();
+    await quiet();
+    sub.dispose();
+    expect(flows).toEqual([]);
+  });
+});

packages/data-bundle/test/merge-real-core.spec.ts

@@ -258,5 +258,44 @@ describe.skipIf(!ready && !REQUIRE_REAL_CORE)(
         );
       });
     }
+
+    it("a template's page-local bounds come from the geometry where the engine names the page's origin, and from a hit-test probe where it does not [data.lower.merge-writer]", async () => {
+      const { h, host } = await openTemplate((spec.fixtures as any[])[0].id);
+      hosts.push(h);
+      let hits = 0;
+      /** The host, counting hit tests; `strip` answers geometry as an engine
+       *  before v0.71.13 does, without the page's origin. */
+      const counted = (strip: boolean): BundleHost => {
+        const doc = new Proxy(host.document, {
+          get(o, p, r) {
+            if (p === "hitTest") {
+              return (...a: Parameters<BundleHost["document"]["hitTest"]>) => {
+                hits += 1;
+                return o.hitTest(...a);
+              };
+            }
+            if (p === "elementGeometry" && strip) {
+              return async (ids: Parameters<BundleHost["document"]["elementGeometry"]>[0]) =>
+                (await o.elementGeometry(ids)).map((g) => {
+                  const { pageOrigin: _origin, ...rest } = g as typeof g & { pageOrigin?: unknown };
+                  return rest;
+                });
+            }
+            return Reflect.get(o, p, r);
+          },
+        });
+        return new Proxy(host, { get: (o, p, r) => (p === "document" ? doc : Reflect.get(o, p, r)) });
+      };
+      const direct = await readMergeTemplate(counted(false));
+      expect(direct.template, direct.diagnostics.join("; ")).not.toBeNull();
+      const named = (await host.document.elementGeometry(direct.template!.frames.map((f) => f.element))).some(
+        (g) => (g as { pageOrigin?: unknown }).pageOrigin != null,
+      );
+      if (named) expect(hits, "no probe where the engine names the page's origin").toBe(0);
+      hits = 0;
+      const probed = await readMergeTemplate(counted(true));
+      expect(hits, "the probe ran without the page's origin").toBeGreaterThan(0);
+      expect(probed.template?.spec).toEqual(direct.template!.spec);
+    });
   },
 );

packages/data-bundle/test/perf/perf-budgets-merge.spec.ts

@@ -93,6 +93,13 @@ const normalise = (s: string) => s.replace(/\r/g, "\n").replace(//g, "");
 //  · −2 reads: the pages and the scene tree are read once for the template
 //    and the re-merge plan (they were read twice, nothing changing between).
 //    host calls 18 → 17, reads 12 → 11.
+//
+// W8 on core v0.71.13, which reports where a page sits in its spread
+// (`ElementGeometryItem.pageOrigin`) and answers the item transform in page
+// space: the template's page-local bounds need no hit test, so the probe for
+// the page's spread origin is gone. host calls 17 → 16, reads 11 → 10. (The
+// engine also reports `frameDropShadowKnockedOut` on every frame, a fresh one
+// included; it is a fresh-frame default and not copied: mutation ops stay 403.)
 const W8: Measured = onLabelled({
   // 148 → 147, 13 → 12: the same −1 read net as below (template frame read
   // +1, shared pages + tree −2); the reads do not depend on the engine.
@@ -109,7 +116,7 @@ const W8: Measured = onLabelled({
   keyAllocs: 0,
   fingerprints: 0,
   duckQueries: 0,
-}, { hostCalls: 17, hostReads: 11, mutates: 1, mutationOps: 403, undoSteps: 1, wasmCalls: 6 });
+}, { hostCalls: 16, hostReads: 10, mutates: 1, mutationOps: 403, undoSteps: 1, wasmCalls: 6 });
 
 describe.skipIf(!RUN_BUDGETS)("perf budgets — Data Merge [data.perf.gates]", () => {
   let h: HeadlessHost | null = null;

packages/data-host-model/package.json

@@ -15,7 +15,7 @@
     "prepublishOnly": "pnpm run build"
   },
   "dependencies": {
-    "@paged-media/plugin-api": "0.2.45-canary.0"
+    "@paged-media/plugin-api": "0.2.58-canary.0"
   },
   "devDependencies": {
     "typescript": "^5.9.3",