Skip to content

feat(dag): plan-driven fan-out of parallel writers - #861

Merged
LeXwDeX merged 3 commits into
mainfrom
feat/plan-fanout
Oct 22, 2026
Merged

LeXwDeX merged 3 commits into
mainfrom
feat/plan-fanout

Conversation

@LeXwDeX

@LeXwDeX LeXwDeX commented Oct 22, 2026

Copy link
Copy Markdown
Owner

Why

The number of parallel writers was fixed before the plan that decides the work packages had run. Templates either guessed a count or collapsed implementation into one writer and relied on the parent remembering a later extend. The trigger for expansion should be data, not timing.

What changed

  • coding blocks (and implementation nodes) accept fan_out: <plan>; the compiler gives that plan a packages: [{ id, instruction, write_set }] schema and a contract explaining concurrent writers.
  • When the writer becomes ready, the scheduler reshapes it through the durable extend path (DagFanOut.planFanOut): two or more provably disjoint packages run as writers <writer>--<package>, and the writer id is replaced in place by the ADR-0002 aggregator, so verify/review bindings are unchanged.
  • One package runs the writer as authored. Malformed/duplicate ids, empty or overlapping write sets, traversal paths, id collisions, or a node-ceiling breach run it serially with the reason in its prompt and logs; never a failed workflow.
  • Structural validation (create + replan), recovery remapping of fan_out, ADR-0004, CONTEXT glossary, and the block guide.

Evidence

  • New tests: test/dag/fan-out.test.ts (21) and test/dag/dag-fan-out-runtime.test.ts (4, real scheduler loop: expand, single package, overlap decline, ceiling decline).
  • Local: bun test test/dag test/tool 1457 pass (one intended schema snapshot update), bun run typecheck, lint 4673 (no new warnings), bun run test:dag-core pass.
  • Live: isolated dev server, DeepSeek tiers, real repo task. The plan produced 3 packages, the engine ran 3 parallel writers, the aggregator joined 6 files under one fingerprint, verify PASS, review ACCEPT, workflow completed in ~3.5 min; bun test in the target repo 3/3.
  • Local runs used bun 1.4.3 / node 26 (newer than the pins); pushed with --no-verify by owner authorization, so CI on the pinned toolchain is authoritative.

Checklist

  • Behavior covered by unit, runtime-loop, and live-model evidence.
  • No HTTP route, durable event, or database schema change; the model-facing workflow schema gains fan_out (snapshot updated).
  • Persisted workflows without fan_out compile and run unchanged.
  • Required native checks pass on the current head before merge.

🤖 Generated with Claude Code

Closes #859

A coding block (or implementation node) may declare fan_out: <plan>.
The plan then returns work packages; when the writer becomes ready the
scheduler reshapes it through the durable extend path: two or more
provably disjoint packages run as parallel writers and the writer id
becomes their ADR-0002 aggregator, so downstream verify/review bindings
are unchanged. Fewer packages, or a shape the engine cannot prove safe,
run the writer serially with the reason in its prompt.
@LeXwDeX
LeXwDeX marked this pull request as ready for review October 22, 2026 01:40
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-22T01:45:32.242974Z c5d3465 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c5d346547c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/opencode/src/dag/fan-out.ts
Comment thread packages/opencode/src/dag/fan-out.ts Outdated
Comment thread packages/opencode/test/dag/dag-fan-out-runtime.test.ts Outdated
Comment thread packages/opencode/src/dag/runtime/loop.ts Outdated
- Dag.extend takes an optional attempt fence checked under the workflow
  lock; fan-out expansion and its serial fallback are dropped when a
  parent replan or cancel reshaped the writer after the snapshot.
- Package writers get their own recovery identity; the aggregator keeps
  the writer's, so recovering two packages no longer collides.
- Write-set paths drop empty and '.' segments before overlap checks.
- The runtime test waits on a bounded admission predicate, not a sleep.
@LeXwDeX
LeXwDeX enabled auto-merge October 22, 2026 02:26
@LeXwDeX
LeXwDeX merged commit d230249 into main Oct 22, 2026
15 checks passed
@LeXwDeX
LeXwDeX deleted the feat/plan-fanout branch October 22, 2026 02:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: plan-driven fan-out of parallel writers

1 participant