Skip to content

Dispatch a submitted review as one unit — batch reviews lost commands - #74

Merged
JPDuchesne merged 1 commit into
mainfrom
jpd/review-unit-dispatch
Sep 4, 2026
Merged

JPDuchesne merged 1 commit into
mainfrom
jpd/review-unit-dispatch

Conversation

@JPDuchesne

Copy link
Copy Markdown
Contributor

Closes #73.

Root cause (recap)

Submitting a review fires one pull_request_review_comment event per comment, all at once. The dispatch job's per-PR concurrency group admits one running + one pending run, and every newer arrival supersedes the pending one — so a review carrying several commands silently lost all but ~two (dev#140: two of three /asks evicted). One correction to the issue's step 3: skipped jobs don't hold the pending slot — the queued /asks were evicted by each other and by the subsequent /build on the same PR. Root cause unchanged: queue depth 1 per PR.

The dedup axis, verified

The issue asked for a live probe of which events fire per comment shape. The dev#140 run list already answers it: the manual thread replies each fired both a pull_request_review (submitted, empty body) and a pull_request_review_comment run. Every comment shape — batch submission, single comment, thread reply — is accompanied by exactly one review-submitted event, so command detection moves wholly to the review-level handler.

Design

  • pull_request_review_comment never dispatches (it keeps the instant 👀 ack). The review-submitted event is the dispatch unit.
  • ReviewUnit expands a review-summary context into per-comment dispatches: it enumerates the submitted review's own comments (GET /pulls/{n}/reviews/{review_id}/comments — event-sourced, never "latest review"), keeps the command-bearing ones, and bin/dispatch.rb runs each through the unchanged per-comment pipeline, sequentially inside the one run the concurrency group admits. Sibling commands can no longer evict each other.
  • Trust surface (plans#5 doctrine): a review is single-author by construction, so the hosted authorize gate on the submitter covers every enumerated comment. Defense-in-depth: a comment by any other author is dropped fail-closed with a log line.
  • Workflow: the engine can't read comment bodies from the review event, so authorize gains a hosted scan step (post-permission-gate) that detects commands across review body + comments and routes the runner label (build > learn > split > edit > ask); dispatch gates on its commanded output. Bodies ride env vars, never inline expansion. Cost: every authorized human review submission now runs a hosted authorize job (~10s, free on public repos) — previously only command-matching bodies did.
  • Dispatcher#run returns success instead of exiting — the fan-out must reach every comment even when an earlier one failed; bin/dispatch.rb owns the red exit.
  • A comment whose commands parse invalidly (lifecycle command in a batch) still dispatches, so the ⚠️ parse error lands on that comment instead of vanishing.

Verification

  • 282 tests green (new: ReviewUnit fan-out incl. author mismatch, reply-shaped implicit review, prose mentions, invalid batches; GitHub#review_comments; dispatcher soft-failure return), srb tc clean, rubocop clean, every added lib/ line covered (patch target is 100%).

Follow-ups (not this PR)

  • dev's /build sweep skips threads that start with a command, assuming the command pipeline answered them — with guaranteed delivery that assumption is sound again, but an unanswered command thread is still invisible to the sweep.
  • Cross-review races remain: two distinct surfaces submitted in quick succession still share the single pending slot.

Made with Cursor

…#73)

Submitting a review fires one pull_request_review_comment event per
comment, all at once; the dispatch job's per-PR concurrency group keeps
one running + one pending run, so a review carrying several commands
silently lost all but ~two (dev#140: two of three /asks evicted).

GitHub fires a single pull_request_review (submitted) event for every
comment shape — batch, single comment, thread reply — so that event is a
complete surface. Per-comment events no longer dispatch (they keep the
fast 👀 ack); the review event's one run enumerates the review's own
comments (ReviewUnit) and dispatches every command-bearing one
sequentially, each through the unchanged per-comment pipeline.

Trust: a review is single-author by construction, so the hosted authorize
gate on the submitter covers every enumerated comment; a comment by any
other author drops fail-closed. The engine can't read comment bodies from
the review event, so authorize gains a hosted scan step that detects
commands and routes the runner label; bodies ride env vars, never inline
expansion. Dispatcher#run now returns success instead of exiting — the
review fan-out must reach every comment even when an earlier one failed;
bin/dispatch.rb owns the red exit.

Co-authored-by: Cursor <cursoragent@cursor.com>
@codecov

codecov Bot commented Sep 4, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@JPDuchesne
JPDuchesne merged commit 8868506 into main Sep 4, 2026
4 checks passed
@JPDuchesne
JPDuchesne deleted the jpd/review-unit-dispatch branch September 4, 2026 18:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Batch review submissions lose slash commands: per-PR dispatch concurrency evicts queued runs

1 participant