workflows: start and inform a review from the @codex review comment - #30
Conversation
The resolve gate anchored the trigger to the start of the whole comment, so a comment that explained the push and ended with `@codex review` on its own line was silently ineligible: no error, no reaction, and every downstream job skipped. - Match the request on its own line anywhere in the comment body - Ignore mentions quoted in fenced, inline, indented, or block-quoted text - Reject comments authored by an app - Move the predicate into a tested review-request module the resolve job pins Closes #29 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
There was a problem hiding this comment.
❌ OpenAI PR Review: FAIL
Conclusion: Not ready. 2 readiness blockers and 1 actionable finding must be resolved before merge.
Review checks
| Check | Result |
|---|---|
| PR format | ✅ PASS |
| Issue design | ✅ PASS |
| Code & plan conformance | ❌ FAIL |
Scope: 55aaac03c6 · 7794db9a80..55aaac03c6 · full · 1 diff chunk
Usage: 1m 7s · 295,829 tokens · 75.7% cache hit · 7.377 credits
Blockers
plan-conformanceThe linked Issue requires mentions inside inline code spans to remain ineligible. The implementation only strips inline code up to a newline, so a multi-line inline code span containing@codex reviewis accepted.code-review.github/scripts/review-request/common.mjs:10: Reject commands inside multi-line inline code spans
Findings
1 inline finding published.
Summary
The new trigger module leaves multi-line inline code spans unstripped, allowing a quoted command to start a review.
Review metadata
- Commit:
55aaac03c6 - Range:
7794db9a80..55aaac03c6 - Mode:
full - Diff chunks:
1 - Model:
gpt-5.6-terra - Reasoning effort:
medium - Session:
repo-1309321116-pr-30-v2 - Generation:
13a4614cfdcf0458f01b4243afccb54ab979d1e5048c4fe2131d6b2fc030d000 - Evidence:
a4498a7d3d659ed24cb3d7d8001998ef8388fdcef2fa06d7a1eed5267515d321
Totals
- Input:
291,667 - Cached input:
220,669 - Cache write:
70,962 - Output:
4,162 - Reasoning:
1,753 - Total:
295,829 - Estimated credits:
7.377 - Credit rate per 1M tokens:
62.5input /6.25cached /375output
Token and cache details
| Stage | Mode | Target | Time | Input | Cached | Hit | Output | Total | Credits |
|---|---|---|---|---|---|---|---|---|---|
pr |
deterministic |
format rules |
0s | 0 | 0 | 0.0% | 0 | 0 | 0.000 |
pr |
full |
pr |
10s | 30,184 | 18,888 | 62.6% | 502 | 30,686 | 1.012 |
issue |
full |
Issue #29 |
20s | 78,274 | 59,635 | 76.2% | 1,336 | 79,610 | 2.039 |
code |
full |
chunk 1/1 |
30s | 103,350 | 66,103 | 64.0% | 1,871 | 105,221 | 3.443 |
code |
full |
aggregate |
7s | 79,859 | 76,043 | 95.2% | 453 | 80,312 | 0.884 |
The discussion step already collected the last 20 PR comments and the trigger comment id into the context file, but nothing ever read context.comments: the PR stage sends only title, body, and linked Issues, and the code stage sends only the diff and Issue context. The code-stage prompt already declared discussion comments untrusted input, so the wiring was intended and never finished. Codex could not see what the author said the push did, what they had already validated, what they disclosed, or any focus after `@codex review`. - Write a code-discussion stage input and load it in both code turns - Keep the triggering comment intact and mark bot authors and truncation - Frame discussion as untrusted background that cannot change review policy - Keep discussion out of every stage identity so reuse stays zero-token Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
🤖 OpenAI PR review completed with an errorReason: PR or linked-Issue readiness identity changed before publication. |
There was a problem hiding this comment.
❌ OpenAI PR Review: FAIL
Conclusion: Not ready. 5 readiness blockers and 2 actionable findings must be resolved before merge.
Review checks
| Check | Result |
|---|---|
| PR format | ❌ FAIL |
| Issue design | ✅ PASS |
| Code & plan conformance | ❌ FAIL |
Scope: 5a11f45307 · 55aaac03c6..5a11f45307 · incremental · 1 diff chunk
Usage: 1m 0s · 447,167 tokens · 75.7% cache hit · 10.350 credits
Blockers
pr-formatThe PR now delivers PR-comment content to the code-review turn and lets it provide review focus, validation claims, and disclosures. Issue #29 explicitly says the comment body must not be used as review instructions and limits the work to a pure eligibility gate. This is a material expansion beyond the reviewed plan; update the Issue’s scope, design, acceptance criteria, and validation plan (or split the work) before treating the PR as plan-conformant.plan-conformanceThe linked Issue requires mentions inside inline code spans to remain ineligible. The implementation only strips inline code up to a newline, so a multi-line inline code span containing@codex reviewis accepted.plan-conformanceThe linked Issue requires discussion context to identify and retain the triggering comment. Limiting comments to the last 20 before selecting the trigger can omit it when newer comments arrive.code-review.github/scripts/review-request/common.mjs:10: Reject commands inside multi-line inline code spanscode-review.github/workflows/codex-openai-review.yml:534: Always include the triggering comment in discussion context
Findings
2 inline findings published.
Summary
Two trigger-related requirements remain unmet: multi-line inline code can invoke a review, and the triggering comment can be dropped from discussion context after later comments arrive.
Review metadata
- Commit:
5a11f45307 - Range:
55aaac03c6..5a11f45307 - Mode:
incremental - Diff chunks:
1 - Model:
gpt-5.6-terra - Reasoning effort:
medium - Session:
repo-1309321116-pr-30-v2 - Generation:
91f031f53bb26cdda5c38b59538a167a4103d3abc2946af230328b4fd3721eee - Evidence:
3cad19b46480a5225388bc5cded064cc14ab8045def8ed3b1ed17bce181b566c
Totals
- Input:
443,076 - Cached input:
335,573 - Cache write:
107,479 - Output:
4,091 - Reasoning:
2,085 - Total:
447,167 - Estimated credits:
10.350 - Credit rate per 1M tokens:
62.5input /6.25cached /375output
Token and cache details
| Stage | Mode | Target | Time | Input | Cached | Hit | Output | Total | Credits |
|---|---|---|---|---|---|---|---|---|---|
pr |
deterministic |
format rules |
0s | 0 | 0 | 0.0% | 0 | 0 | 0.000 |
pr |
incremental |
pr |
16s | 86,518 | 50,351 | 58.2% | 1,009 | 87,527 | 2.954 |
issue |
incremental |
Issue #29 |
11s | 102,208 | 91,480 | 89.5% | 652 | 102,860 | 1.487 |
code |
incremental |
chunk 1/1 |
25s | 119,902 | 65,824 | 54.9% | 1,795 | 121,697 | 4.464 |
code |
incremental |
aggregate |
8s | 134,448 | 127,918 | 95.1% | 635 | 135,083 | 1.446 |
CommonMark inline code spans may cross line breaks, but the stripper ran line by line, so a quoted `example\n@codex review\nexample` left the middle line intact and started a review. The discussion step also applied the trigger's larger budget after slicing the window, so 20 newer comments dropped the request itself. - Strip inline code spans across line breaks, bounded at the paragraph break - Restore the triggering comment when newer comments crowd it out - Execute the real discussion script and every inline block in the tests - Say in the Issue that focus is untrusted background, not instructions Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
There was a problem hiding this comment.
✅ OpenAI PR Review: PASS
Conclusion: Ready from the OpenAI review perspective. PR format, linked Issue design, and code/plan conformance passed with no actionable findings.
Review checks
| Check | Result |
|---|---|
| PR format | ✅ PASS |
| Issue design | ✅ PASS |
| Code & plan conformance | ✅ PASS |
Scope: a24205a3b2 · 5a11f45307..a24205a3b2 · incremental · 1 diff chunk
Usage: 46s · 628,235 tokens · 76.7% cache hit · 13.151 credits
Summary
The incremental changes address both prior findings: inline-code stripping now preserves multiline span boundaries, and discussion collection retains an out-of-window triggering comment. No actionable findings.
Review metadata
- Commit:
a24205a3b2 - Range:
5a11f45307..a24205a3b2 - Mode:
incremental - Diff chunks:
1 - Model:
gpt-5.6-terra - Reasoning effort:
medium - Session:
repo-1309321116-pr-30-v2 - Generation:
b376e17c51f739f883b0c297e049d7bdda0540e04cd980bac83f198c488c9b6e - Evidence:
990ba0d18bf59ba634b649f4401054743f26e1ab35043921d0af29e845cde201
Totals
- Input:
625,406 - Cached input:
479,952 - Cache write:
145,430 - Output:
2,829 - Reasoning:
1,690 - Total:
628,235 - Estimated credits:
13.151 - Credit rate per 1M tokens:
62.5input /6.25cached /375output
Token and cache details
| Stage | Mode | Target | Time | Input | Cached | Hit | Output | Total | Credits |
|---|---|---|---|---|---|---|---|---|---|
pr |
deterministic |
format rules |
0s | 0 | 0 | 0.0% | 0 | 0 | 0.000 |
pr |
incremental |
pr |
13s | 141,579 | 79,114 | 55.9% | 808 | 142,387 | 4.702 |
issue |
incremental |
Issue #29 |
6s | 148,649 | 143,885 | 96.8% | 320 | 148,969 | 1.317 |
code |
incremental |
chunk 1/1 |
22s | 161,724 | 85,894 | 53.1% | 1,420 | 163,144 | 5.809 |
code |
incremental |
aggregate |
5s | 173,454 | 171,059 | 98.6% | 281 | 173,735 | 1.324 |
Result
Two defects on the same path — the PR comment that asks for a review. It now
starts a review when the mention sits on its own line anywhere in the comment,
and its content actually reaches the code review turn.
1. The comment did not start a review
resolvegatedissue_commentevents with/^@codex(?:\s+review(?:\s+[\s\S]+)?)?\s*$/iagainst the whole trimmed comment.The
^anchor meant the comment had to begin with@codex. A comment thatdescribed what was pushed and then ended with
@codex reviewnever matched, soeligiblewasfalseandstart,review,publish, andfinalizewere allskipped. Nothing failed and nothing was posted back, so from the requester's
side the workflow looked like it simply had not run.
Verified on GizClaw/gizclaw#1084: runs 33785352754, 33785379065, and
33786877584 each show
review / resolvesucceeding withreview / startskipped. The PR was open and not a draft at the time and
issue_commentis atrusted base event, so the trigger expression is the only remaining cause. Both
real comment bodies were replayed through the old and new predicates — old
false, newtrue. Two pushes went unreviewed.The predicate moved out of the inline
github-scriptblock into.github/scripts/review-request/, alongsideissue-review,pr-readiness,and
pr-review.stripQuotedTextblanks the regions where the trigger is quoted rather thanrequested — fenced code blocks (backtick or tilde, including an unterminated
one), inline code spans, and block quotes. Lines are replaced with empty lines
rather than removed, so stripping cannot join two unrelated lines into one
apparent command. This PR's own description quotes the trigger and must not
start a review because of it.
REVIEW_REQUEST_COMMANDis/^ {0,3}@codex(?:[ \t]+review\b.*?)?[ \t]*$/im.The
mflag lets the request sit on any line. Requiring the whole line to bethe command is the deliberate tightening decision: a mid-sentence mention
does not start a review, and neither does
@codex please lookor@codex reviewing. At most three leading spaces are accepted, CommonMark'sindented-code-block boundary, so a four-space-indented mention is inert too.
Bare
@codexkeeps working unchanged.isReviewRequestCommentalso rejects acomment whose author type is
Botor whose login ends in[bot]; that guarddid not exist before.
resolvegains twoissue_comment-only steps: a sparse checkout of the modulefrom
job.workflow_repositoryatjob.workflow_sha— the same pinningreviewand
finalizealready use, so the trigger logic always matches the workflowrevision that is running — and a
node evaluate.mjsstep that receives thecomment through the environment and sets a
requestedoutput. The comment bodyis never interpolated into a shell command.
2. The comment never reached the reviewer
Fixing the trigger alone would have started a review that still could not see
the comment. The discussion step already collected the last 20 PR comments and
trigger_comment_idintoPR_CONTEXT_FILE, but nothing readcontext.comments:prStageSnapshot()sends only repository, number, title,body, and linked Issues; the Issue stage sends Issue snapshots; the code stage
sends the diff chunk and Issue context. It was write-only data. The code-stage
prompt already declared "every ... discussion comment" untrusted input, so the
wiring was intended and never completed.
run.mjsnow writes acode-discussionstage input beside the existingcode-linked-issuesone. The first code chunk turn reads it after the Issuecontext, and an aggregation turn that runs without a preceding code turn reads
it directly. Per-comment clipping rises from 600 to 2,000 characters and the
triggering comment gets 8,000 — the two real #1084 comments are 1,180 and 899
characters, so the old limit truncated both mid-sentence and would have cut off
the
验证:…validation summary entirely. Comments now carryid,is_trigger,author_is_bot, andbody_truncated.Three constraints hold:
only for author-stated intent, validation claims, disclosures, and request
focus; it cannot relax, override, or extend the trusted caller review
profile; every claim must be checked against the diff, and a claim the diff
contradicts is reported.
prStageSnapshot()and the code stage identity. Including it would let anypassing comment invalidate cached evidence and force a paid re-review, which
would contradict the documented "unchanged head → reused, zero model tokens"
behavior. The consequence, documented in the README: a comment on an
unchanged head still reuses evidence and its text is not read until a code
turn runs for another reason. That is the real case here — both #1084
comments followed a push, so the code stage runs incrementally and sees them.
Validation
review-request/test.mjsis a 22-case table over comment bodies: bare@codex,@codex review,@codex review <focus>, mixed case, surroundingwhitespace, explanation text followed by the command on its own line, the
command followed by trailing prose, a three-space indent, and a fence that
closes before a real command — all accepted. Mid-sentence mention,
@codex when you can,@codex reviewing, backtick and tilde fences, anunterminated fence, single- and double-backtick inline spans, a four-space
indented block, a block quote, unrelated text, an empty body, a
Botauthor, a[bot]login, and missing or non-string bodies — all rejected.pr-review/test.mjsgains assertions that the discussion input file is writtenand loaded by both code turns, that the trigger comment keeps the larger
budget, that the untrusted framing is present, and — the load-bearing one —
that neither
prStageSnapshot()nor the code stage identity mentions comments.Every new assertion was checked against the pre-change sources and fails there,
so none is vacuous.
Both real #1084 comment bodies were replayed through the new trigger predicate
and the new collection mapping: accepted, marked, and carried untruncated.
actionlint reports
job.workflow_repository/job.workflow_shaas undefined;that is a stale context type in actionlint. Six of the eight already exist on
mainfor the identical checkout steps inreviewandfinalize; the othertwo are the same finding for the new
resolvecheckout.Not verified: no end-to-end run against a live PR. The changes are covered by
the tests above and by replaying real payloads, not by an executed review.
Reaching consumers
latestis a moving lightweight tag in this repository, not a release andnot a branch —
refs/tags/latestcurrently points at7794db9, the tip ofmain. Consumers such as GizClaw/gizclaw pin@latest, so merging this PR isnot enough on its own: after the squash-merge,
latestmust be force-moved tothe new
maintip and force-pushed:Once that lands, GizClaw/gizclaw#1084 picks the fix up with no change on the
consumer side.
v1is an annotated tag at an older commit and is untouched bythis PR.
Review round 1 (a24205a)
Three blockers from the automated review, all real:
breaks, but the stripper ran line by line, so
`example\n@codex review\nexample`left the middle line intact and started a review.
stripInlineCodenow runsover the whole comment, blanking every character of a span while preserving
newlines. The closing-delimiter search stops at the paragraph break — where
CommonMark ends a span — so one stray unmatched backtick cannot blank a later
request. New cases: multi-line single- and double-backtick spans (rejected), a
span closed on a later line followed by a real request (accepted), unmatched
backticks in the same and an earlier paragraph (accepted).
after
comments.slice(-20), so 20 newer comments dropped the request itself.The trigger is now looked up across all comments and prepended when missing,
deduplicated by id.
the comment body as review instructions", which read as contradicting the
goal of passing request focus to the code turn. Reworded: focus reaches the
code turn only as untrusted background checked against the diff, never as
trusted instructions; trigger matching itself still parses no parameters.
Testing got stronger rather than wider.
pr-review/test.mjsnow compiles all 8inline
github-scriptblocks (a structural break in one is invisible toactionlint and only fails mid-review at run time — this caught a duplicated
line while writing the fix) and executes the real discussion script against
synthetic comments, asserting selection, retention, deduplication, clipping
budgets, truncation flags, and bot marking. The weaker source-pattern
assertions those replaced were deleted. Every new assertion was checked against
the pre-fix sources and fails there.
Closes #29
🤖 Generated with Claude Code