Skip to content

ci: fix AI review workflow crash and make it advisory - #60

Open
shawnmuggle wants to merge 1 commit into
masterfrom
fix/ai-review-workflow
Open

ci: fix AI review workflow crash and make it advisory#60
shawnmuggle wants to merge 1 commit into
masterfrom
fix/ai-review-workflow

Conversation

@shawnmuggle

Copy link
Copy Markdown
Member

Problem

The review check is failing on every PR in this repo, red-flagging healthy code and putting PRs into BLOCKED merge state. It is a workflow defect, not a code defect.

Root cause

  1. --max-turns 15 was too low for the prompt's workload (read diff → inline comments → summary comment → structured verdict). Runs ended as error_max_turns with an empty structured_output.
  2. That empty value hit an unguarded expression in the approve step's env: block:
    SUMMARY: ${{ fromJSON(steps.review.outputs.structured_output).summary }}
    
    GitHub evaluates env: even when the step's if: is false, so the existing if: guard did not protect it. fromJSON('') is a workflow template parse error — Error reading JToken from JsonReader — which fails the whole run instead of skipping the step.

Observed in job logs: subtype: error_max_turns--json-schema was provided but Claude did not return structured_outputThe template is not valid ... (Line: 120, Col: 20).

Fix

  • Guard the SUMMARY expression independently of the if:, with a comment explaining why both guards are needed.
  • --max-turns 15 → 40 so the verdict is actually emitted.
  • continue-on-error: true on the review step — infra failures shouldn't gate a healthy PR.
  • New step surfaces a missing verdict in the job summary, so a soft failure is visible rather than silently green.

Safety

  • Cannot cause a silent approval. The approve step still requires an explicit has_blockers == false verdict. Failing soft degrades to no automated approval, never to approved.
  • Security boundary unchanged — still pull_request, never pull_request_target.
  • Model ID claude-opus-4-8 verified current and valid; left untouched.

Verification

YAML parses; trigger, permissions, and token handling confirmed unchanged.

🤖 Generated with Claude Code

The `review` check has been failing on every PR, red-flagging and blocking
merges on healthy code. Root cause chain:

1. `--max-turns 15` was too low for the prompt's workload (read the diff, leave
   inline comments, post a summary comment, AND return a structured verdict).
   The run ended as error_max_turns with an empty structured_output.

2. That empty value hit an UNGUARDED expression in the approve step's `env:`
   block: `fromJSON(steps.review.outputs.structured_output).summary`. GitHub
   evaluates `env:` even when the step's `if:` is false, so the guard on the
   `if:` did not protect it. fromJSON('') is a workflow *template* parse error
   ("Error reading JToken from JsonReader"), which fails the entire run rather
   than skipping the step.

Fixes:
- Guard the SUMMARY env expression independently of the `if:`, with a comment
  explaining why both guards are required.
- Raise --max-turns 15 -> 40 so the verdict is actually emitted.
- Mark the review step continue-on-error: an infra failure (turn exhaustion,
  rate limit, action outage) should not gate a healthy PR. The approve step
  still runs only on an explicit has_blockers == false verdict, so this
  degrades to "no automated approval", never to "silently approved".
- Add a step that records a missing verdict in the job summary, so a soft
  failure is visible rather than silently green.

Security boundary unchanged: still `pull_request`, never pull_request_target.
Model ID claude-opus-4-8 left as-is (verified current and valid).

Verified: YAML parses; confirmed the trigger and token handling are untouched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@vercel

vercel Bot commented Jul 26, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
intent-example Ready Ready Preview, Comment Jul 26, 2026 12:17pm

Request Review

@shawnmuggle

Copy link
Copy Markdown
Member Author

CI evidence on this PR

The review check now passes, and the step trace confirms the crash is fixed:

success  Claude Code review
success  Note when the review did not produce a verdict
skipped  Label + approve when no blockers

This is the exact scenario that previously killed the workflow. structured_output came back empty — and instead of fromJSON('') raising Error reading JToken from JsonReader and failing the whole run, the guarded env: expression handled it and the job completed. The approve step correctly stayed skipped, since there was no has_blockers == false verdict.

One caveat, stated plainly: the review step reported outcome: success in 14s without producing a verdict, which is too fast for a real review — the action almost certainly no-op'd (most likely no CLAUDE_CODE_OAUTH_TOKEN in this run's context). So this run proves the crash fix and the fail-soft behaviour; it does not exercise the --max-turns 15 → 40 change or verify review quality. That will only be observable on a run where the action genuinely executes.

Worth noting the no-op path is itself now safe: previously an empty verdict was fatal, so this failure mode would have red-flagged the PR regardless of cause.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant