Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 45 additions & 0 deletions .github/actions/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,13 @@ The `pr-review` action additionally accepts:
Automatic mode runs the independent production-failure analysis for initial and high-risk
reviews, but skips it for ordinary incremental updates.
`on` always enables it and `off` disables it.
- `state_artifact` - optional, defaults to `true`.
Uploads the run's `.pr-review` state directory as an artifact. Set it to `false` only if the
repository forbids artifacts.
- `state_artifact_retention_days` - optional, defaults to `14`.
- `debug_logs` - deprecated and ignored. The analysis session's full output is now always
printed. Consumers still passing it are unaffected; drop it at your convenience.


The semantic-analysis stage runs under a turn budget: 12 for a low-risk incremental review,
44 for a strong-tier one, and 56 for `deep`. Set `max_turns` to override any of these.
Expand Down Expand Up @@ -155,8 +162,46 @@ Questions published before the `<details>` shape existed fall back to a single-l
A question that is already open is never re-asked; the original stays the copy the author
answers.

### Inspecting a review run

The published review says what the reviewer concluded. These say how it got there, and they
are the starting point for tuning the rubric, the prompt, or the turn budget.

**Step summary** (the run's front page) carries the routing decision and its reason, the head
range, the model and turn budget the analysis actually ran with, the model tier, high-risk and
pre-mortem flags, and this round's counts — new findings split into inline and review-body,
new questions, prior findings resolved. It then folds in two blocks: the **pipeline trace** and
the **raw model output before compilation**. A failed run gets the same trace, which shows how
far the round got before it stopped.

**Job log** groups, in step order:

- `Review routing` (prepare) — mode and why, prior state source and whether its version still
matches, previous and current head, the compare status and file count, both diff sizes, how
much of the PR conversation was included versus truncated, the paths in scope, and every open
prior finding and question the model was handed.
- `Analysis settings` (compose) — model, tier, depth, pre-mortem, turn budgets, allowed tools.
- The analysis step itself prints the session's full output, always: every tool call, which
files it opened, and which it never read. The step is collapsed until you expand it.
- `Model output (raw, before compilation)` and `Compilation decisions` (compile) — one line per
model result the compiler accepted, suppressed as a duplicate of an open finding, dropped at
the per-severity cap of five, or rerouted to the review body because its line is not
commentable, plus each prior finding and question disposition.
- `Publication` (publish) — review ID, inline comments requested versus posted, whether inline
publication fell back to the review body, threads resolved, and the sticky comment ID.

**Run artifact** `pr-review-state-<pr>-<run>-<attempt>` holds the bytes themselves, for 14 days
by default: `review-input.json` (everything the model was given), `review.diff` and `full.diff`,
`analysis-transcript.json` (the session's turn-by-turn record, plus a `-retry-` twin when the
retry ran), `structured-output.json`, `model-output.json`, `review-payload.json` (the compiled
review, including its decision trace), `publish-result.json`, and `trace.log`.

"The model missed it" and "the pipeline dropped it" look identical in the published review and
different in these. Compare `model-output.json` against the `Compilation decisions` group first.

### Comment-triggered reconciliation (opt-in)


By default the review only runs on `pull_request` events, so an author who answers an open
question in a PR comment sees nothing happen until the next push. A consumer can also let a
comment drive a reconcile round by adding an `issue_comment` trigger:
Expand Down
102 changes: 96 additions & 6 deletions .github/actions/claude-pr-review/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,10 +53,24 @@ inputs:
required: false
default: "false"
description: >
`true` prints the analysis session's full output, including its tool
calls, into the job log. Off by default because the log is long and the
review itself is the product; turn it on to answer why a review reached
the verdict it did — which files it opened, and which it never read.
Deprecated and ignored. The analysis session's full output — its tool
calls, the files it opened, and the ones it never read — is now always
printed, and the whole `.pr-review` state directory plus the session
transcript are uploaded as a run artifact. Kept only so consumers that
still pass it do not break.
state_artifact:
required: false
default: "true"
description: >
`true` uploads the `.pr-review` state directory — routing input, both
diffs, the raw model output, the compiled payload, the pipeline trace,
and the analysis session transcript — as a run artifact. This is the
only copy that outlives the runner, so leave it on unless the repository
forbids artifacts.
state_artifact_retention_days:
required: false
default: "14"
description: Retention in days for the review state artifact.
premortem:
required: false
default: auto
Expand Down Expand Up @@ -172,6 +186,21 @@ runs:
echo "allowed_tools=${allowed_tools}" >> "$GITHUB_OUTPUT"
echo "runtime_flags=${runtime_flags}" >> "$GITHUB_OUTPUT"
echo "retry_runtime_flags=${retry_runtime_flags}" >> "$GITHUB_OUTPUT"
# Surfaced so the run report can state the settings the analysis
# actually used. Reading them back out of the flag string later would
# be guesswork.
echo "selected_model=${selected_model:-default}" >> "$GITHUB_OUTPUT"
echo "max_turns=${max_turns}" >> "$GITHUB_OUTPUT"
echo "retry_max_turns=${retry_turns}" >> "$GITHUB_OUTPUT"

echo "::group::Analysis settings"
echo "model: ${selected_model:-Claude Code default}"
echo "model tier: ${MODEL_TIER}"
echo "review depth: ${REVIEW_DEPTH}"
echo "pre-mortem: ${RUN_PREMORTEM}"
echo "max turns: ${max_turns} (retry ${retry_turns})"
echo "allowed tools: ${allowed_tools}"
echo "::endgroup::"

- name: Analyze and verify findings
id: review
Expand All @@ -182,7 +211,10 @@ runs:
claude_code_oauth_token: ${{ inputs.claude_code_oauth_token }}
allowed_bots: ${{ inputs.allowed_bots }}
display_report: false
show_full_output: ${{ inputs.debug_logs == 'true' }}
# Always on. The review body says what the model concluded; only this
# says how it got there — which files it opened, and which it never
# read. The step is collapsed in the job log until you expand it.
show_full_output: true
additional_permissions: |
actions: read
claude_args: |
Expand Down Expand Up @@ -323,7 +355,7 @@ runs:
claude_code_oauth_token: ${{ inputs.claude_code_oauth_token }}
allowed_bots: ${{ inputs.allowed_bots }}
display_report: false
show_full_output: ${{ inputs.debug_logs == 'true' }}
show_full_output: true
additional_permissions: |
actions: read
claude_args: |
Expand Down Expand Up @@ -359,6 +391,46 @@ runs:
Markdown, or a fenced JSON block. Empty arrays are correct for a clean
change.

- name: Capture analysis session transcript
if: always()
continue-on-error: true
shell: bash
env:
EXECUTION_FILE: ${{ steps.review.outputs.execution_file }}
RETRY_EXECUTION_FILE: ${{ steps.review_retry.outputs.execution_file }}
SESSION_ID: ${{ steps.review.outputs.session_id }}
RETRY_SESSION_ID: ${{ steps.review_retry.outputs.session_id }}
STRUCTURED_OUTPUT: ${{ steps.review.outputs.structured_output }}
RETRY_STRUCTURED_OUTPUT: ${{ steps.review_retry.outputs.structured_output }}
STATE_DIR: ${{ github.workspace }}/.pr-review
run: |
set -uo pipefail
# The execution file is the model's own turn-by-turn record: every
# tool call, its arguments, and its result. It lives in a temp path
# that dies with the runner, so copy it into the state directory the
# artifact step uploads.
mkdir -p "$STATE_DIR"
copy_transcript() {
local source="$1" name="$2" session="$3"
if [[ -n "$source" && -f "$source" ]]; then
cp "$source" "$STATE_DIR/$name"
echo "captured $name from $source (session ${session:-unknown})"
else
echo "no transcript available for $name"
fi
}
copy_transcript "$EXECUTION_FILE" "analysis-transcript.json" "$SESSION_ID"
copy_transcript "$RETRY_EXECUTION_FILE" "analysis-retry-transcript.json" "$RETRY_SESSION_ID"
# Keep the raw structured output beside the transcript. `compile` also
# writes a parsed copy, but it never runs when compilation is what
# failed, and this is exactly the case you want the bytes for.
if [[ -n "$STRUCTURED_OUTPUT" ]]; then
printf '%s' "$STRUCTURED_OUTPUT" > "$STATE_DIR/structured-output.json"
fi
if [[ -n "$RETRY_STRUCTURED_OUTPUT" ]]; then
printf '%s' "$RETRY_STRUCTURED_OUTPUT" > "$STATE_DIR/structured-output-retry.json"
fi

- name: Compile and validate review
id: compile
shell: bash
Expand Down Expand Up @@ -399,7 +471,25 @@ runs:
PUBLISH_OUTCOME: ${{ steps.publish.outcome }}
PUBLISH_PUBLISHED: ${{ steps.publish.outputs.published }}
PUBLISH_STALE: ${{ steps.publish.outputs.stale }}
ANALYSIS_MODEL: ${{ steps.compose.outputs.selected_model }}
ANALYSIS_MAX_TURNS: ${{ steps.compose.outputs.max_turns }}
ANALYSIS_RETRY_MAX_TURNS: ${{ steps.compose.outputs.retry_max_turns }}
ANALYSIS_REVIEW_DEPTH: ${{ inputs.review_depth }}
ANALYSIS_ALLOWED_TOOLS: ${{ steps.compose.outputs.allowed_tools }}
run: |
set -euo pipefail
python3 "$GITHUB_ACTION_PATH/review_pipeline.py" report \
--state-dir "$STATE_DIR"

- name: Upload review state for inspection
if: always() && inputs.state_artifact == 'true'
continue-on-error: true
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
# Run ID and attempt keep reruns from colliding; the PR number keeps
# concurrent reviews in one run distinguishable.
name: pr-review-state-${{ github.event.pull_request.number || github.event.issue.number }}-${{ github.run_id }}-${{ github.run_attempt }}
path: ${{ github.workspace }}/.pr-review
if-no-files-found: warn
retention-days: ${{ inputs.state_artifact_retention_days }}
include-hidden-files: true
Loading
Loading