Skip to content

Stop auto-fired full reviews from overwriting a concurrent #update-review - #21905

Merged
CamSoper merged 1 commit into
masterfrom
CamSoper/review-autofire-clobber
Sep 25, 2026
Merged

CamSoper merged 1 commit into
masterfrom
CamSoper/review-autofire-clobber

Conversation

@CamSoper

Copy link
Copy Markdown
Contributor

On #21871 the author's resolved card ("v4 — nothing blocks merge") was replaced with a fresh "v1 — N items block merge" three times: 09-24 15:32, 09-25 10:41 and 09-25 14:51. None of those came from a request to regenerate the review.

Cause: The author pushes, then comments #update-review seconds later. The update lane sets review:in-progress before the full review that triage auto-fires (via workflow_run) reaches its gate. That gate's already-reviewed skip didn't include in-progress, so the full review ran too (proceed (labels=…review:in-progress) in runs 36019781626, 36124379051 and 36149244382). It published several minutes after the update, at the same head.

Two existing guards should have caught it, and neither did:

  • handoff_guard.py only compares SHAs, and the head hadn't moved.
  • The stale-publish guard in pinned-comment.sh (Stop a stale review run from overwriting a newer card #21788) compares card stamps. The full review stamps its card at compose time, 6+ minutes after it read the PR. On 09-25 its stamp was 14:48:35 and the update's was 14:43:54, so the stale card looked newer.

Changes

  • Skip the auto-fire path on review:in-progress. review-label-reconcile.yml already clears orphaned in-progress labels.
  • Move Compute review timestamp to run directly after Resolve PR context, so the stamp shows when the run looked at the PR. Against Add blog post about component state migrations #21871's timeline, this change alone would have refused all three overwrites.
  • Add regression tests to test_handoff_guard.py. Both fail on master and pass here. make test-review-pipeline passes.

Trade-offs

  • A ready-transition that lands while an update is running now defers to the update instead of starting a second, full review.
  • The card's updated time now shows when the run started reading the PR, not when it finished composing.

🤖 Generated with Claude Code

…view

When an author pushes and then comments #update-review seconds later, the
update lane sets review:in-progress before triage's auto-fired full review
reaches its already-reviewed gate. The full review proceeded, finished
minutes after the update, and republished v1 over the author's resolved
card at the same head. This happened three times on #21871.

- Skip the auto-fire path on review:in-progress. Orphaned labels are
  already cleared by review-label-reconcile.yml.
- Take the card timestamp right after the run reads PR state, not at
  compose time, so pinned-comment.sh's stale-publish guard (#21788) can
  see that a card was published after this run looked.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Sentinel — merge gate status (preview)

Warning

⚠️ Preview mode — this check is NOT blocking your merge

The Sentinel is in preview (report-only) mode: its check-run concludes neutral whatever the gates below say, so nothing here gates this PR. The rows are informational and safe to ignore for now. It would have concluded failure.

This will be enforced in the near future. Once it is, every row below has to be green before this PR can merge. The rows are the sign-offs a merge needs — some yours, some a reviewer's or a deploy's. Tell us in #docs if one looks wrong.

1 of 5 gates would block this merge once the Sentinel is enforced.

Gate What it needs
✅ G1 review-ran Nothing to do: review current at 5c935b0b8.
✅ G2 findings-answered Nothing to do: every finding answered.
🔴 G3 right-approver Needs approval from a member of any review team (pulumi/docs-blog-review, pulumi/docs-guild, pulumi/docs-marketing-review, pulumi/docs-tools) or a repository administrator — no qualifying human approval at this head (bot approvals never count).
➖ G4 infra-evidence Doesn't apply to this PR: no changed path affects the deploy.
➖ G5 oversized-ack Doesn't apply to this PR: not oversized.

Break glass: a member of a routing team (.github/review-routing.yml teams:) can apply review:waived, which is logged.

Evaluated at head 5c935b0b8.

@CamSoper
CamSoper marked this pull request as ready for review September 25, 2026 17:34
@github-actions github-actions Bot added review:triaging Claude Triage is currently classifying the PR domain:infra PR touches workflows, scripts, infra, Makefile, or build config labels Sep 25, 2026
@pulumi-bot
pulumi-bot requested a review from a team September 25, 2026 17:35
@github-actions github-actions Bot added review:in-progress Claude review is currently running and removed review:triaging Claude Triage is currently classifying the PR labels Sep 25, 2026
@pulumi-bot

Copy link
Copy Markdown
Collaborator

@github-actions

Copy link
Copy Markdown
Contributor

Reviewer's guide v1 — not for the author

Tip

This is the reviewer's guide. Work through the ⚠️ checklist below, then approve — approving asserts only that the ⚠️ items looked right to you. Machine-verified this run: links, shortcodes, page metadata, and every claim marked verified (receipts on the evidence page). Code samples are read, not compiled.

PR author: your to-do list is the other review comment, "Author action guide" — nothing on this card is yours.

Approval needed from: @pulumi/docs-tools — any member's approval satisfies the merge gate.

Note

What this PR changes:

  • claude-code-review.yml: the auto-fire already-reviewed skip now also covers review:in-progress.
  • claude-code-review.yml: the "Compute review timestamp" step now runs right after "Resolve PR context" instead of at compose time.
  • test_handoff_guard.py: two new invariant tests pin the in-progress skip and the step order.

The risk here is a guard that skips legitimate reviews, or a stamp that lets a stale full review overwrite a newer card. The infra pass read both workflow changes against the new tests and the existing redispatch and reconcile paths.

Review confidence:

Dimension Level Notes
mechanics HIGH
facts HIGH

⚠️ Check these before approving

No findings to check — but the editorial stances below still need a human eye.

Editorial stances introduced by this PR

Superlative, ranking, or comparative language the diff adds. No verdict — a page's own framing isn't fact-checkable — but confirm each is a stance the docs should take, and that no agent-written rewrite introduced it unasked.

None — the extractor found no positioning or comparison language in this PR's added lines.

✅ What you can rubber-stamp

  • Facts: no factual claims found in the changed lines.
  • Mechanics: no internal links added.
  • Style: 0 advisory suggestions left with the author; never blocking.

💡 Pre-existing issues in touched files: 0 — details on the evidence page.

📎 Full evidence: verification trail, investigation log, review history.

Review v1 · updated 2026-09-25T17:36:40Z · head commit 5c935b0


For the reviewer: the ⚠️ items above are the minutes that matter — the receipts for everything machine-verified are on the evidence page. The author's open items live on their own card (the comment headed "Author action guide"); while any are open, a Waiting on the author list above tracks them, and merge stays blocked until they're answered. If something here seems off, comment on the PR — @claude <your point> #update-review re-adjudicates with your input.

@github-actions

Copy link
Copy Markdown
Contributor

Author action guide v1 — nothing blocks merge

Note

Nothing here blocks merge — no open items need an answer from you. A human reviewer still approves the merge.

This PR makes auto-fired full reviews skip PRs already labeled review:in-progress and moves the card timestamp to right after PR context is read; the review checked the workflow guard logic and the new invariant tests, and found nothing to fix.

🚨 Fix or disagree

Nothing to fix — this section is empty.

❓ Questions for you

No open questions for you.

📎 Full evidence: verification trail, investigation log, review history.

Review v1 · updated 2026-09-25T17:36:40Z · head commit 5c935b0


How to answer

Every 🚨 and ❓ item above needs one of these before merge:

  1. Fix it — push the change. If your push lines up with the flagged lines, this card shows a 🔄 banner within a minute and then refreshes itself. No banner? Comment:

    @claude I pushed a fix for F1 #update-review
    
  2. Disagree with it — say which item and why; the review re-checks with your input:

    @claude F2: <your reasoning — e.g. where the figure comes from> #update-review
    

    Either way your answer counts: the review marks the item resolved, or keeps it with a 🛡️ note for your human reviewer to weigh — it stops blocking merge in both cases.

  3. Accept it as-is — you own the PR; a one-line reason is a valid answer, and your reviewer sees it beside the finding:

    @claude F2: accepting as-is — <your reason> #update-review
    

    To accept every open item at once: @claude accepting all open items — <reason> #update-review.

The #update-review hashtag matters — it routes your reply to this review, and (besides pushing a fix) it is the only reply that unblocks merge. A bare @claude gets you ad-hoc help and leaves this card — and the merge block — untouched.

Please don't edit, hide, or delete this comment — it is the review's record. Full mechanics: CONTRIBUTING.md §AI-assisted contributions.

@github-actions github-actions Bot added review:no-blockers Claude review completed cleanly; outstanding is empty and removed review:in-progress Claude review is currently running labels Sep 25, 2026
@CamSoper
CamSoper merged commit 138ed61 into master Sep 25, 2026
18 checks passed
@CamSoper
CamSoper deleted the CamSoper/review-autofire-clobber branch September 25, 2026 18:03

This branch was successfully deployed

1 active deployment
testing — 5c935b0b Deployed Sep 25, 2026 by CamSoper via Do cleanup #14413
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

domain:infra PR touches workflows, scripts, infra, Makefile, or build config review:no-blockers Claude review completed cleanly; outstanding is empty

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants