feat(approve): opt-in approve.github_review mirrors approved as a GitHub review - #263
Conversation
…oken's APPROVE review While a pull request carries approved, the token keeps one APPROVE review on the head commit; when approved goes away the marked reviews are dismissed with the reason. Off by default: no GET /user, no review read or written. The mirrored review and every review by the token's own user never count toward approved, and its own pull_request_review events evaluate nothing. A refused approval (403, Actions not permitted, self-approval) is a warning. Signed-off-by: Mario Fahlandt <mfahlandt@pixel-haufen.de>
Signed-off-by: Mario Fahlandt <mfahlandt@pixel-haufen.de>
…penSSF Scorecard Signed-off-by: Mario Fahlandt <mfahlandt@pixel-haufen.de>
Signed-off-by: Mario Fahlandt <mfahlandt@pixel-haufen.de>
|
Coverage check on this head (
Every line this PR adds is exercised — including the new 🐝 Hive Agent: — hive: agent=quality backend=copilot model=claude-fable-5.1 copilot=1.0.88 |
jeefy
left a comment
There was a problem hiding this comment.
The code looks solid: it's off by default, re-runs are safe, only marked bot reviews get dismissed, and dist/ matches a fresh npm run pack. The docs need to be accurate before this merges:
docs/automatic-merging.md(~L453–464):approvedstays on a PR across pushes, so the bot re-approves every new head. That makes "Dismiss stale approvals" and "Require approval of the most recent reviewable push" ineffective. Please say so plainly, and say that the real protection after a push is thatlgtmis removed on every push, plus the requiredprow/lgtmstatus, which must stay required.docs/automatic-merging.md(~L444) and thegithub_reviewrow indocs/configuration.md: "Allow GitHub Actions to create and approve pull requests" is a repo-wide setting. Once it's on, any workflow, including one a write-access collaborator adds on their own branch, can approve PRs withGITHUB_TOKEN. Add a warning and suggest a dedicated App or machine-user token as the safer setup.- Scorecard section (~L473): add one sentence saying the required-review credit is nominal with this setup, because the review mirrors the Prow decision rather than being an independent check.
Nits, not blocking:
approveReview.ts:201turns every 403 into the "enable Allow GitHub Actions…" warning, including a workflow that's just missingpull-requests: write. Only give that advice when the response says "not permitted".- A refused dismissal (
:170) throws on every evaluation until someone dismisses the review by hand. Document that. - Check whether GitHub accepts an approval on a draft PR, or skip review sync when
pull.draftis true.
Signed-off-by: Jeffrey Sica <me@jeefy.dev>
…er msw 3 msw 3 ignores onUnhandledRequest and lets unmocked requests through to the network; use utils.failOnUnhandledRequest like the other suites. Signed-off-by: Jeffrey Sica <me@jeefy.dev>
…says it is off A plain 403 on the review (for example a workflow without pull-requests: write) now warns about the missing permission instead of advising to enable "Allow GitHub Actions to create and approve pull requests". Both stay warnings. Signed-off-by: Jeffrey Sica <me@jeefy.dev>
A draft gets its mirrored review once it is ready for review, so approve now re-evaluates on ready_for_review (before tide's merge evaluation in the same run). Dismissals still apply to drafts. Signed-off-by: Jeffrey Sica <me@jeefy.dev>
…proval rules Because approved is sticky, the bot re-approves every new head, so "Dismiss stale approvals" and "Require approval of the most recent reviewable push" are neutralized; the post-push protection is lgtm removal plus the required prow/lgtm status. Warn that "Allow GitHub Actions to create and approve pull requests" is repo-wide and recommend an App or machine-user token, note the Scorecard credit is nominal, and document refused dismissals and drafts. Signed-off-by: Jeffrey Sica <me@jeefy.dev>
…used dismissals Signed-off-by: Jeffrey Sica <me@jeefy.dev>
Signed-off-by: Jeffrey Sica <me@jeefy.dev>
|
@mfahlandt I pushed a few commits on top of your branch to cover the review. Your commits are unchanged and nothing was force-pushed.
|
jeefy
left a comment
There was a problem hiding this comment.
All review asks addressed. Thanks @mfahlandt!
|
/kind cleanup |
The OpenSSF Scorecard Branch-Protection
check scores in tiers and only credits a tier once the previous one is complete:
A Prow-gated repository (OWNERS +
/lgtm+/approve+ requiredprow/lgtmstatus) has norequired review count, so it stops in tier 2 with partial credit and tier 3 is never counted:
4/10 (observed on seebom-labs/BOMHort:
Warn: branch 'main' does not require approvers). With one required approving review tiers 2and 3 are complete: 8/10. Today such a repository only gets there by approving twice: a
/approvecomment and a GitHub review.Disclaimer: Parts of this PR and the PR template have been written with GitHub Copilot Model Claude Opus 5.5
This PR adds a opt-in setting of the approve plugin (repositories with OWNERS files only):
While a pull request carries
approved, the token keeps anAPPROVEreview on the PR's currenthead commit. When
approvedgoes away, it dismisses that review. Humans keep using/approveand/lgtmonly. A branch protection rule or ruleset requiring 1 approving review is then met by theProw decision.
approvedAPPROVEDreview by the token on the current head (commit_id); nothing is written when it already exists. Body:Approved via /approve by bob, carol (OWNERS)., one explanatory line, hidden marker<!-- prow-github-actions/approve-review -->. Approvers are named without@: a fresh review is submitted after every push, and a mention would notify the approvers each timeapproved(/approve cancel,CHANGES_REQUESTED, files no longer covered, ?)APPROVEDreview by the token that carries the marker is dismissed, on any commit, withapproved removed: <reason>, ex:approved removed: no approver covers sdk/x.go; withdrawn by bob (/approve cancel)synchronize(new head) whileapprovedstayscannot submit the approval review: #<n> was opened by the token's own identity (<login>), and GitHub does not let an author approve their own pull request (approve.github_review), no review. Checked up front for a user token; for an installation token GitHub's 422Can not approve your own pull requestis caughtGitHub Actions is not permitted to approve pull requests.(any status)cannot submit the approval review: enable "Allow GitHub Actions to create and approve pull requests" (Settings ? Actions ? General) or pass a token that can (approve.github_review): <GitHub's message>could not submit the approval review: <error>/could not dismiss the approval review <id>: <error>thrown like the plugin's other errors: collected by the handler runner, which still runs tide, thensetFailed(error annotation) at the endInvariants
approvalEventsskips any review carrying the marker(always, also with the setting off; nobody has such reviews unless the setting was on) and,
with the setting on, every review by the token's own user. The login comes from
GET /user(once per client).
GITHUB_TOKENand App installation tokens get a 403 there, which istreated as "installation token": their reviews are by a
Botuser, which never counted anyway.Consequence, documented: with a maintainer's PAT as
token, that maintainer's own reviews stopcounting; use a machine user.
GITHUB_TOKENthe review fires no workflow. With a user token it firespull_request_review;approveOnReviewreturns early when the payload's review carries themarker (also for
dismissed). Tide still evaluates on that event, which can only help (thereview may have unblocked
mergeable_state).handler registry and before the post-command sweep).
APPROVEDreview whosecommit_idis the head means no write.github_reviewunset there is noGET /user, no extralistReviews(withignore_review_state: truenone at all, as before), no review write.Not breaking: nothing changes unless the key is set. The legacy path for repositories without
OWNERS files (
src/issueComment/approve.ts) is unchanged.Required repository setting
With
GITHUB_TOKEN: Settings ? Actions ? General ? Workflow permissions ? Allow GitHub Actionsto create and approve pull requests (GitHub docs: this setting configures "whether
GITHUB_TOKENcan create and approve pull requests"; organizations may force it off). A GitHubApp token or a machine user's PAT does not need it. The workflow permissions are unchanged:
pull-requests: writeis already required (docs/installing.md lists it for reviews), soinstalling.mdis untouched.What is uncertain about GitHub's behaviour
Written down as uncertain in the docs as well:
GitHub Actions is not permitted to approve pull requests.is widely reported (GraphQL:
? (addPullRequestReview)); third-party code reports HTTP 422 forREST, GitHub does not document the status. The code matches the message regardless of status,
and treats any 403 the same way (per the issue). A 403 for another reason (ex: missing
pull-requests: write) therefore also becomes a warning; GitHub's message is appended so theoperator sees the real cause.
mergeable_stateright after the review. Tide runs after the review in the same run. Theexisting
unknownretries (1 s, 2 s, 4 s) cover anunknownanswer; whether GitHub can insteadreturn the previous
blockedfor a moment is not documented. If it does, the PR is skipped asnot mergeable (blocked)and the next event or the cron merges it. With a user token the review'sown
pull_request_reviewevent re-evaluates the merge.synchronizerun submits a new review on the new head. Whether GitHub's asynchronous dismissalcould land after, and dismiss, the fresh review is not documented. The next evaluation would
put it back.
every head, as long as the token's identity is not the pusher (GitHub docs: "approved by
someone other than the person who pushed it").
GITHUB_TOKENapproval counts toward branch protection's required countis what the old bot review relied on; that a ruleset's "Required approvals" counts it the same
way is expected but not verified here.
APPROVEreview on a draft PR. If it refuses,the evaluation fails with
could not submit the approval review: ?.token may dismiss its own review is not documented; a refusal fails the run with
could not dismiss the approval review <id>: ?.github-actions[bot].Docs
docs/configuration.md: the key in theapproveexample and table.docs/commands.md(approve): "No bot review by default" paragraph, new"Mirroring
approvedas a GitHub review" subsection with the situation table and every message verbatim.docs/automatic-merging.md: step 4 of "Upgrading to aggregated approval" gets the third option;new section "Required reviews and OpenSSF Scorecard" (what it is and is not, the repository
setting, dismiss-stale, most-recent-push, mergeability, the Scorecard tiers stated neutrally:
the check credits a required review count and does not judge who approves).
docs/installing.md: checked,pull-requests: writealready listed; unchanged.Tests
__tests__/plugins/approveReview.test.ts(vitest + msw), every table row and invariant:flag off ? no
GET /user, no review write, nothing dismissed; flag off +ignore_review_state? no
listReviews; approved ? one review onheadshaafter label and notifier; idempotentre-run; old-head review +
synchronize? new review on the new head, old not dismissed;stale-dismissed own review ? re-submitted; marked review by someone else is not ours; self PR
(user token) ? warning; 422 self-approval ? warning; 403 ? warning with the exact text;
"not permitted" 422 ? warning; 500 ? error, label/notifier still written, run fails at the end
while tide still evaluates; closed PR ? nothing; cancel ? every marked own review dismissed with
the reason; unmarked bot/human reviews and marked reviews by others are never dismissed;
CHANGES_REQUESTEDreason; dismissal 500 ? error; user token that is an OWNERS approver:its marked and unmarked reviews never count;
GET /user500 ? error before any write; 404 ?installation token, memoized;
pull_request_reviewfor the mirrored review (submitted anddismissed) evaluates nothing; order label ? notifier ? review ? merge through
handlePullReq.dist/index.js):/approve?approvedlabel + oneAPPROVEreview on the headbefore tide's read;
/approve cancel? label removed, only the marked review dismissed.approve.github_review must be a boolean).npm run build,npm run lint(0 errors, 2 pre-existing warnings),npm run pack,npx vitest run --coverage: 70 files, 1486 tests pass; coverage 99.83 % lines, 98.42 % branches,100 % functions (thresholds 95/93/98/95);
approveReview.ts100 %.dist/rebuilt and committed;a rebuild leaves
git statusclean.