feat(config): Add postChecks toggle to disable GitHub Check-run posting - #458
Conversation
ae13665 to
96c3a17
Compare
|
@dcramer I've updated this as it had conflicts with main after the output file updates merged. If it's helpful context, we added this control in our fork because we were getting complaints about "double posting" from our engineers when they would see both a comment and an annotation check. We aren't using the checks in our CI as our current implementation is non-blocking by design, so this control was added to prevent posting the checks we aren't using. |
Check-run creation is entirely write-only (nothing in Warden reads it back, and it doesn't gate dedup or the findings-file build), so it's safe to let it be switched off independently of PR review comments. Defaults to true to preserve existing behavior; a defaults.postChecks in warden.toml can override the post-checks action input. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
initializeWorkflow already treats runnerConcurrency and auxiliaryOptions as enforced by the org base config, with repo config only filling in what the base omits (documented inline right above where postChecks was resolved). postChecks is the same class of workflow-level setting, but was reading the already-merged (repo-wins) config instead, so a repo's own warden.toml could silently defeat an org policy meant to turn checks off fleet-wide. Proved by reverting the fix and watching the new org-base-config test fail exactly as expected. Also drops the dead `options.postChecks ?? true` fallback in finalizeReportWorkflow — its only caller always supplies postChecks, so the fallback could never actually run. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
fight-me review of this PR caught that post-checks had no docs entry in either the workflow guide or llms.txt, and more importantly that the guide's own Required Status Checks section recommends requiring the warden check — setting post-checks: false on such a repo would silently stop that check from ever running again, permanently blocking merges through the normal GitHub UI. Documents the input in both places and adds an explicit warning against that combination. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
96c3a17 to
99a2df5
Compare
|
@CalebKAston is attempting to deploy a commit to the Sentry Team on Vercel. A member of the Team first needs to authorize it. |
|
sweet this is good, thank you |
|
@CalebKAston you can ask your clanker to clean up the typecheck, it's failing on current main, I can merge afterwards |
|
btw. i can fix some of those myself in the future, i just think you'd have to submit PRs from personal fork vs. org fork because org security overrides the "maintainerCanModify" flag, but up to you, that just gives you a little bit longer back and forth with us |
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Good to know! Thanks for the explanation. With these last couple of PRs we should fully be at feature parity with this maintained version of Warden, so the goal is to deprecate our local fork. But I'll keep that in mind if I need to submit additional PRs. |
|
@gricha , I've fixed the typecheck 👍 |
…checks-toggle-upstream # Conflicts: # packages/warden/src/action/workflow/pr-workflow.ts
|
thank you! |
Summary
Adds a
postChecksboolean (action inputpost-checks, defaulttrue; overridable per-repo viawarden.toml's[defaults] postChecks) that disables creation/update of GitHub Check runs — the corewardencheck and every per-skillwarden: <skill>check, including their inline annotations — while leaving PR review comment posting untouched.Why
Check-run creation is entirely write-only today: nothing in the codebase reads it back via the API, and it isn't load-bearing for the findings-output file or for comment dedup (which reads back PR review comments instead, a separate mechanism). Some teams want PR review comments as the single review surface without a second, non-threaded, non-dedup'd GitHub Checks UI surface running alongside it — this makes that toggleable without losing the option to fail a required status check via
fail-check/fail-onwhen wanted.Implementation
checkOptionsForPullRequest(context, postChecks)returnsundefinedwhenpostChecksis false. Two call sites that previously bypassed it with inline option objects (setupGitHubState,finalizeWorkflow's core-check update) were refactored to route through it too, so every check-creation path inpr-workflow.tsshares one gate.postChecksresolves with the same enforced-baseline precedence already used forrunnerConcurrency/auxiliary-model settings ininitializeWorkflow: org base config (viabase-config-path) wins over the repo's ownwarden.toml, which wins over thepost-checksaction input default. This matters because it's the same class of workflow-level (not per-trigger) setting, and a repo-level override should not be able to silently defeat an org-wide policy set via the base config.ActionInputs.postChecksis non-optional (unlike the default-falsebooleansrequestChanges/failCheck) because its default istrueand gets resolved at parse time rather than deferred to each call site — the opposite polarity needs the default applied once, explicitly.Verification
Applied this change to a local fork (
babylist/warden) and verified it against real PRs in a downstream consumer repo before opening this PR:post-checksunset (defaulttrue): unchanged existing behavior — checks and comments both post, as today.post-checks: false: confirmed via the GitHub API that nowarden/warden: <skill>check runs were created on the PR, while a PR review comment still posted for a real finding.post-checks: trueset explicitly at the repo level (overriding an org base-config default offalse): confirmed the full check-run set (core + every configured skill) was created again, alongside the review comment.Also added regression tests exercising
postChecks: falseend-to-end throughrunPRWorkflow(both legacy run mode and report mode) assertingchecks.create/checks.updateare never called whilepulls.createReviewstill is, plus config/schema tests for thewarden.tomloverride and its precedence against the org base config.pnpm lint && pnpm build && pnpm testall pass (two unrelated, pre-existing flaky tests incli/files.test.ts/cli/output/jsonl.test.tsfail only under full-suite parallel load and pass in isolation — not touched by this change).