[dnm]: test cancellation - #524
Draft
jamieQ wants to merge 3 commits into
Draft
Conversation
Abort in-flight analysis, preserve partial findings and usage, and finalize findings artifacts with a cancelled outcome. Keep analyze/report publication ownership intact and conclude active checks consistently. Co-Authored-By: GPT-5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
![sentry-warden[bot]](https://avatars.githubusercontent.com/in/2755838?s=60&v=4){kind=small}
Comment on lines
+47
to
+66
| if: ${{ cancelled() }} | ||
| env: | ||
| FINDINGS_FILE: ${{ steps.warden-analyze.outputs.findings-file }} | ||
| run: | | ||
| if [ -z "$FINDINGS_FILE" ]; then | ||
| echo "Analyze was cancelled without producing a findings file" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| FINDINGS_FILE="$FINDINGS_FILE" node --input-type=module -e ' | ||
| const fs = await import("node:fs"); | ||
| const path = process.env.FINDINGS_FILE; | ||
| const findings = JSON.parse(fs.readFileSync(path, "utf8")); | ||
| if (findings.outcome !== "cancelled") { | ||
| throw new Error(`Expected cancelled outcome, received ${findings.outcome}`); | ||
| } | ||
| if (!fs.existsSync(`${path}.done`)) { | ||
| throw new Error(`Missing finalization marker: ${path}.done`); | ||
| } | ||
| console.log(`Preserved ${findings.summary.totalFindings} finding(s)`); |
Contributor
There was a problem hiding this comment.
Cancellation after Analyze completes makes Verify reject a successful artifact
When concurrency cancellation arrives after Analyze has written its normal successful artifact but before Report runs, the job-level cancelled() guard executes Verify against that artifact. Because successful analyze output has no outcome: "cancelled", Verify fails, and the following broad cancellation guard can archive the valid artifact as cancelled-findings; gate these steps on an Analyze cancellation that corresponds to a cancelled artifact rather than job cancellation alone.
Evidence
- The PR concurrency group uses
cancel-in-progress: true, so a newersynchronizerun can cancel the prior job between the sequential Analyze and Report steps. - Normal
runAnalyzeModecompletion callswriteFindingsOutputwithout anoutcome, whilefinalizeCancelledPRRunis the path that writesoutcome: 'cancelled'. - Verify runs before Report under only
cancelled(), reads the Analyze output, and throws when the successful artifact's outcome is missing. - Upload also uses only
cancelled()plus the non-empty findings-file output, so the cancellation state remains active after Verify fails and the successful artifact can be uploaded under a cancelled name.
Identified by Warden · code-review · WMS-WUJ
Comment on lines
+2547
to
+2592
| if (cancellation.requested && inputs.mode !== 'report') { | ||
| await finalizeCancelledPRRun({ | ||
| ...buildCancelledPRFinalizationBase(inputs, initResult), | ||
| results: [], | ||
| publish: inputs.mode === 'run', | ||
| failOnWriteError: inputs.mode === 'analyze', | ||
| }); | ||
| span.setAttribute('warden.finding.count', 0); | ||
| return; | ||
| } | ||
|
|
||
| if (inputs.mode === 'analyze') { | ||
| return runAnalyzeMode(inputs, initResult, span); | ||
| return runAnalyzeMode(inputs, initResult, span, cancellation); | ||
| } | ||
|
|
||
| if (inputs.mode === 'report') { | ||
| return runReportMode(octokit, inputs, initResult, repoPath, span); | ||
| return runReportMode(octokit, inputs, initResult, repoPath, span, cancellation); | ||
| } | ||
|
|
||
| const { coreCheckId, previousReviewInfo } = await Sentry.startSpan( | ||
| { op: 'workflow.setup', name: 'setup github state' }, | ||
| () => setupGitHubState(octokit, context, postChecks), | ||
| ); | ||
|
|
||
| const finalizeCancelledRun = async ( | ||
| results: TriggerResult[] = [], | ||
| findingObservations: FindingObservation[] = [], | ||
| ): Promise<boolean> => { | ||
| if (!cancellation.requested) { | ||
| return false; | ||
| } | ||
| await cancelCoreCheck(octokit, context, coreCheckId, results, postChecks); | ||
| const outputs = await finalizeCancelledPRRun({ | ||
| ...buildCancelledPRFinalizationBase(inputs, initResult), | ||
| results, | ||
| findingObservations, | ||
| publish: true, | ||
| }); | ||
| span.setAttribute('warden.finding.count', outputs.findingsCount); | ||
| logAction(`Analysis cancelled: preserved ${formatCancelledPreservation(outputs.findingsCount, results)}`); | ||
| return true; | ||
| }; | ||
|
|
||
| if (await finalizeCancelledRun()) { | ||
| return; | ||
| } |
Contributor
There was a problem hiding this comment.
Pre-execution cancel finalizes with empty results, dropping matched triggers
When cancellation is requested before trigger execution, finalizeCancelledRun/finalizeCancelledPRRun are called with results: [] even if matchedTriggers is non-empty, so cancelled findings omit pending triggerResults/skippedTriggers for those matched skills.
Evidence
- The early cancel path at lines 2547–2555 and the first finalizeCancelledRun() call at 2590–2592 both finalize with results defaulting to [] before executeAllTriggers runs.
- executeAllTriggers only synthesizes pending results for undispatched matched triggers when cancellation is requested (lines 699–706); that path is never reached here.
- finalizeCancelledPRRun builds triggerResults/skippedTriggers only from options.results via toReplayTriggerResults/toUnfinishedSkippedTriggers, so matched-but-never-dispatched triggers disappear from the cancelled artifact.
- The same contract is enforced after dispatch (pending: true) and covered by the mid-execution cancel test expecting pending triggerResults/skippedTriggers.
Identified by Warden · code-review · V7P-XNY
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
testing