Repository navigation
fix: an autopilot pull request whose checks fail is never fixed - #744
dilux-bot[bot] wants to merge 9 commits into
Conversation
The draft pull request opens before any code: the Spec as its checklist.
|
The pull request's board
🤖 AI-generated · Claude Opus 5.5 (Anthropic) |
An autopilot pull request whose conventions, actionlint or scripts job fails gets a fix round with the failed jobs' log in .dx-io/checks.md, read from the calling run's jobs by the new checks-failed command, instead of waiting for a person nobody told. Assisted-by: Claude Opus 5.5 (Anthropic)
The pipeline's fix round also starts when the review's job fails, and checks-failed counts that job as a check round when no verdict of the review is at the head (its did-not-finish record carries no sha, a job that died before its verdict left none), so the agent reads its log; a review that blocks with a verdict keeps the review's round. Assisted-by: Claude Opus 5.5 (Anthropic)
At autopilot-rounds a check round starts no more: May it start adds needs-human and comments from the new rounds_spent, which says the checks still fail rather than that the review blocks. next_step reads a checks_failed state the same way: one round below the cap, a person at it. Assisted-by: Claude Opus 5.5 (Anthropic)
An autopilot pull request whose failed checks include one the pipeline fixes waits on the autopilot's fix round while its highest autopilot:round-N is below policy/autonomy.yml's autopilot-rounds; at the cap, or with only a repository's own check failed, it waits on the maintainer with the action to take it over; an autopilot draft's failed check is its own run's work. No board tells the autopilot to fix the failed check any more. autopilot.py reads the same check names from waiting.py, and no fix round runs beside a draft's own run. Assisted-by: Claude Opus 5.5 (Anthropic)
The rounds and the queue tests of May it start stub checks-failed and rounds-spent, expect its checks output, and let autopilot-fix wait on the conventions beside the review, as C1 to C3 made them. Assisted-by: Claude Opus 5.5 (Anthropic)
org-pull-request.yml's autopilot-fix waits on the conventions and the review and starts on the same trigger as this repository's, its gate list the conventions alone: a failed conventions job or a review with no verdict gets its round, its cap and its board in every repository once v7 moves. docs/workflows.md and the pipelines' comments say so. Assisted-by: Claude Opus 5.5 (Anthropic)
Claude review · risk high · complexity high · type fix · blocking1 to fix (1 major) First review. The description says
Policy floor: high (touches high-risk paths: .github/workflows/autopilot.yml, .github/workflows/org-pull-request.yml, .github/workflows/pull-request.yml, scripts/autopilot.py, scripts/fixtures/waiting.json …). Reviewed 3234a4b (whole pull request; review 1 of 5 automatic). Author trusted for auto-merge: false. 🤖 AI review · claude-sonnet-5-5 (Anthropic) · $0.32, 9 turns |
| #723). Empty: a review round, as before, on the verdict at the head. | ||
| `record` is the review's last dx-review-record, `head` the pull | ||
| request's head. Pure.""" | ||
| verdict = isinstance(record, dict) and bool(head) and record.get("sha") == head |
There was a problem hiding this comment.
major: A capped review (stale blocking verdict) is misread as a review with no verdict
checks_failed treats a failed review / … job as "no verdict" unless the record's sha equals the head. In claude-review.yml the capped mode (max-auto-reviews reached) reuses the previous record, which keeps the older sha and is still blocking. It then does exit 1, so the review job fails with a valid blocking verdict that is simply not at this head. The record check fails, so the job counts as a check round. The agent then reads only the review job's log ("The last review found a blocking problem; see the summary comment") instead of the review's threads in review.md. Before this PR the same state ran the review round with the threads. At the cap the comment also says "its checks still fail" instead of "the review still blocks". It is reachable wherever max-auto-reviews is below autopilot-rounds, and after round 5 with the defaults (6th push is capped). Fix: count a review job as "no verdict" only when the record is missing or has no verdict fields. The "did not finish" record written by the workflow carries only count and findings, with no risk, model_risk or blocking, while a reused or capped record has them. Add a --test fixture for a record with an older sha and blocking: true (expected: empty, review round) next to the existing "did not finish" fixture.
🤖 AI review · claude-sonnet-5-5 (Anthropic)
|
The autopilot could not read which checks failed (), so no fix round starts: a person takes it from here. 🤖 AI-generated · Claude Opus 5.5 (Anthropic) |
|
No check failed here: what blocks is the review (#744 (comment) from 03:48, one major). Please fix the major at scripts/autopilot.py:2007 (a capped review with a stale blocking verdict is misread), make the production state actually carry 🤖 AI-assisted · Claude Opus 5.5 (Anthropic) |
|
@soydiloreto: a person takes it from here ( 🤖 AI-generated · Claude Opus 5.5 (Anthropic) |
|
Answering the stop: what blocks is the review's one major, not a failed check. Please fix the major at scripts/autopilot.py:2007 (a capped review with a stale blocking verdict is misread as a fresh one), make the production state carry 🤖 AI-assisted · Claude Opus 5.5 (Anthropic) |
|
Closing: this pull request's own new fix trigger stopped it ("could not read which checks failed ()"), so no round can bring it back. #741 starts again with the review's findings written on the issue. 🤖 AI-assisted · Claude Opus 5.5 (Anthropic) |
📝 What changes
The PRD passes G1: the problem is clear and C1 to C5 can each be tested from the pipelines' YAML and the scripts' pure functions.
Trigger (C1, C2, C5). Each pipeline's
autopilot-fixgetsneeds:on every job the review waits on (pull-request.yml:[conventions, actionlint, scripts, review]; org-pull-request.yml:[conventions, review]). Itsif:keeps every guard it has now:always(),pull_request, notedited, theautopilotlabel, dilux-bot as author, and the repository clause. Only the trigger part becomesneeds.review.outputs.blocking == 'true' || needs.review.result == 'failure' || needs.<gate>.result == 'failure'(one clause for each gate). That covers a gate that failed (the review is then skipped and has no outputs) and a review that ended without a verdict, whether its verdict step said so (blocking=true, "did not finish") or the job died before that step (fix: the bug reproduced from #720 #723: no outputs). The two conditions stay the same apart from the repository and the gate list, and the existing fix: the autopilot answers a review in the organisation's repository #504 test checks this.Fix mode reads the failed jobs' log (C1, C2). No new input: org-pull-request.yml calls
autopilot.yml@v7, and the queue round (ci: test before the pull request, a faster queue, an unstuck autopilot #479) avoided a new input for the same reason. In fix mode,autopilot.pygets achecks-failedcommand andinputs fixgets a matching branch. They read the jobs of the calling run (GITHUB_RUN_ID; a reusable workflow's jobs belong to it). Two cases make a check round: a failed job other than the review's, or a failed review job with no verdict at the head (the bot'sdx-reviewsummary says "did not finish", or its record is not at the head SHA). The failed jobs'gh run view --job <id> --log-failedgoes into.dx-io/checks.md, scrubbed and cut likequeue.md:queue_log_textis generalised and keeps the queue's text.prompt_fix(..., checks=True)points the agent at that file instead ofreview.md, and the round writes no review ids. A review that blocks with a verdict keeps today's review round. A round that fails as the round before did still stops on the existingdx-prepr-sig.Cap (C3). The rounds check in "May it start" stays where it is (
ROUND > autopilot-rounds→needs-human, a comment, and the issue opened to claims). Its comment now comes from a purerounds_spent(max, checks)in autopilot.py, so it says "its checks still fail" on a check round instead of always "the review still blocks".next_stepgets achecks_failedstate, so the sweep and the board's next step agree with the trigger: one round below the cap, the person at the cap.Board (C4).
waiting.py blocker: an autopilot pull request whose failed checks include at least one the pipeline fixes (check-run namesconventions /,review /,actionlint,Scripts (self-tests)) waits on "the autopilot's fix round", with rounds left. Rounds left means the highestautopilot:round-Nlabel is belowautopilot-rounds, read from policy/autonomy.yml. At the cap, or when only other checks failed (a repository's own workflow, which nothing re-runs), it waits on the maintainer, with the action to take it over (needs-humanalready comes first). The board then never tells the autopilot to "fix the failed check".Options a maintainer may care about:
workflow_run/check_suitetrigger outside the two pipelines, which is beyond the PRD's scope, so it is listed as a finding instead.What this widens: a fix round (the agent's job, with the Claude subscription and a read-only token) now also starts on a failed gate or a review without a verdict. It stays within the same
autopilot-roundsand runs caps. No new permission or secret is needed: reading the run's jobs needsactions: read, which autopilot.yml'sprepjob already holds or will be given, and the pull request says which. In repositories other than this one it takes effect oncev7is moved by a release.💡 Why
Closes #741. The autopilot works it here in the open: one commit per criterion, each box ticked once its commit is pushed.
🎯 Acceptance criteria
Test: scripts/autopilot.py (
--test: pull-request.yml's autopilot-fix needs the gates and fires onneeds.scripts.result == 'failure';checks-failed/checks_log_textselect the failed jobs and their log;prompt_fix(checks=True)reads.dx-io/checks.md)Test: scripts/autopilot.py (
--test: the condition fires onneeds.review.result == 'failure'and on a blocking review; a review job that failed without a verdict at the head is a check round, one with a verdict a review round)autopilot-rounds, when a check fails again, then no round starts and it is handed to a person withneeds-human.Test: scripts/autopilot.py (
--test:next_stepwithchecks_failedpastautopilot-roundswaits on a person;rounds_spenttext; the fix branch of "May it start" addsneeds-humanabove the cap)Test: scripts/waiting.py (
--testwith scripts/fixtures/waiting.json: an autopilot pull request whose scripts or review check failed with rounds left waits on the autopilot's fix round; at the cap, or with only a repository's own check failed, it waits on the maintainer; never "fix the failed check" for the autopilot)Test: scripts/autopilot.py (
--test: org-pull-request.yml's autopilot-fix carries the same trigger as pull-request.yml's apart from the repository and its gate list, which is conventions and review)🧭 Decisions
🙌 Credits
🧪 How I tested it
python3 scripts/autopilot.py --testandpython3 scripts/waiting.py --testpass, with the new checks for C1 to C5: the two pipelines'autopilot-fixtrigger and needs read from their YAML,checks_failedon job lists (a failed gate, a review that failed with or without a verdict at the head, other workflows', cancelled and running jobs), the log text cut and scrubbed, the check round's prompt,next_stepandrounds_spentat and below the cap, "May it start"'s fix branch run in bash with gh and the check answer stubbed (below the cap, at it, unread jobs, a draft), and the board on the new fixtures 15 to 19 in scripts/fixtures/waiting.json. The tests under tests/ pass with pytest (268 passed); test_autopilot_rounds.py and test_queue_drop_fix.py now expect the gate'schecks=output. I ran every step of pull-request.yml's scripts job, and actionlint 1.7.12 is clean. review-brief.sh and dx-runner-slot.sh failed only with this run's PR and DX_BASE variables set, and pass without them. The repository has nomake pre-pr. Nothing ran against GitHub: reading a real run's jobs and logs (checks-failed,job_log) is tested only through stubs.🤖 AI-generated · Claude Opus 5.5 (Anthropic)