fix: the release lane tests the pull-request path at v7, not v7-next #748

What happened?

The release lane cannot prove a fix to the pull-request path, so v7.14.0 can never pass. The lane points v7-next at the candidate and moves the test repository's callers (issues.yml, pull-request-comments.yml) to @v7-next, but the pull-request pipeline of every repository, the test repository included, is the organisation's required workflow: ruleset 24694715 ("org: main") runs .github/workflows/org-pull-request.yml at refs/tags/v7, which is v7.13.5 (902e3d9). So in DiluxOne/diluxone-test the review, the auto-merge decision and the autopilot's fix rounds of every scenario pull request run the released engine, not the candidate. The review of scenario pull request DiluxOne/diluxone-test#759 checked the engine out at 902e3d9 and decided trusted: false for its author, although #737 and #743 (merged, in the candidate) make the test identity trusted there. Every fix to that path (trusted authors, auto-merge, review, fix rounds) fails the lane until v7 moves, and v7 moves only when the lane passes: #715, #724, #725, #726, #740, #742, #745 and #747 are this loop.

Steps to reproduce

1. gh api orgs/DiluxOne/rulesets/24694715 --jq '.rules[]|select(.type=="workflows")|.parameters.workflows' gives org-pull-request.yml at refs/tags/v7.
2. Run the release lane on a candidate that changes scripts/policy.py (as fix: the test identity is the pack's login, not its token's owner #743 did).
3. In the test repository, the "Claude review" job of any scenario pull request checks out .dx-central at v7's commit, not v7-next's.

Version

v7.13.5 (902e3d9) as v7; candidate v7-next after #743.

Logs or screenshots

https://github.com/DiluxOne/.github/actions/runs/43554598611 (the lane), https://github.com/DiluxOne/diluxone-test/actions/runs/43554709817 (scenario pull request #759's pipeline, at 902e3d9, trusted: false), #747 (the lane's report).

Proposed solution

The test repository runs the candidate's pull-request pipeline while the lane plays. The organisation's test repository (org.yml test-repo) is left out of the required-workflow ruleset's repositories, and it gets a pull-request caller of its own (written by scripts/sync-repos.py callers for kind engine-test, from workflow-templates/), which the lane moves to @v7-next and back together with the other callers. Its required checks stay the same, so a pull request there is checked as strictly as before. The ruleset change is written by scripts/sync-repos.py from data (org.yml test-repo), never by hand, and a PUT sends the whole conditions and is verified afterwards (the 2026-10-15 incident). scripts/release-scenarios.py coverage/plan refuses a plan in which any pipeline of the test repository is not at the candidate.

Problem

A release that fixes how pull requests are reviewed or merged cannot pass the release lane, so the engine cannot ship those fixes at all and every repository stays on the old version.

Scope

The required-workflow ruleset's repository conditions (via scripts/sync-repos.py, from org.yml test-repo), the test repository's pull-request caller and its template, release-lane.yml's move of the callers to v7-next and back, the check in release-scenarios.py, their self-tests, and docs/versioning.md and docs/workflows.md.

Out of scope

The scenarios' own expectations. Any other repository's pipeline, which keeps the required workflow at v7.

Acceptance criteria

C1: Given the release lane playing a candidate, when a scenario pull request in the test repository runs its pipeline, then its review, auto-merge decision and fix rounds check out the candidate's engine (v7-next), not v7.
C2: Given the organisation's rulesets after the sync, when they are read, then the required-workflow ruleset covers every adopted repository except org.yml test-repo, its conditions are complete (ref and repository conditions), and every other repository keeps the required workflow at v7.
C3: Given the test repository, when its callers are synced, then it has a pull-request caller with the same required checks the required workflow gives, and the lane moves it to @v7-next and back with the others.
C4: Given a release plan in which a pipeline of the test repository would not run at the candidate, when release-scenarios.py checks the plan, then it refuses it and says which one.
C5: Given docs/versioning.md and docs/workflows.md, when a reader looks for how the lane tests the pull-request path, then they say the test repository runs the candidate's pipeline and why.

Criticality

Critical

Who works on it

• I want to work on this myself

🤖 AI-generated · Claude Opus 5.5 (Anthropic)

Spec: approach

Today the required workflow is the workflows rule of the organisation ruleset "org: main". That rule runs org-pull-request.yml at v7 in every adopted repository, the test repository too, and its inner calls are hard-wired to @v7. So in the test repository a scenario pull request's review, auto-merge decision and fix rounds always run v7, whatever the lane moves. The fix has three parts. First, the test repository stops getting the required workflow. Second, it runs the same pipeline from a caller of its own, which the lane's point_callers already moves to @v7-next and back like any other caller there. Third, the lane refuses to play when a pipeline there would still run v7.

1. Ruleset (C2), scripts/sync-repos.py rulesets. GitHub takes only one repository condition per org ruleset (name, id or property), so the test repository cannot be dropped by name from a ruleset that targets by property. Recommended: option A.

   • Option A (recommended). Move the workflows rule out of "org: main" into a new ruleset, "org: required workflow". It is created first and copies "org: main"'s workflows rule unchanged (the same org-pull-request.yml, at the same v7 ref), its bypass actors, its target and its ref_name (~DEFAULT_BRANCH). Only then is the rule removed from "org: main", so no repository is ever left without the required workflow. Its repository_property condition includes dx-adopted=true and excludes a second custom property of the organisation: test-property in org.yml, dx-engine-test by default, a true_false property the sync creates and sets to true on org.yml test-repo alone. "org: main" keeps its other rules on the test repository: pull request, deletion and signed commits. The PUT body always carries both ref_name and repository_property ("complete conditions": with_property_condition and the new builder are tested for it).
     • The exclusion is written only once the test repository has its pull-request caller (item 2) on its default branch. Until then the sync says so and leaves the test repository covered, because otherwise its pull requests would run no pipeline. For that short time both pipelines run there at v7.
     • ORG_RULESET_NAMES, by_property and covers learn the new ruleset and its exclusion.
     • sync-repos.py check (the daily drift check) flags four things: a workflows rule still in "org: main", the required-workflow ruleset covering the test repository, a ruleset with no ref_name, and a required-workflow ref other than v7.
     • This widens the organisation's settings by one custom property and one ruleset. The pull request will say so.
   • Option B. Same split, but the new ruleset targets by repository_name: it includes every repository repos.yml adopts and excludes test-repo. It needs no new property, but it brings back the list of names that feat: adopting a repository is data and one command #562 removed, and every adoption has to extend it.
   • Option C. No split: exclude the test repository from "org: main" as a whole, and give it repository rulesets that copy "org: main"'s other rules. This means more rulesets to keep in step, and superseded() would have to stop deleting them.

2. Caller (C3). A new template, workflow-templates/test-pull-request.yml (plus its .properties.json). Its on: and its jobs are org-pull-request.yml's, job for job: the same ids and uses:, so the same check names. It adds one job, pipeline, which needs every other job, runs if: always() on both pull_request and merge_group, and fails when any of them failed or was cancelled.

   • Why pipeline: a required workflow has no named checks; it requires the whole run to succeed. pipeline is that run's result as one check, and it reports in the merge queue too. Requiring each job's check instead would block the queue for good, because the jobs skip on merge_group under another name.
   • The pack (kinds/engine-test/pack.yml callers) lists the new template. new_callers also writes a kind's pack callers, so sync-repos.py callers writes it to the test repository and holds it job for job.
   • The sync writes, in the test repository only, a repository ruleset "required pipeline" that requires the pipeline check on the default branch. That gives the test repository the same gate the required workflow gives everywhere else.
   • The lane's move needs no new code: point_callers rewrites every engine call under .github/workflows/, so the new caller moves to @v7-next and back with the others (shown by a test).
   • The template sits in workflow-templates/ with the other shipped callers, so the organisation's template picker also offers it to other repositories. Its description says it belongs to the test repository alone.

3. The check (C4), scripts/release-scenarios.py. test_callers() reads the pack's callers (now including test-pull-request.yml) in place of org-pull-request.yml. The pairs are unchanged, so the scenarios' expectations are untouched. Two new checks, each naming the file or rule it refuses:

   • problems (static, coverage) refuses three things: a pack with no caller that wires the pipeline's pull_request and merge_group jobs, org-pull-request.yml among the test repository's callers, and an engine call in a test caller that at_ref would not move (one that does not match USES).
   • move (run time, before any group plays) refuses two things, each as a failing callers row: the test repository's default-branch rules (repos/{repo}/rules/branches/{default}, readable with its token) still carrying a workflows rule for org-pull-request.yml, and any .github/workflows/*.yml there still calling the engine off the ref after the move.

4. Docs (C5). docs/versioning.md (Scenarios step) and docs/workflows.md (the org-pull-request.yml and release-lane rows) say that the test repository runs the candidate's pipeline from test-pull-request.yml at @v7-next, and why: the required workflow is read at v7 with its calls hard-wired. To stay in step, the same goes into docs/adopting.md (the ruleset list), kinds/README.md (pack callers are now written), the pack's comments, and the header comments of org-pull-request.yml and release-lane.yml.

Spec: files

• scripts/sync-repos.py (rulesets split and exclusion, the test property, new_callers with pack callers, the "required pipeline" ruleset, drift check, --test)
• scripts/org.py (test_property from org.yml, its --test)
• org.yml (test-property)
• workflow-templates/test-pull-request.yml, workflow-templates/test-pull-request.properties.json
• kinds/engine-test/pack.yml (callers, comments)
• scripts/release-scenarios.py (test_callers, the static and run-time checks, --test)
• scripts/caller-contract.py (only if the new template needs it recognised)
• .github/workflows/release-lane.yml (header comment), .github/workflows/org-pull-request.yml (header comment)
• tests/workflows/test_test_repo_pipeline.py (new)
• docs/versioning.md, docs/workflows.md, docs/adopting.md, kinds/README.md

Spec: tests

• C1 → scripts/release-scenarios.py (--test: after point_callers, every engine call of test-pull-request.yml's review, auto-merge and autopilot-fix jobs is at @v7-next), tests/workflows/test_test_repo_pipeline.py
• C2 → scripts/sync-repos.py (--test: the split ruleset's PUT/POST bodies carry ref_name and repository_property with the test property excluded, the same v7 workflows rule, "org: main" kept for the test repository, created before the rule is removed, the drift check's four findings)
• C3 → tests/workflows/test_test_repo_pipeline.py (the template's jobs and on: equal org-pull-request.yml's plus pipeline), scripts/sync-repos.py (--test: new_callers writes it to the engine-test repository only, and "required pipeline" requires pipeline), scripts/release-scenarios.py (--test: moved to @v7-next and put back)
• C4 → scripts/release-scenarios.py (--test: problems names a missing pull-request caller, org-pull-request.yml among the callers, an unmovable call; move names a remaining workflows rule and a caller off the ref)
• C5 → tests/workflows/test_test_repo_pipeline.py (docs/versioning.md and docs/workflows.md name test-pull-request.yml at @v7-next and say that the required workflow runs at v7)