diff --git a/.github/workflows/org-pull-request.yml b/.github/workflows/org-pull-request.yml index 8c73b378..9919e5b5 100644 --- a/.github/workflows/org-pull-request.yml +++ b/.github/workflows/org-pull-request.yml @@ -1,8 +1,14 @@ name: Pull request (organisation) -# The pull request pipeline of every DiluxOne repository, in one place. An -# organisation ruleset requires this workflow on every repository's default -# branch, so it runs on each pull request with no file in the repository: +# The pull request pipeline of every DiluxOne repository, in one place. The +# organisation ruleset "org: required workflow" requires this workflow, at +# `v7`, on every adopted repository's default branch, so it runs on each pull +# request with no file in the repository, in every one but the test +# repository (org.yml `test-repo`, #748): there the same jobs run from its own +# caller, workflow-templates/test-pull-request.yml, which the release lane +# moves to `@v7-next`, since this workflow is read at `v7` with its calls +# hard-wired to `@v7` and so could never try a candidate. Keep its `on:` and +# jobs and that template's in step (tests/workflows/test_test_repo_pipeline.py): # the conventions with the accepted-issue gate, then the Claude review and # the auto-merge decision; beside them, the issues it closes go to its # author when a member of the organisation (pull-request-assignee.yml, #73); diff --git a/.github/workflows/release-lane.yml b/.github/workflows/release-lane.yml index feba707b..f5d3a800 100644 --- a/.github/workflows/release-lane.yml +++ b/.github/workflows/release-lane.yml @@ -10,7 +10,13 @@ name: Release lane # through pull requests the bot opens there and queues itself once their # checks pass, after checking that they change nothing but the engine's ref: # the auto-merge decision never merges a change to .github/workflows; never -# a write to its main) with the test identity's token (whose owner must be +# a write to its main; its pull-request pipeline among them, +# test-pull-request.yml, since the required workflow, org-pull-request.yml, +# runs at `v7` with its calls hard-wired and leaves the test repository out, +# so the scenarios' reviews, auto-merge decisions and fix rounds run the +# candidate; `move` plays nothing while the test repository still requires +# org-pull-request.yml or a caller there calls the engine off `@v7-next`, +# #748) with the test identity's token (whose owner must be # the organisation variable DX_TEST_LOGIN, or the lane stops before the # callers move, naming the value to set and its URL, #739), in groups that play # at once, one job each (`move`, then `play`, a matrix on ubuntu-latest, diff --git a/docs/adopting.md b/docs/adopting.md index 2f6064ca..9df3a193 100644 --- a/docs/adopting.md +++ b/docs/adopting.md @@ -101,7 +101,7 @@ Most of it is the organisation's already: a new repository gets the pull request its `repos.yml` entry's `pre-pr` says it has none by design). 3. **What it is, then labels, settings and shared files.** First add it to `repos.yml` `repos:`, one entry with everything said of it (#562): its kinds, the paths each covers and their roles, its `project`, `design-system`, `repro`, `merge-queue`, `pre-pr`, `dependabot` and `callers` (`scripts/repos-config.py check` holds it; the daily drift check reports a repository missing there); the kind is read from there only. Then `python3 scripts/sync-repos.py all --repo ` from a checkout of this repository: the labels, the merge and security settings (Dependabot's security updates among them), `.github/CODEOWNERS`, and the opening block of `AGENTS.md` (a pull request with its issue); then `python3 scripts/sync-repos.py dependabot --repo ` for `.github/dependabot.yml` (a pull request of its own). `all` also turns its merge queue on where GitHub gives one (org.yml `merge-queue`, #451): a public repository, once every check its rules require comes from a workflow that runs on `merge_group` (#562; the drift check names a check and its workflow that do not; a required check no run of its five most recent pull requests produces keeps a new queue off but leaves an existing one as it is, since it could not tell). `python3 scripts/sync-repos.py adopt --repo ` does all of it at once. The `main` and tag rulesets are the organisation's; a plugin that publishes adds its environment `wordpress-org` (deployment policy: tag `*.*.*` plus `main` and the branches `security/*`, where dilux-bot alone pushes a security patch, #247; the SVN credentials and the release App's key as environment secrets; no required reviewer is needed, since a person accepting the release issue is the approval) and installs the `dilux-release` App. - The organisation's side: the three organisation rulesets (`org: main`, `org: release tag creation`, `org: release tags`) cover the repositories whose custom property `dx-adopted` is `true`, which `sync-repos.py rulesets` (and `adopt`) writes from `repos.yml` (#562); it moves a ruleset from its list of names to the property only when every repository that list covers today (`~ALL` and patterns included, archived ones not) is listed in `repos.yml` and not excluded, and otherwise names those and changes nothing, and secret scanning and push protection come from the code security configuration `repos.yml` `security: configuration` names for the repository's visibility (one name for every repository, or `public:` and `private:`, the latter covering internal ones, #603), which `sync-repos.py security` attaches and keeps the default for new repositories of that visibility, narrowing one already attached that is still the default for every visibility (the daily drift check names a default that differs); a private repository with no `private:` entry gets none, and one already attached stays until a maintainer detaches it in the organisation's settings; what GitHub holds only at the enterprise level the maintainer changes once there. Done by an administrator of the organisation (a maintainer, or the session working in this repository): for a private repository, also to the organisation variable `DX_RUNNER_HEAVY` and to the runner group `private-repos-vm`, so its jobs run on the organisation's VM and spend no minutes; and give it a Project in its `repos.yml` entry (`project:`; `sync-repos.py project` links it, and the drift check says when it has none) and, when it has screens, its design system (`design-system:`, [`design-systems/`](../design-systems/)). A repository that versions third-party code with Markdown of its own (a site's `wp-content/plugins/` and `themes/`) lists those paths in a `.lycheeignore` at its root, lychee's own file, so the docs check does not follow their links. + The organisation's side: the four organisation rulesets (`org: main`, `org: required workflow`, `org: release tag creation`, `org: release tags`) cover the repositories whose custom property `dx-adopted` is `true`, `org: required workflow` (the required workflow, `org-pull-request.yml` at `v7`, split out of `org: main` by `sync-repos.py rulesets`, #748) less the test repository (org.yml `test-repo`, whose custom property org.yml `test-property` is `true`), which runs the same pipeline from its own caller, `test-pull-request.yml`, so the release lane moves it to the candidate, which `sync-repos.py rulesets` (and `adopt`) writes from `repos.yml` (#562); it moves a ruleset from its list of names to the property only when every repository that list covers today (`~ALL` and patterns included, archived ones not) is listed in `repos.yml` and not excluded, and otherwise names those and changes nothing, and secret scanning and push protection come from the code security configuration `repos.yml` `security: configuration` names for the repository's visibility (one name for every repository, or `public:` and `private:`, the latter covering internal ones, #603), which `sync-repos.py security` attaches and keeps the default for new repositories of that visibility, narrowing one already attached that is still the default for every visibility (the daily drift check names a default that differs); a private repository with no `private:` entry gets none, and one already attached stays until a maintainer detaches it in the organisation's settings; what GitHub holds only at the enterprise level the maintainer changes once there. Done by an administrator of the organisation (a maintainer, or the session working in this repository): for a private repository, also to the organisation variable `DX_RUNNER_HEAVY` and to the runner group `private-repos-vm`, so its jobs run on the organisation's VM and spend no minutes; and give it a Project in its `repos.yml` entry (`project:`; `sync-repos.py project` links it, and the drift check says when it has none) and, when it has screens, its design system (`design-system:`, [`design-systems/`](../design-systems/)). A repository that versions third-party code with Markdown of its own (a site's `wp-content/plugins/` and `themes/`) lists those paths in a `.lycheeignore` at its root, lychee's own file, so the docs check does not follow their links. Then the readme: the newest entry under `== Changelog ==` is headed `= X.Y.Z =` (or `= Unreleased =`) and its first line is `Unreleased.` diff --git a/docs/configuration.md b/docs/configuration.md index 9116e4ad..288dc507 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -27,6 +27,7 @@ What a level may do: `any` value; `lower-only`; `stricter` (switch off, narrow, | `test-repo`: The test repository, where the release lane plays every scenario before `v7` moves. | the organisation's own, never the engine's | `any` | `never` | a repository of the organisation of kind `engine-test`, and no product | [`org.py`](../scripts/org.py) | #563 | | `core`: The engine's core files; while an open autopilot pull request touches one, no other issue whose Spec touches one starts. | `[]` | `any` | `never` | globs of the engine's files | [`org.py`](../scripts/org.py) | #563, #568 | | `adopted-property`: The custom property that says a repository is adopted, which the organisation's rulesets cover. | `"dx-adopted"` | `any` | `never` | a custom property of the organisation | [`sync-repos.py`](../scripts/sync-repos.py) | #562 | +| `test-property`: The custom property, true on the test repository alone, that the required workflow's ruleset excludes, so the test repository runs the candidate's pipeline from its own caller. | `"dx-engine-test"` | `any` | `never` | a custom property of the organisation | [`org.py`](../scripts/org.py), [`sync-repos.py`](../scripts/sync-repos.py) | #748 | | `conventions-contradictions`: A repository's own commit rules that contradict the organisation's, which the daily drift check lists. | the patterns in org.yml | `any` | `never` | each pattern with the lines it must find and must not | [`sync-repos.py`](../scripts/sync-repos.py) | #562 | ## `repos.yml` diff --git a/docs/versioning.md b/docs/versioning.md index 91751e1c..9162c0c9 100644 --- a/docs/versioning.md +++ b/docs/versioning.md @@ -25,7 +25,7 @@ No person's step for a fix or a feature: the release lane ([`release-lane.yml`]( 1. **Plan.** When the pull request pipeline ends green on a push to `main` (or a maintainer dispatches the lane), `release-lane.py plan` reads what merged since the last `vX.Y.Z` (`scripts/next-version.py`, from the labels of what merged) and whether a commit since says `!`. Nothing releasable (docs, chores): the lane ends. 2. **Candidate.** `release-lane.py candidate` commits a `release-candidate` branch that is `main` with the reusable workflows' inner calls of the engine moved from `@v7` to `@v7-next` (GitHub reads them before any expression, so they would otherwise run `v7`'s), checks it differs from `main` in nothing else, and points the lightweight `v7-next` tag at it. Its token has `workflows`, since that commit rewrites workflow files, and the commit is made through the API with that token (`scripts/api-commit.py`), so GitHub signs it: the organisation's ruleset refuses an unsigned commit on any branch. -3. **Scenarios.** The test repository (org.yml `test-repo`, `-test` by default; kind [`engine-test`](../kinds/engine-test/), which calls no model and publishes nothing) has its callers at `@v7-next`: they move there, and back after the scenarios, through a pull request the bot opens in the test repository (with an App token scoped to it, `DX_CALLERS_TOKEN`, minted again for the way back by `release-scenarios.py back`, since one lasts an hour and the scenarios can take longer) and merges itself through the merge queue, since its rulesets refuse a write to `main`: the auto-merge decision never merges a change to `.github/workflows`, so the lane enables no auto-merge, reads from GitHub what the pull request really changes (every file, at its base and its head), refuses one that changes anything but the engine's ref in the callers it wrote (`@v7` ↔ `@v7-next`, and the pin's comment; `release-scenarios.py only_ref`), closing it, and once its required checks pass and every other check finished, queues it with `gh pr merge --squash --match-head-commit`; the first scenario plays only once it merged, and one that does not merge, or that does not put them back, is a failing `callers` row, so nothing is released; so is a record of what moved that `back` cannot read (`move`, like `run`, writes it once the callers moved, before the first scenario, so a run cancelled or killed while moving them leaves none, and that reads as "could not tell", never "nothing moved"). The lane plays there every scenario of [`tests/release/scenarios.yml`](../tests/release/scenarios.yml) in three jobs: `move` first checks that the organisation variable `DX_TEST_LOGIN` is the owner GitHub names for `DX_TEST_TOKEN` (`org.py test-identity --check`, #739) and stops, naming the value to set and the URL, when it is unset or differs, so nothing moves or plays; then it points the callers (`release-scenarios.py move`, which records what moved) and prints the groups (`release-scenarios.py groups`); `play` is a matrix with one job per group, all at once (`fail-fast: false`), on `ubuntu-latest`, never `DX_RUNNER_HEAVY`, where the test repository's own jobs run and would queue behind the play jobs that wait for them (#707; `release-scenarios.py coverage` refuses it), each playing its group (`release-scenarios.py play`) with the test identity's token alone (`DX_TEST_TOKEN`, in the `release` environment; no App token); and `back`, under `always()` once every group ended, puts the callers back. The groups are data: every scenario is a group of its own (#694), since each writes only under its own directory of the test repository, `scenarios//` (its fixed change writes `scenarios/{scenario}/{n}.md`, `{scenario}` the `dx-scenario` mark that opens every body a scenario writes), and the engine-test pack's `core` is `scenarios/core-one-at-a-time/**` alone (#709), so only that scenario's two issues wait on each other's change and no other scenario's Spec touches a core; every wait of a scenario polls its state every 50 seconds, since every scenario at once shares the test identity's 5000 requests an hour, which `release-scenarios.py plan` and `coverage` refuse a plan to pass (`poll_budget`, #726) (an `until` step, never a fixed wait; an issue's pull request read from the issue's own references, never a search, whose API allows 30 a minute across every scenario at once, and a `gh` that fails for a while, a rate limit or a 5xx, retried after GitHub's back-off within the scenario's limit, #698), a scenario that needs the autopilot's result (its seal, its label, its pull request) opens its issue without `accepted`, waits `until` the triage's `awaiting-acceptance`, accepts it as a maintainer would, and waits `until` the seal and then the result (`autopilot_problems`, #713), a `/claim` comes before `accepted`, a `push` writes on the head branch of the pull request it observes (the autopilot's, for an issue) and lands on `main` (`conflict` the same file, otherwise; `behind` one of its own) through a docs pull request of the scenario that merges, never a write to `main`, a scenario stops as soon as its expectation can no longer hold (`hopeless`: merged when it should not be, closed unmerged, a check that ended otherwise, an `absent` label still there once nothing runs, an expected check whose run GitHub holds for a maintainer's approval, a fork's, #709), and each has a limit of 15 minutes or less (`release-scenarios.py plan`), under its play step's 20; `release-scenarios.py coverage` fails for a scenario in no group or in two, two scenarios that could write to the same part of the core, and a group id unfit for an artifact name. `act` merges what every job wrote into one table (`release-scenarios.py merge`, into the run's summary): each group's rows of its own scenarios, and a failing row, with the lane's run, for a group that left no results (its job failed, timed out or was cancelled), a scenario of a group with no row, and a `move`, `play` or `back` job that did not succeed. In the test repository the autopilot writes the fixed change (`autopilot.py fixed`: one file per criterion, committed and ticked) where it would call the model, and the reproduction copies its pack's fixed test (`repro-ways.py fixed`, #707) where the writer would run: it fails while the scenario's `scenarios//greeting.md` says "helo", which `bug-reproduced` lands on main first (a `before` with `lands`), so a bug there is confirmed or not with no model; `coverage` refuses a bug's verdict expected in a test repository with no way to reproduce it. A row that fails names what it saw: the issue's labels and the first line of the bot's newest comment. The scenarios are data: one per pair of a reusable workflow and an event the test repository's callers wire (its pack's `callers`, and org-pull-request.yml; a `schedule` aside, which no scenario can cause), and one per transition of the autopilot's state machine (but those the file's `statically:` names, each with the test that holds it instead, as no scenario can cause them there: `next_step:fix`, the review's fix round, since the test repository's review is a model's, #707), which `release-scenarios.py coverage` holds in the scripts job, with each scenario whose setup cannot make it hold (a dispatch of a workflow the test repository cannot run on demand, an issue body the schema refuses); [`scripts/caller-contract.py`](../scripts/caller-contract.py) checks before any run, for every shipped caller (the products' batteries and the nightly and security runs, which the test repository does not have, among them), that no reusable workflow asks its callers for a permission, an input or a secret they do not give (#585), and that the waiting job mints its App token for the item's repository alone, with no more than it needs (its `TOKENS`, #619). +3. **Scenarios.** The test repository (org.yml `test-repo`, `-test` by default; kind [`engine-test`](../kinds/engine-test/), which calls no model and publishes nothing) has its callers at `@v7-next`, its pull-request pipeline among them (#748): every other repository runs that pipeline as the required workflow, `org-pull-request.yml`, which the organisation ruleset "org: required workflow" reads at `v7`, its calls hard-wired to `@v7`, so a candidate's review, auto-merge decision and fix rounds would never run in a scenario; that ruleset leaves the test repository out (the custom property org.yml `test-property`, true there alone, `sync-repos.py rulesets`), which runs the same jobs from its own caller, [`test-pull-request.yml`](../workflow-templates/test-pull-request.yml) at `@v7-next`, whose `pipeline` check its own ruleset "required pipeline" requires, and `move` plays nothing (a failing `callers` row) while its default branch still requires `org-pull-request.yml` or a caller there still calls the engine off `@v7-next` (`release-scenarios.py at_candidate`; `coverage` refuses a pack with no such caller statically). They move there, and back after the scenarios, through a pull request the bot opens in the test repository (with an App token scoped to it, `DX_CALLERS_TOKEN`, minted again for the way back by `release-scenarios.py back`, since one lasts an hour and the scenarios can take longer) and merges itself through the merge queue, since its rulesets refuse a write to `main`: the auto-merge decision never merges a change to `.github/workflows`, so the lane enables no auto-merge, reads from GitHub what the pull request really changes (every file, at its base and its head), refuses one that changes anything but the engine's ref in the callers it wrote (`@v7` ↔ `@v7-next`, and the pin's comment; `release-scenarios.py only_ref`), closing it, and once its required checks pass and every other check finished, queues it with `gh pr merge --squash --match-head-commit`; the first scenario plays only once it merged, and one that does not merge, or that does not put them back, is a failing `callers` row, so nothing is released; so is a record of what moved that `back` cannot read (`move`, like `run`, writes it once the callers moved, before the first scenario, so a run cancelled or killed while moving them leaves none, and that reads as "could not tell", never "nothing moved"). The lane plays there every scenario of [`tests/release/scenarios.yml`](../tests/release/scenarios.yml) in three jobs: `move` first checks that the organisation variable `DX_TEST_LOGIN` is the owner GitHub names for `DX_TEST_TOKEN` (`org.py test-identity --check`, #739) and stops, naming the value to set and the URL, when it is unset or differs, so nothing moves or plays; then it points the callers (`release-scenarios.py move`, which records what moved) and prints the groups (`release-scenarios.py groups`); `play` is a matrix with one job per group, all at once (`fail-fast: false`), on `ubuntu-latest`, never `DX_RUNNER_HEAVY`, where the test repository's own jobs run and would queue behind the play jobs that wait for them (#707; `release-scenarios.py coverage` refuses it), each playing its group (`release-scenarios.py play`) with the test identity's token alone (`DX_TEST_TOKEN`, in the `release` environment; no App token); and `back`, under `always()` once every group ended, puts the callers back. The groups are data: every scenario is a group of its own (#694), since each writes only under its own directory of the test repository, `scenarios//` (its fixed change writes `scenarios/{scenario}/{n}.md`, `{scenario}` the `dx-scenario` mark that opens every body a scenario writes), and the engine-test pack's `core` is `scenarios/core-one-at-a-time/**` alone (#709), so only that scenario's two issues wait on each other's change and no other scenario's Spec touches a core; every wait of a scenario polls its state every 50 seconds, since every scenario at once shares the test identity's 5000 requests an hour, which `release-scenarios.py plan` and `coverage` refuse a plan to pass (`poll_budget`, #726) (an `until` step, never a fixed wait; an issue's pull request read from the issue's own references, never a search, whose API allows 30 a minute across every scenario at once, and a `gh` that fails for a while, a rate limit or a 5xx, retried after GitHub's back-off within the scenario's limit, #698), a scenario that needs the autopilot's result (its seal, its label, its pull request) opens its issue without `accepted`, waits `until` the triage's `awaiting-acceptance`, accepts it as a maintainer would, and waits `until` the seal and then the result (`autopilot_problems`, #713), a `/claim` comes before `accepted`, a `push` writes on the head branch of the pull request it observes (the autopilot's, for an issue) and lands on `main` (`conflict` the same file, otherwise; `behind` one of its own) through a docs pull request of the scenario that merges, never a write to `main`, a scenario stops as soon as its expectation can no longer hold (`hopeless`: merged when it should not be, closed unmerged, a check that ended otherwise, an `absent` label still there once nothing runs, an expected check whose run GitHub holds for a maintainer's approval, a fork's, #709), and each has a limit of 15 minutes or less (`release-scenarios.py plan`), under its play step's 20; `release-scenarios.py coverage` fails for a scenario in no group or in two, two scenarios that could write to the same part of the core, and a group id unfit for an artifact name. `act` merges what every job wrote into one table (`release-scenarios.py merge`, into the run's summary): each group's rows of its own scenarios, and a failing row, with the lane's run, for a group that left no results (its job failed, timed out or was cancelled), a scenario of a group with no row, and a `move`, `play` or `back` job that did not succeed. In the test repository the autopilot writes the fixed change (`autopilot.py fixed`: one file per criterion, committed and ticked) where it would call the model, and the reproduction copies its pack's fixed test (`repro-ways.py fixed`, #707) where the writer would run: it fails while the scenario's `scenarios//greeting.md` says "helo", which `bug-reproduced` lands on main first (a `before` with `lands`), so a bug there is confirmed or not with no model; `coverage` refuses a bug's verdict expected in a test repository with no way to reproduce it. A row that fails names what it saw: the issue's labels and the first line of the bot's newest comment. The scenarios are data: one per pair of a reusable workflow and an event the test repository's callers wire (its pack's `callers`, `test-pull-request.yml` among them; a `schedule` aside, which no scenario can cause), and one per transition of the autopilot's state machine (but those the file's `statically:` names, each with the test that holds it instead, as no scenario can cause them there: `next_step:fix`, the review's fix round, since the test repository's review is a model's, #707), which `release-scenarios.py coverage` holds in the scripts job, with each scenario whose setup cannot make it hold (a dispatch of a workflow the test repository cannot run on demand, an issue body the schema refuses); [`scripts/caller-contract.py`](../scripts/caller-contract.py) checks before any run, for every shipped caller (the products' batteries and the nightly and security runs, which the test repository does not have, among them), that no reusable workflow asks its callers for a permission, an input or a secret they do not give (#585), and that the waiting job mints its App token for the item's repository alone, with no more than it needs (its `TOKENS`, #619). 4. **Decide** (`release-lane.py decide`, then `act`): - every scenario passed and nothing is breaking: only while `main` is still the commit the plan named and the scenarios tried (else nothing, and the next green run plans again), the release pull request (`scripts/changelog.py build --next vX.Y.Z`) is opened from that commit and merged as dilux-bot, `vX.Y.Z` is tagged at its merge commit when its parent is that commit (else nothing is tagged and `v7` stays), `v7` moves to it (lightweight: `git tag -f` and a force push of that ref alone) and its GitHub Release is published, or edited when [`engine-release.yml`](../.github/workflows/engine-release.yml) made it first on the tag's push; a Release that fails after `v7` moved fails the job, and the watch still runs; - one scenario failed: nothing is tagged, `v7` stays, and a bug names the scenario and its run; a job of the scenarios that failed (a step that broke, its time run out) still reaches this decision, as a failing row of the merged table with the lane's run (the job's, a group that left no results, a scenario with no row), so it opens a bug and never a release (#644); so does a play job that ran out of time (#690): its play step stops at 20 minutes (a failure), the job at 30 (GitHub then cancels the job, not the run), and the callers still go back; only a run a person cancelled stays silent, releasing nothing and opening nothing; diff --git a/docs/workflows.md b/docs/workflows.md index 8b7ac28e..60142498 100644 --- a/docs/workflows.md +++ b/docs/workflows.md @@ -27,12 +27,12 @@ scripts with nothing else to set. A workflow another one calls is named at | [`pr-screenshots.yml`](../.github/workflows/pr-screenshots.yml) | When an autopilot pull request whose head is a branch of the same repository (never a fork's) is marked ready (#564), from `pull-request.yml` and `org-pull-request.yml`: the before and after screenshots of the browser-layer specs it affects, in its "📸 Screenshots" section ([`scripts/screenshots.py`](../scripts/screenshots.py)). Which specs comes from the pack: the way whose layer is `browser` and `repro.affected` on the paths it changes; none, nothing runs. The base and the head each run in their own checkout with the way's `setup`, its `shots` command through a generated `playwright.dx-shots.config.ts` that imports the repository's configuration and turns `screenshot` on (the repository's own is not changed), and its `teardown`, on a fresh GitHub runner with a read-only token and no secret; a second job, which never checks the code out, commits the images to the orphan branch `dx-screenshots` (`pr-/{before,after}//.png`) and rewrites the section between its markers as a before/after table. A run that fails says so there and blocks nothing. | every repository with a browser way | | [`autopilot-sweep.yml`](../.github/workflows/autopilot-sweep.yml) | Reusable, and run here every hour (#561, #669): a merge into main brings every open autopilot pull request it leaves behind or in conflict back in step on its own, with no label a person adds. [`scripts/autopilot.py`](../scripts/autopilot.py) `sweep` (next-step's answer) says `update`, and the bot merges main into it (GitHub's update-branch), or `continue`, and dilux-bot adds `autopilot:continue` to its issue, whose fix round merges main and leaves the conflict to the agent (free: `autopilot:catchup` beside the continue labels it `autopilot:free-N`, since it only catches up with another merge, #563). A draft only behind main gets the same update-branch, with no model and no run (#563); one in conflict is left to its own run, which merges main first. An issue waiting for another autopilot pull request on the engine's core (`autopilot:core-wait`, org.yml `core`, #563) gets its `autopilot:continue` once no open one touches the core its Spec touches. Within minutes of a merge, in the repository it merged in: the `autopilot-sweep` job of its issues.yml ([`workflow-templates/issues.yml`](../workflow-templates/issues.yml), and its mirror here) calls it on a pull request merged into the default branch (`pull_request_target` `closed`, metadata only), with `repo` and an App token for that repository alone; every hour for all, which covers a repository whose callers are not in step yet; and by hand. It runs only the engine's scripts, cloned at `job.workflow_sha` into `.dx-central`, never the caller's code. | `repo` (one repository; empty, every one) | | [`flow-numbers.yml`](../.github/workflows/flow-numbers.yml) | Not reusable: every Monday and by hand (#479), how the flow spent its time over the last seven days, in one issue of this repository, labelled `run:flow-numbers` and pinned, whose body [`scripts/flow-numbers.py`](../scripts/flow-numbers.py) rewrites every run: the lead time from the issue a merged pull request closes to its merge (median and average, the autopilot's and the others' apart), the merge queue's failure rate and the median minutes its groups' runs took, the autopilot's runs per merged pull request (its `autopilot:run-N` and `autopilot:free-N` labels) and the flaky tests its E2E jobs reported (the "Flaky test" warnings of `scripts/flaky.py`), each once. The issue is public: a private repository is in the numbers, its flaky tests counted, never named. | none | -| [`release-lane.yml`](../.github/workflows/release-lane.yml) | Not reusable: the engine releases itself (#563). On the pull request pipeline ending green on a push to `main`, or by hand: [`scripts/release-lane.py`](../scripts/release-lane.py) plans what is releasable since the last `vX.Y.Z`, points the lightweight `v7-next` tag at a release candidate (main with the inner `@v7` calls at `@v7-next`, nothing else), plays every scenario of [`tests/release/scenarios.yml`](../tests/release/scenarios.yml), at once (one job each, on GitHub's runners and never `DX_RUNNER_HEAVY`, where the test repository's own jobs run, #707; each scenario on its own core of the test repository, #694), in the test repository (org.yml `test-repo`, whose callers reach `@v7-next`, and come back after, through a pull request the bot opens there and queues itself in the merge queue once its checks pass, after checking that it changes nothing but the engine's ref, the way back with an App token minted after the scenarios) with the test identity's token, and only when all pass, and main is still the commit they tried, merges the release pull request, tags `vX.Y.Z`, moves `v7` (lightweight) and publishes the Release; a failed scenario opens a bug naming it and its run (of Type Bug, in the issue schema's sections, its `dx-release-lane` mark first, so the triage labels it and the reproduction runs on it as on a person's bug, #694), and nothing moves, a job of the scenarios that failed or left no results too (#644); a breaking change opens the release issue for a person. Then, for 30 minutes, it watches every adopted repository and moves `v7` back to the previous release on the first run that never started or failed in one of the engine's own steps of an engine job, with a bug and a line on the release ([docs/versioning.md](versioning.md)). Its tag-pushing jobs and the test identity's token run in the `release` environment, main only. | none; `DX_TEST_TOKEN`, `DX_TEST_FORK_TOKEN` (#694) and dilux-bot's key in the `release` environment | +| [`release-lane.yml`](../.github/workflows/release-lane.yml) | Not reusable: the engine releases itself (#563). On the pull request pipeline ending green on a push to `main`, or by hand: [`scripts/release-lane.py`](../scripts/release-lane.py) plans what is releasable since the last `vX.Y.Z`, points the lightweight `v7-next` tag at a release candidate (main with the inner `@v7` calls at `@v7-next`, nothing else), plays every scenario of [`tests/release/scenarios.yml`](../tests/release/scenarios.yml), at once (one job each, on GitHub's runners and never `DX_RUNNER_HEAVY`, where the test repository's own jobs run, #707; each scenario on its own core of the test repository, #694), in the test repository (org.yml `test-repo`, whose callers reach `@v7-next`, its pull-request pipeline `test-pull-request.yml` among them, since the required workflow runs at `v7` with its calls hard-wired, so the scenarios' reviews, auto-merge decisions and fix rounds run the candidate's engine, #748; `move` plays nothing while the test repository still requires `org-pull-request.yml` or calls the engine off `@v7-next`, and come back after, through a pull request the bot opens there and queues itself in the merge queue once its checks pass, after checking that it changes nothing but the engine's ref, the way back with an App token minted after the scenarios) with the test identity's token, and only when all pass, and main is still the commit they tried, merges the release pull request, tags `vX.Y.Z`, moves `v7` (lightweight) and publishes the Release; a failed scenario opens a bug naming it and its run (of Type Bug, in the issue schema's sections, its `dx-release-lane` mark first, so the triage labels it and the reproduction runs on it as on a person's bug, #694), and nothing moves, a job of the scenarios that failed or left no results too (#644); a breaking change opens the release issue for a person. Then, for 30 minutes, it watches every adopted repository and moves `v7` back to the previous release on the first run that never started or failed in one of the engine's own steps of an engine job, with a bug and a line on the release ([docs/versioning.md](versioning.md)). Its tag-pushing jobs and the test identity's token run in the `release` environment, main only. | none; `DX_TEST_TOKEN`, `DX_TEST_FORK_TOKEN` (#694) and dilux-bot's key in the `release` environment | | [`revert-on-red.yml`](../.github/workflows/revert-on-red.yml) | Not reusable: gate G8 (#142), every 15 minutes and by hand, over every repository dilux-bot is installed on, with nothing in any of them. [`scripts/revert.py`](../scripts/revert.py) `scan`, with a read-only token, finds where each workflow's runs on the default branch went red, the pull request that merge came from, and whether the same jobs passed on its head. The first red: its failed jobs run again once (a token for that repository, `actions: write`); green on the re-run, the run says it was flaky and nothing is reverted. Red again: a Bug issue linking the merge, the failing run and the original pull request; a pull request `revert: ` (label `revert`) closing it; `reverted` on the original; its closed issues reopened. A merge that does not revert cleanly, or that changed workflow files (the App may not push them), gets the issue alone, with who does what. Never twice, never a revert's own merge, never with `DX_AUTOPILOT=off` or the autopilot off in the policy (the run says why). The plan between its jobs holds numbers only, and a private repository is masked in the logs and counted, not named. | — | | [`runner-vm.yml`](../.github/workflows/runner-vm.yml) | Not reusable: every five minutes from the clock in Azure (`scripts/runner-vm/clock/`, #183; its own ten-minute schedule is the fallback, since GitHub runs it only a few times a day), when a private repository's pull request dispatches it (`runner-wake.yml`) and by hand, starts the organisation's VM (`vm-gh-runner-01`, shared with AI Studio) when a private repository's job waits for a `diluxone` runner, and, once no runner on it has worked for 20 minutes, cleans what jobs left (every image only past 70% of the data disk) and deallocates it. Every run keeps one issue here, "Jobs waiting for the organisation's VM", listing each job queued for more than 15 minutes with its link, closed when none waits that long (#154). Before counting, it cancels the queued or in-progress runs of pull requests closed without merging, so nothing nobody reads holds the VM (#341, the App's token with `actions: write`). The issue is public, so unlike the autonomy digest it names the private repositories (`repos.yml` already does) and links their jobs, which only members can open; it says nothing a repository holds: no job name, branch or title. The decisions are [`scripts/runner-demand.sh`](../scripts/runner-demand.sh), with its `--test`. Runs here because this repository is public, so its minutes are free. | `action`: auto, start, deallocate | | [`runner-wake.yml`](../.github/workflows/runner-wake.yml) | In a private repository (`DX_RUNNER_HEAVY` names a self-hosted runner), on a pull request from a branch of the repository: dispatches `runner-vm.yml` with `start`, so the VM its jobs wait for starts at once instead of on the schedule, which GitHub delays or skips under load (#154). On GitHub's runner, a few seconds; reads no code of the pull request; dilux-bot's token limited to this repository and `actions: write`. Without it, a warning, and the schedule starts the VM. Called from `org-pull-request.yml`. | secret `DILUX_BOT_PRIVATE_KEY`, optional | | [`runner-image.yml`](../.github/workflows/runner-image.yml) | Not reusable: every day, on a change to `runner/` and by hand, builds `ghcr.io/diluxone/runner` (`runner/Dockerfile`: the latest actions/runner, checked against its SHA-256, Node LTS, Docker CLI, Chromium's libraries, the shared libraries setup-php's prebuilt PHP links against), smoke-tests it (Docker through the socket, sudo, Chromium launching, and PHP 7.4, 8.3 and 8.5 set up by setup-php at the commit the workflows pin, each with `php -v` and `composer --version`, after checking that php-builder's 8.3 to 8.5 find every library they need in the image itself) and only then tags it `:stable` and with its date. A failed build leaves `:stable` as it was. Each build is an issue of its own, labelled `run:runner-image` ([`scripts/run-issue.sh`](../scripts/run-issue.sh), #188): a promoted one closes as completed; a failed one stays open with `needs:maintainer`, opened by dilux-bot as a `platform` lane issue (`lane:platform`, `accepted-by:policy`, the run named in its body), so a fix for it needs no person to accept. No person approves it: the `platform` lane (#78). | — | -| [`org-pull-request.yml`](../.github/workflows/org-pull-request.yml) | Not reusable: the pull request pipeline of every repository but this one, required by the organisation's `main` ruleset. Calls `conventions.yml`, `claude-review.yml`, `auto-merge.yml`, `pull-request-assignee.yml` and `runner-wake.yml` (not on `edited`) at `@v7` with today's job names, on every pull request event, `edited` included. In a repository with a merge queue (#451) it runs on the queue's group too (`merge_group`), where only its `queue` job runs: the conventions, the review and the merge decision were the pull request's. When the queue drops an autopilot pull request (`dequeued`, #479), only `autopilot-queue` runs: one `autopilot.yml` fix round, within `autopilot-rounds`, when a check failed in its last merge group (`autopilot.py queue-failed`), from the failed jobs' log. An autopilot pull request whose review blocks, whose conventions fail or whose review ends without a verdict gets one `autopilot.yml` fix round from `autopilot-fix` (#83, #741), which waits for the conventions and the review. | none; the repository's `.github/review-policy.yml` | +| [`org-pull-request.yml`](../.github/workflows/org-pull-request.yml) | Not reusable: the pull request pipeline of every repository but this one, the required workflow of the organisation ruleset "org: required workflow" (#748), at `v7`, in every adopted repository but the test repository (org.yml `test-repo`): the ruleset reads it at `v7`, its calls hard-wired to `@v7`, so it could never try a candidate there; the test repository runs the same jobs from its own caller, [`test-pull-request.yml`](../workflow-templates/test-pull-request.yml), which the release lane moves to `@v7-next`, plus a `pipeline` job, the run as one check, which its ruleset "required pipeline" requires. Calls `conventions.yml`, `claude-review.yml`, `auto-merge.yml`, `pull-request-assignee.yml` and `runner-wake.yml` (not on `edited`) at `@v7` with today's job names, on every pull request event, `edited` included. In a repository with a merge queue (#451) it runs on the queue's group too (`merge_group`), where only its `queue` job runs: the conventions, the review and the merge decision were the pull request's. When the queue drops an autopilot pull request (`dequeued`, #479), only `autopilot-queue` runs: one `autopilot.yml` fix round, within `autopilot-rounds`, when a check failed in its last merge group (`autopilot.py queue-failed`), from the failed jobs' log. An autopilot pull request whose review blocks, whose conventions fail or whose review ends without a verdict gets one `autopilot.yml` fix round from `autopilot-fix` (#83, #741), which waits for the conventions and the review. | none; the repository's `.github/review-policy.yml` | | [`pull-request-assignee.yml`](../.github/workflows/pull-request-assignee.yml) | When a pull request from a branch of the repository opens, reopens, leaves draft or changes its description, and its author is a member of the organisation (`author_association` `OWNER` or `MEMBER`): each open issue of the repository it closes is reassigned to that author, unless they are already among its assignees (#73, [`scripts/issue-assignee.sh`](../scripts/issue-assignee.sh) `pr`; the author is added first, the others removed only once GitHub has them). An outside contributor's, a bot's (the autopilot's: the person who accepted the issue stays its assignee, even after a maintainer takes the pull request over) and a fork's pull request leave the assignee as it is. A maintainer's later reassignment by hand holds until the pull request reopens, leaves draft or changes its description again. A failure is a warning, never a red check. Called from `org-pull-request.yml`, and here from `pull-request.yml` and `pull-request-edited.yml`. | `central-ref`; secret `DILUX_BOT_PRIVATE_KEY`, optional: without it, nothing changes | | [`conventions.yml`](../.github/workflows/conventions.yml) | Branch, title, commits, description sections, no "Generated with" footer, a `docs` title that changes only docs, the size limit for authors who are not trusted, no new public surface in a pull request that is not a `feat` from an author who is not trusted (the kind's `surface.yml`, #144) (all in `scripts/conventions.sh`), the accepted issue the pull request closes (`scripts/issue-gate.sh`), relative doc links, retired names. Where org.yml `pre-pr` is `required`, a repository whose Makefile has no `pre-pr` target, the one check before a pull request (#479, `scripts/pre-pr.py`; repos.yml `pre-pr` names one that has none by design); it is `optional` until every repository has one, and the daily drift check lists those that lack it meanwhile. Callers grant `issues: read` and `actions: read` (the run that opened a lane's issue). Reads the labels and the review App's last record live, so a re-run sees today's `type:*` label. | `retired-names`, `required-sections`, `max-header`, `central-ref`, `review-bot` | | [`claude-review.yml`](../.github/workflows/claude-review.yml) | The review of [the pull request flow](pull-requests.md) (step 2). `profile` (default `general`) names a kind or a pack's alias (`plugin-wp`); with `general`, the kind the repository's policy declares adds its profile. Outputs `risk`, `complexity`, `floor`, `trusted`, `blocking`. | `profile`, `central-ref`, `max-auto-reviews` | diff --git a/kinds/README.md b/kinds/README.md index 6006f3a7..b4b53efa 100644 --- a/kinds/README.md +++ b/kinds/README.md @@ -46,7 +46,7 @@ kinds// | `settings` | What the workflows read from the pack, free-form per kind (for wordpress-plugin, `php`: `tools`, the PHP every tool runs on, `minimum`, the oldest every plugin supports, and `analysis-only`, the dev packages the unit tests leave out, read by [`scripts/php-versions.py`](../scripts/php-versions.py); `plugin-check`: `strict`, `categories`, `include-experimental`, `ignore-codes`). | | `autopilot` | What the autopilot adds to every change of this kind beside what the Spec names (#341), one instruction per line, read into its work prompt by [`scripts/autopilot.py`](../scripts/autopilot.py) for the kind the repository's `.github/review-policy.yml` declares. For wordpress-plugin, the change's changelog line in a file of its own, `changelog.d/.md`, never in `readme.txt`, which the release writes (#452), and the translations in their normalised form (`languages/*.pot` and `*.po`, every new string translated, never a `.mo`). | | `repro` | How a bug is reproduced, by the layer the triage says it shows in (#352), read by [`scripts/repro-ways.py`](../scripts/repro-ways.py), whose header says what each key means. `ways`: one per way, each for a `layer` (`logic`, `browser`, `cli`, `api` or `workflow`, a CI workflow's wiring, or a list of them; a repository's own way comes first), with the `test` file the agent writes (`{n}` the issue's number) and the `needs` it must find; `sandbox: php-unit` runs it in [`scripts/repro-sandbox.sh`](../scripts/repro-sandbox.sh), and any other is a command way: `setup` (with the network, before the test is in the tree), `theme` (installs a theme a report names), `run` and the `junit` report it writes (`case`, what the report names the test by when no test case carries its file, and `assertion`, what a failure that is an assertion's says, for a framework the verdict does not know), run by [`scripts/repro-run.sh`](../scripts/repro-run.sh) with no environment and no network but the machine's; `minutes`, and `context`, `list`, `example` and `guide` for the writer's brief; `fixed` (#707), a test no model writes, copied to `test` in place of the writer's (`{kind}` the pack's folder; `{n}` and `{scenario}`, the issue's `dx-scenario` mark, filled), a command way's. `affected`: paths mapped to the tests a pull request that touches them runs instead of the whole suite, when every path it touches is mapped (an entry with no tests names what no test covers, such as wordpress-plugin's `readme.txt`). `smoke` (#479): the few specs a pull request runs beside the mapped ones where the repository has a merge queue, whose group runs every suite whole (`scripts/e2e-run.sh`, `LEVEL=pr`); the pack's, then the repository's. A repository changes a way, adds one or switches one off (`false`) in its `.github/review-policy.yml`, and adds its own `affected` and `smoke`; the organisation's `repos.yml` may hold one for it too (`repro:`, under its name), read between the pack's and its own (the installer proposes one there from the repository's stack, [`policy/repro-stacks.yml`](../policy/repro-stacks.yml), the stacks every kind shares, for a repository that declares none, #377). Nothing falls back to a stack (#376): a layer with no way, or a way whose `needs` are missing, gets `repro:no-way` in place of `bug:unconfirmed`, with the reason and how to declare one, before any model runs; a repository that declares no way at all (no kind, nothing in `repos.yml` `repro:` and nothing in its policy) has none, and the daily drift check lists it unless `repos.yml` says it has none by design (`repro: {: false}`). DiluxOne/.github, which has no kind, declares its own in its `.github/review-policy.yml`: `python` (a pytest test under `tests/repro/` that calls the script the report names) and `workflow` (one that reads the workflow's file as YAML). For wordpress-plugin, `unit` (PHPUnit under `tests/Unit/Repro/`, for logic, WP-CLI and REST bugs) and `browser` (Playwright on wp-env under `tests/e2e/repro/`, through [`playwright.dx-repro.config.ts`](wordpress-plugin/playwright.dx-repro.config.ts)). For ai-agent-plugin (#653), `unit`: pytest, with PyYAML, under `tests/repro/`, for logic and command bugs; a repository of the kind with no Python tests switches it off (`repro: {ways: {unit: false}}`) and declares its own. For engine-test (#707), `fixed`, for every layer: [`repro-fixed.py`](engine-test/repro-fixed.py), which fails while `scenarios//greeting.md` says "helo" (what the release scenario `bug-reproduced` lands on main first) and passes otherwise, so the test repository's bugs are reproduced with no model. | -| `identity`, `fixed-change`, `core`, `callers`, `cleanup`, `publishes` | engine-test only (#563, read by [`scripts/org.py`](../scripts/org.py) `test_pack`): `identity`, the test identity (`secret`, its token's secret in the `release` environment, and `login`, `{org}` the organisation's login lowercased), which may accept in a repository of this kind and nowhere else; `autopilot: fixed-change` with `fixed-change.file`, the change the autopilot commits per criterion instead of calling a model (`autopilot.py fixed`: autopilot.yml skips the model's steps there); `core`, the test repository's own core globs in place of org.yml `core` (`org.py core_of`), the directory of the scenario of one change to the core at a time alone (`scenarios/core-one-at-a-time/**`, with `fixed-change.file` `scenarios/{scenario}/{n}.md`, `{scenario}` the issue's `dx-scenario` mark; #709), so only that scenario's two issues touch the core and wait on each other, and every other scenario's Spec touches none and plays at once (`release-scenarios.py core_problems` fails for any other scenario that writes under it); `callers`, the shipped callers (workflow-templates/) the installer writes there, the only ones the scenarios play (`release-scenarios.py coverage`; every other is held by `caller-contract.py check`); `cleanup: after-run`, its issues and pull requests closed and their branches deleted after each run; `publishes: never`, no release and no catalog. The daily drift check fails when a repository of this kind is also of a product kind or in a catalog, or when the identity can accept elsewhere. | +| `identity`, `fixed-change`, `core`, `callers`, `cleanup`, `publishes` | engine-test only (#563, read by [`scripts/org.py`](../scripts/org.py) `test_pack`): `identity`, the test identity (`secret`, its token's secret in the `release` environment, and `login`, `{org}` the organisation's login lowercased), which may accept in a repository of this kind and nowhere else; `autopilot: fixed-change` with `fixed-change.file`, the change the autopilot commits per criterion instead of calling a model (`autopilot.py fixed`: autopilot.yml skips the model's steps there); `core`, the test repository's own core globs in place of org.yml `core` (`org.py core_of`), the directory of the scenario of one change to the core at a time alone (`scenarios/core-one-at-a-time/**`, with `fixed-change.file` `scenarios/{scenario}/{n}.md`, `{scenario}` the issue's `dx-scenario` mark; #709), so only that scenario's two issues touch the core and wait on each other, and every other scenario's Spec touches none and plays at once (`release-scenarios.py core_problems` fails for any other scenario that writes under it); `callers`, the shipped callers (workflow-templates/) the installer writes there, and `sync-repos.py callers` too (#748: one that no other repository has, `test-pull-request.yml`, the pull-request pipeline in place of the required workflow, it writes to this kind's repository alone and holds job for job there, and compares with nothing elsewhere), the only ones the scenarios play (`release-scenarios.py coverage`; every other is held by `caller-contract.py check`); `cleanup: after-run`, its issues and pull requests closed and their branches deleted after each run; `publishes: never`, no release and no catalog. The daily drift check fails when a repository of this kind is also of a product kind or in a catalog, or when the identity can accept elsewhere. | | `release` | How a version of this kind is released (#82), read by [`scripts/release-issue.py`](../scripts/release-issue.py): `publishes-to` (where it goes, for people; `{catalog}` is the organisation's catalog for the kind, org.yml `catalogs`, so the pack names no organisation's own, #568), `workflow` (the reusable workflow that releases it, ending at the GitHub release with its build, #287; the repository's caller of it holds the inputs), `publish` (the reusable workflow that, on that release's publishing, sends its build where the kind publishes; for wordpress-plugin, `plugin-publish-wp.yml` to wordpress.org), `prepare` (the command that turns the default branch into the release pull request, one argument per item: `{version}`, `{central}` for the checkout of this repository, any other `{name}` the caller's input) `defaults` (an input the caller leaves out) and `stable` (where the version in production is published, `url` with `{slug}` and the JSON `field`, which [`scripts/production-version.py`](../scripts/production-version.py) checks the last GitHub release against before a security patch, #247). A kind without it publishes nothing, and its release issue says so. | A repository can add to a list in `settings` from its own policy, never diff --git a/kinds/engine-test/pack.yml b/kinds/engine-test/pack.yml index 3c34233b..ba3e8ec0 100644 --- a/kinds/engine-test/pack.yml +++ b/kinds/engine-test/pack.yml @@ -19,8 +19,11 @@ workflows: [] pre-pr-tools: [] # The shipped callers (workflow-templates/) the test repository has, which -# scripts/install.py writes there, beside org-pull-request.yml, the -# organisation's required workflow. A scenario plays only what these wire (scripts/release-scenarios.py +# scripts/install.py and scripts/sync-repos.py callers write there (#748). +# test-pull-request.yml is its pull-request pipeline, the jobs of +# org-pull-request.yml, the required workflow, which runs at `v7` in every +# other repository and leaves this one out ("org: required workflow"): its +# calls are this repository's, so they move to the candidate. A scenario plays only what these wire (scripts/release-scenarios.py # coverage); every other caller is held to the engine statically # (scripts/caller-contract.py check). While the lane tries a candidate they # move to `v7-next` through a pull request the bot opens there and queues @@ -32,6 +35,7 @@ pre-pr-tools: [] callers: - issues.yml - pull-request-comments.yml + - test-pull-request.yml # No checks of its own to run, so no engines. adapters: [] diff --git a/org.yml b/org.yml index 4bc5d675..bd4d6ec3 100644 --- a/org.yml +++ b/org.yml @@ -85,6 +85,14 @@ core: # repos.yml lists, written by the sync, and the organisation's rulesets # cover the repositories that have it instead of a list of names. adopted-property: dx-adopted +# The custom property that sets the test repository apart (#748, +# scripts/sync-repos.py rulesets): `true` on org.yml `test-repo` alone, +# written by the sync. The ruleset "org: required workflow", which runs +# org-pull-request.yml at `v7` in every adopted repository, excludes the +# repositories that have it: the test repository runs the same pipeline +# from its own caller (workflow-templates/test-pull-request.yml), which the +# release lane moves to `v7-next`. Absent, `dx-engine-test`. +test-property: dx-engine-test # A repository's own commit rules that contradict the organisation's (#562, # scripts/sync-repos.py check): each pattern is matched against its # commit-msg hook (.githooks/, .husky/), its commitlint configuration, and diff --git a/policy/settings-schema.yml b/policy/settings-schema.yml index 2617803d..67f995a6 100644 --- a/policy/settings-schema.yml +++ b/policy/settings-schema.yml @@ -143,6 +143,13 @@ files: read-by: [scripts/sync-repos.py] issue: "#562" says: The custom property that says a repository is adopted, which the organisation's rulesets cover. + - key: test-property + org: any + repo: never + limit: a custom property of the organisation + read-by: [scripts/org.py, scripts/sync-repos.py] + issue: "#748" + says: The custom property, true on the test repository alone, that the required workflow's ruleset excludes, so the test repository runs the candidate's pipeline from its own caller. - key: conventions-contradictions default: the patterns in org.yml org: any diff --git a/scripts/install.py b/scripts/install.py index 0f240914..c4b3052b 100755 --- a/scripts/install.py +++ b/scripts/install.py @@ -572,7 +572,9 @@ def tcmd(argv, env=None): import base64 as b64 issues_caller = b64.b64decode(next(x for x in written[0] if x.startswith("content=")).split("=", 1)[1]).decode() check("C9: its callers are written, the engine pinned at its commit for another organisation", - (len(written), f"issue-triage.yml@0123abcd # v7" in issues_caller, "@v7\n" in issues_caller), (2, True, False)) + (len(written), f"issue-triage.yml@0123abcd # v7" in issues_caller, "@v7\n" in issues_caller), (len(TEST_CALLERS), True, False)) + check("#748: test-pull-request.yml, its pull-request pipeline, is among them", + any("/contents/.github/workflows/test-pull-request.yml" in a[3] for a in written), True) check("C9: it is adopted", any(c[2:5] == ["adopt", "--repo", "acme-test"] for c in tcmds), True) check("C9: the test identity gets write there", ("api", "-X", "PUT", "repos/Acme/acme-test/collaborators/someone-else", "-f", "permission=push") in tcalls, True) check("C9: every scenario runs there once", sum(1 for c in tcmds if "release-scenarios.py" in c[1] and c[2:6] == ["run", "--repo", "Acme/acme-test", "--ref"]), 1) diff --git a/scripts/org.py b/scripts/org.py index 90cd9b96..33e1ef68 100755 --- a/scripts/org.py +++ b/scripts/org.py @@ -133,6 +133,19 @@ def test_repo(conf): TEST_KIND = "engine-test" +def test_property(conf): + """org.yml `test-property` (#748): the custom property of the + organisation, true on the test repository alone, that the + required-workflow ruleset excludes (scripts/sync-repos.py rulesets); + absent, `dx-engine-test`.""" + value = str(conf.get("test-property") or "").strip() + if not value: + return "dx-engine-test" + if not all(c.isalnum() or c in "-_" for c in value): + raise ValueError(f"org.yml `test-property` is `{value}`: a custom property's name") + return value + + def test_identity(env=os.environ): """The test identity's login (#739): DX_TEST_LOGIN, the owner of its token as the release lane read it from GitHub, lowercased; "" when it is @@ -284,6 +297,13 @@ def check(name, got, want): check("C1 (#563): a test-repo with an owner fails", "no error", "ValueError") except ValueError: check("C1 (#563): a test-repo with an owner fails", "ValueError", "ValueError") + check("#748: no test-property is dx-engine-test; org.yml's is its own", + (test_property({}), test_property({"test-property": "acme-test"}), test_property(_conf)), ("dx-engine-test", "acme-test", "dx-engine-test")) + try: + test_property({"test-property": "a b"}) + check("#748: a test-property that is no property's name fails", "no error", "ValueError") + except ValueError: + check("#748: a test-property that is no property's name fails", "ValueError", "ValueError") pack = test_pack({"org": "Acme"}) check("C1 (#739): the pack names the test identity's token, and no login", pack["identity"], {"secret": "DX_TEST_TOKEN"}) check("C1, C4 (#739): the test identity is DX_TEST_LOGIN, whoever owns the token, lowercased; none when it is unset", diff --git a/scripts/release-scenarios.py b/scripts/release-scenarios.py index be5ea4c3..60883265 100644 --- a/scripts/release-scenarios.py +++ b/scripts/release-scenarios.py @@ -4,8 +4,9 @@ moves, as data in tests/release/scenarios.yml. Every pair of a reusable workflow and an event the test repository's -callers wire (its pack's `callers` among workflow-templates/*.yml, and -org-pull-request.yml, the organisation's required workflow), but a +callers wire (its pack's `callers` among workflow-templates/*.yml, its +pull-request pipeline test-pull-request.yml among them: the required +workflow, org-pull-request.yml, runs at `v7` everywhere else, #748), but a `schedule` no scenario can cause, and every transition of the autopilot's state machine (scripts/autopilot.py: `next_step`'s BOT_ACTIONS, its waits on PERSON_DECISIONS and BOT_WAITS, the @@ -67,7 +68,10 @@ group or in two, two scenarios that could write to the same part of the test repository's core, a scenario that writes under it but expects no wait on it (#709), a group id unfit for an - artifact name + artifact name; and (#748, `pipeline_problems`) a pack with no + caller that wires the pull-request pipeline's jobs on + pull_request and merge_group, org-pull-request.yml among its + callers, and an engine call in one of them the lane cannot move release-scenarios.py plan prints each scenario's limit in minutes and how often it polls, [{id, minutes, poll}] as JSON; exits 1 naming a fixed wait or a @@ -80,7 +84,11 @@ the callers moved to the ref, as `run` does (with DX_CALLERS_TOKEN), what moved written to MOVED as soon as the pointing is over, and to FILE the failing `callers` row when they - did not reach it (exits 1 then: no group plays) + did not reach it, or (#748, `at_candidate`) one for each pipeline + there that would still run off the ref: its default branch still + requiring org-pull-request.yml (a `workflows` rule), or a + .github/workflows/*.yml calling the engine off the ref (exits 1 + then: no group plays, and `back` puts back what moved) release-scenarios.py play --repo OWNER/NAME --group ID --out FILE [--ref v7-next] [--minutes N] the group's scenarios, one after another in its order, with the @@ -192,10 +200,61 @@ def pairs(root=ROOT, callers=CALLERS): UNPLAYABLE = ("schedule",) +REQUIRED = ".github/workflows/org-pull-request.yml" + + def test_callers(pack): """The test repository's callers, as globs: its pack's `callers` among - the shipped ones, and org-pull-request.yml, which runs everywhere.""" - return tuple(f"workflow-templates/{c}" for c in pack.get("callers") or []) + (".github/workflows/org-pull-request.yml",) + the shipped ones. The required workflow, org-pull-request.yml, runs + everywhere else, at `v7`: there, its pipeline is test-pull-request.yml's + (#748), which the lane moves to the candidate.""" + return tuple(f"workflow-templates/{c}" for c in pack.get("callers") or []) + + +def engine_calls(text, central): + """The lines of a caller that call the engine (`uses:` of a workflow of + `central`), as (number, line). Pure.""" + pat = re.compile(r"^\s*-?\s*uses:\s*['\"]?" + re.escape(central) + r"/\.github/workflows/") + return [(i, line) for i, line in enumerate(text.split("\n"), 1) if pat.match(line)] + + +def unmovable(text, central): + """The engine calls of a caller `at_ref` would not move (one that does + not match USES: quoted, with no ref, a step's), as "line N: ...". Pure.""" + uses = re.compile(USES.format(re.escape(central))) + return [f"line {i}: {line.strip()!r}" for i, line in engine_calls(text, central) if not uses.match(line)] + + +def off_ref(text, central, ref): + """The engine calls of a caller not at `ref`, as "line N: ...". Pure.""" + uses = re.compile(USES.format(re.escape(central))) + return [f"line {i}: {line.strip()!r}" for i, line in engine_calls(text, central) if not ((m := uses.match(line)) and m.group(2) == ref)] + + +def pipeline_problems(root, pack, central): + """What would leave a pipeline of the test repository at `v7` while the + lane plays the candidate (#748), statically: no caller of its pack's + that wires the pull-request pipeline's jobs on `pull_request` and + `merge_group` (every pair org-pull-request.yml wires), org-pull-request.yml + among its callers (the required workflow, whose calls are hard-wired to + `@v7`), and an engine call in one of its callers `at_ref` would not move. + Each names the file. Pure but for reading the files.""" + out, callers = [], test_callers(pack) + named = [c for c in pack.get("callers") or [] if os.path.basename(str(c)) == os.path.basename(REQUIRED)] + if named: + out.append(f"kinds/engine-test/pack.yml `callers` names {named[0]}, the required workflow, which runs at `v7` with its calls hard-wired: " + "the test repository runs the pipeline from test-pull-request.yml, which the lane moves") + want, have = pairs(root, (REQUIRED,)), pairs(root, callers) + events = {e for pattern in callers for path in glob.glob(os.path.join(root, pattern)) for e in caller_events(load_yaml(path))} + gaps = sorted(want - have) + [f"`{e}`" for e in ("pull_request", "merge_group") if e not in events] + if gaps: + out.append(f"kinds/engine-test/pack.yml `callers` has no caller that wires the pull-request pipeline ({', '.join(gaps[:4])}" + f"{', …' if len(gaps) > 4 else ''}): its pull requests would run no candidate's pipeline; add test-pull-request.yml") + for pattern in callers: + for path in sorted(glob.glob(os.path.join(root, pattern))): + for line in unmovable(open(path, encoding="utf-8").read(), central): + out.append(f"{os.path.relpath(path, root)} {line}: an engine call the lane cannot move to the candidate (`uses: {central}/.github/workflows/.yml@`)") + return out def played(root, pack): @@ -210,7 +269,7 @@ def dispatchable(root, pack): """The test repository's callers a scenario can dispatch: its own, with `workflow_dispatch`.""" out = set() - for pattern in test_callers(pack)[:-1]: + for pattern in test_callers(pack): for path in glob.glob(os.path.join(root, pattern)): if "workflow_dispatch" in caller_events(load_yaml(path)): out.add(os.path.basename(path)) @@ -1274,11 +1333,42 @@ def read_moved(path, ref): return [], [{"id": "callers", "ok": False, "why": [f"could not read what moved to {ref}, the callers may be left there: {e}"], "run": ""}] +def at_candidate(repo, ref, run=gh, central=None): + """What would still run a pipeline of the test repository off `ref` once + its callers moved (#748), as failing `callers` rows, each naming it: its + default branch's rules (`repos/{repo}/rules/branches/{default}`) still + carrying a `workflows` rule for org-pull-request.yml (the required + workflow, at `v7` with its calls hard-wired), and any + .github/workflows/*.yml there that still calls the engine off `ref`.""" + import base64 + if central is None: + central = central_of() + why = [] + base = run("api", f"repos/{repo}", "--jq", ".default_branch").strip() + pages = json.loads(run("api", "--paginate", "--slurp", f"repos/{repo}/rules/branches/{base}?per_page=100") or "[]") + rules = [r for page in pages for r in (page if isinstance(page, list) else [page])] if isinstance(pages, list) else [] + for r in rules: + for w in ((r.get("parameters") or {}).get("workflows") or []) if r.get("type") == "workflows" else []: + if os.path.basename(str(w.get("path") or "")) == os.path.basename(REQUIRED): + why.append(f"{repo}'s {base} still requires {REQUIRED} (a `workflows` rule of {r.get('ruleset_source') or 'a ruleset'}): its pull requests " + f"run the pipeline at `v7`, not {ref}; `sync-repos.py rulesets` leaves it out once it has test-pull-request.yml") + for f in json.loads(run("api", f"repos/{repo}/contents/.github/workflows") or "[]"): + name = str(f.get("name", "")) + if not name.endswith(".yml"): + continue + got = json.loads(run("api", f"repos/{repo}/contents/.github/workflows/{name}") or "{}") + for line in off_ref(base64.b64decode(got.get("content") or "").decode(), central, ref): + why.append(f".github/workflows/{name} {line}: calls the engine off {ref} after the move") + return [{"id": "callers", "ok": False, "why": [w], "run": ""} for w in why] + + def move_callers(repo, ref, callers=gh, central=None, sleep=time.sleep, minutes=20, record=None): """The test repository's callers moved to `ref` (`point_callers`): - (what moved, []) once they are, else ([], one failing `callers` row that - says why). `record`, when given, is called with what moved as soon as - the pointing is over ([] when it failed cleanly).""" + (what moved, []) once they are and nothing would run a pipeline there + off `ref` (`at_candidate`, #748), else (what moved, the failing + `callers` rows that say why; [] moved when the pointing failed). + `record`, when given, is called with what moved as soon as the + pointing is over ([] when it failed cleanly).""" try: moved = point_callers(repo, ref, callers, central, sleep=sleep, minutes=minutes) except (RuntimeError, ValueError) as e: @@ -1287,7 +1377,10 @@ def move_callers(repo, ref, callers=gh, central=None, sleep=time.sleep, minutes= return [], [{"id": "callers", "ok": False, "why": [f"the callers did not reach {ref}: {e}"], "run": ""}] if record: record(moved) - return moved, [] + try: + return moved, at_candidate(repo, ref, callers, central) + except (RuntimeError, ValueError) as e: + return moved, [{"id": "callers", "ok": False, "why": [f"could not tell whether every pipeline of {repo} runs at {ref}: {e}"], "run": ""}] GROUP_ID = re.compile(r"[a-z0-9-]+") @@ -1465,7 +1558,12 @@ def run_all(conf, repo, ref, only=(), run=gh, sleep=time.sleep, minutes=20, cent callers = callers or (callers_gh() if run is gh else run) moved, rows = move_callers(repo, ref, callers, central, sleep, minutes, record) if rows: - return rows + # What moved still goes back (#748: a pipeline off the ref plays no + # scenario, but the callers did move). + if moved_out is not None: + moved_out.extend(moved) + return rows + return rows + put_back(repo, ref, moved, callers, central, sleep, minutes) if moved_out is not None: moved_out.extend(moved) results = [] @@ -1781,17 +1879,25 @@ def lane_problems(lane): return [] +def central_of(): + sys.path.insert(0, HERE) + import org + return org.CENTRAL + + def problems(conf, root=ROOT): """What `coverage` names: every played pair and transition with no scenario, every scenario its setup cannot make hold, what is wrong - with the groups (`partition`), and with the lane's `play` job - (`lane_problems`, #707).""" + with the groups (`partition`), with the lane's `play` job + (`lane_problems`, #707), and with the test repository's pipeline + (`pipeline_problems`, #748).""" pack = test_pack(root) scenarios = conf.get("scenarios") or [] return (coverage(scenarios, played(root, pack) | transitions(root), conf.get("statically") or {}) + preconditions(scenarios, conf.get("bodies") or {}, pack, dispatchable(root, pack), schema_passes(root), conf.get("types") or {}, test_repro(root)) + lane_problems(load_yaml(os.path.join(root, LANE))) + + pipeline_problems(root, pack, central_of()) + partition(scenarios, groups(scenarios), pack) + timing(scenarios) + autopilot_problems(scenarios) @@ -2510,6 +2616,76 @@ def none_yet(*a, input=None): ([c for c in fresh if c[:2] == ("pr", "merge")], [c for c in fresh if "--auto" in c], any(c[:2] == ("pr", "checks") for c in fresh)), ([("pr", "merge", "21", "--repo", "o/t", "--squash", "--match-head-commit", "head-21")], [], True)) check("#622: put_back with nothing moved does nothing", put_back("o/t", "v7-next", [], lambda *a, input=None: 1 / 0), []) + # #748 C1: the test repository's pull-request caller, the shipped + # template as it is, moves with the others. + import yaml + shipped = open(os.path.join(ROOT, "workflow-templates", "test-pull-request.yml"), encoding="utf-8").read() + shipped = shipped.replace("DiluxOne/.github/", "O/.github/") + st, fake = repo_fake() + st["main"]["test-pull-request.yml"] = shipped + moved, rows = move_callers("o/t", "v7-next", fake, "O/.github", lambda s: None, 3) + jobs = yaml.safe_load(st["main"]["test-pull-request.yml"])["jobs"] + check("#748 C1: after point_callers, every engine call of test-pull-request.yml's review, auto-merge and autopilot-fix jobs is at @v7-next", + (rows, sorted(m["name"] for m in moved), {j: jobs[j]["uses"].rsplit("@", 1)[1] for j in ("review", "auto-merge", "autopilot-fix")}), + ([], ["issues.yml", "test-pull-request.yml"], {"review": "v7-next", "auto-merge": "v7-next", "autopilot-fix": "v7-next"})) + check("#748 C3: and put back after, as it was, with the others", + (put_back("o/t", "v7-next", moved, fake, "O/.github", lambda s: None, 3), st["main"]["test-pull-request.yml"] == shipped, st["main"]["issues.yml"] == caller), + ([], True, True)) + # #748 C4: what would leave a pipeline there at v7, refused and named. + def rules_fake(rules, tamper=None): + st, fake = repo_fake() + st["main"]["test-pull-request.yml"] = shipped + + def with_rules(*a, input=None): + if a[:3] == ("api", "--paginate", "--slurp") and a[3].startswith("repos/o/t/rules/branches/main"): + return json.dumps([rules]) + got = fake(*a, input=input) + if tamper and a[:2] == ("pr", "view") and st["main"].get("test-pull-request.yml", "").count("@v7-next") > 0: + tamper(st) + return got + return st, with_rules + required_rule = {"type": "workflows", "ruleset_source_type": "Organization", "ruleset_source": "DiluxOne", "ruleset_id": 3, + "parameters": {"workflows": [{"path": ".github/workflows/org-pull-request.yml", "repository_id": 1, "ref": "refs/tags/v7"}]}} + st, fake = rules_fake([{"type": "pull_request"}, required_rule]) + got = run_all(one, "o/t", "v7-next", run=fake, sleep=lambda s: None, minutes=3, central="O/.github") + check("#748 C4: move refuses a test repository whose default branch still requires org-pull-request.yml, naming the rule, plays nothing, and puts the callers back", + ([(r["id"], r["ok"]) for r in got], "org-pull-request.yml" in got[0]["why"][0] and "`workflows` rule" in got[0]["why"][0], + any(c[:2] == ("issue", "create") for c in st["calls"]), st["main"]["test-pull-request.yml"] == shipped), + ([("callers", False)], True, False, True)) + + def left_behind(st): + st["main"]["test-pull-request.yml"] = st["main"]["test-pull-request.yml"].replace("claude-review.yml@v7-next", "claude-review.yml@v7", 1) + st, fake = rules_fake([{"type": "pull_request"}], left_behind) + moved_rows = move_callers("o/t", "v7-next", fake, "O/.github", lambda s: None, 3) + check("#748 C4: move refuses a caller still off the ref after the move, naming the file and the line", + ([m["name"] for m in moved_rows[0]], [r["why"] for r in moved_rows[1]]), + (["issues.yml", "test-pull-request.yml"], + [[".github/workflows/test-pull-request.yml line " + str(st["main"]["test-pull-request.yml"].split("\n").index(" uses: O/.github/.github/workflows/claude-review.yml@v7") + 1) + + ": 'uses: O/.github/.github/workflows/claude-review.yml@v7': calls the engine off v7-next after the move"]])) + st, fake = rules_fake([{"type": "pull_request"}, dict(required_rule, parameters={"workflows": [{"path": ".github/workflows/other.yml"}]})]) + check("#748 C4: every caller at the ref and no required pull-request workflow left: move passes", + move_callers("o/t", "v7-next", fake, "O/.github", lambda s: None, 3)[1], []) + with tempfile.TemporaryDirectory() as root: + os.makedirs(os.path.join(root, "workflow-templates")) + os.makedirs(os.path.join(root, ".github", "workflows")) + open(os.path.join(root, REQUIRED), "w").write( + "on:\n pull_request:\n merge_group:\njobs:\n review:\n if: github.event_name == 'pull_request'\n uses: O/.github/.github/workflows/claude-review.yml@v7\n") + open(os.path.join(root, "workflow-templates", "issues.yml"), "w").write("on:\n issues:\njobs:\n t:\n uses: O/.github/.github/workflows/issue-triage.yml@v7\n") + open(os.path.join(root, "workflow-templates", "test-pull-request.yml"), "w").write(open(os.path.join(root, REQUIRED)).read()) + open(os.path.join(root, "workflow-templates", "quoted.yml"), "w").write( + "on:\n issues:\njobs:\n t:\n uses: \"O/.github/.github/workflows/issue-triage.yml@v7\"\n") + check("#748 C4: the pack's callers with test-pull-request.yml pass", pipeline_problems(root, {"callers": ["issues.yml", "test-pull-request.yml"]}, "O/.github"), []) + got = pipeline_problems(root, {"callers": ["issues.yml"]}, "O/.github") + check("#748 C4: a pack with no caller that wires the pull-request pipeline is refused, naming what it lacks", + (len(got), "no caller that wires the pull-request pipeline" in got[0], "claude-review.yml:pull_request" in got[0], "`merge_group`" in got[0]), (1, True, True, True)) + got = pipeline_problems(root, {"callers": ["issues.yml", "test-pull-request.yml", "org-pull-request.yml"]}, "O/.github") + check("#748 C4: org-pull-request.yml among the test repository's callers is refused, named", [l.split(",")[0] for l in got], + ["kinds/engine-test/pack.yml `callers` names org-pull-request.yml"]) + got = pipeline_problems(root, {"callers": ["quoted.yml", "test-pull-request.yml"]}, "O/.github") + check("#748 C4: an engine call the lane cannot move is refused, naming the file and the line", + got, ["workflow-templates/quoted.yml line 5: 'uses: \"O/.github/.github/workflows/issue-triage.yml@v7\"': an engine call the lane cannot move to the candidate " + "(`uses: O/.github/.github/workflows/.yml@`)"]) + check("#748 C4: the shipped pack passes", pipeline_problems(ROOT, test_pack(), central_of()), []) st, fake = repo_fake() moved = [] run_all(one, "o/t", "v7-next", run=fake, sleep=lambda s: None, minutes=3, central="O/.github", moved_out=moved) diff --git a/scripts/sync-repos.py b/scripts/sync-repos.py index 584c85a2..2309a179 100755 --- a/scripts/sync-repos.py +++ b/scripts/sync-repos.py @@ -27,6 +27,17 @@ names to the repositories with the property, only after every adopted one holds it, and never while that list covers a repository repos.yml does not list or excludes (it names them). + Then (#748) the required workflow in a ruleset of its own, "org: + required workflow": created first with "org: main"'s `workflows` + rule as it is (org-pull-request.yml at `v7`), its target, bypass + actors and `ref_name`, and only then that rule taken out of "org: + main", which keeps its other rules everywhere. It leaves out the + test repository (org.yml `test-repo`) by a second custom property, + org.yml `test-property` (dx-engine-test), true there alone, once + that repository has its own pull-request caller + (workflow-templates/test-pull-request.yml) on its default branch; + until then it covers it, and says so. A ruleset whose conditions + would lack `ref_name` or `repository_property` is never written. pending the repositories repos.yml lists that do not hold that property yet, one per line (#562): what org-adopt.yml adopts, read from the organisation rather than from a push, so a run @@ -156,7 +167,10 @@ not run on merge_group, and its own commit rules (commit-msg hook, commitlint, AGENTS.md, CONTRIBUTING.md) that match org.yml `conventions-contradictions`; for the organisation, a ruleset - still naming its repositories. + still naming its repositories, and (#748) a `workflows` rule + still in "org: main", "org: required workflow" covering the test + repository, an organisation ruleset with no `ref_name`, and a + required workflow at a ref other than the current major. i18n a WordPress plugin's translations in the form that merges (#452, scripts/plugin-i18n.sh): languages/*.pot and *.po with @@ -404,11 +418,34 @@ def configuration_defaults(): # says they are adopted (#562), never a list of names that each adoption # has to extend by hand. ADOPTED = str(sys.modules["org"].load().get("adopted-property") or "dx-adopted") -ORG_RULESET_NAMES = ("org: main", "org: release tag creation", "org: release tags") +# The required workflow (#748) is a ruleset of its own, "org: required +# workflow", split out of "org: main": GitHub takes one repository condition +# per org ruleset, so it leaves the test repository out by a second +# property, org.yml `test-property`, true there alone. The test repository +# keeps "org: main"'s other rules, and runs the same pipeline from its own +# caller (workflow-templates/test-pull-request.yml), which the release lane +# moves to `v7-next`. +TEST_PROPERTY = sys.modules["org"].test_property(sys.modules["org"].load()) +TEST_REPO = sys.modules["org"].test_repo(sys.modules["org"].load()) +TEST_CALLER = ".github/workflows/test-pull-request.yml" +MAIN_RULESET, REQUIRED_RULESET = "org: main", "org: required workflow" +ORG_RULESET_NAMES = (MAIN_RULESET, REQUIRED_RULESET, "org: release tag creation", "org: release tags") -def property_condition(): - return {"repository_property": {"include": [{"name": ADOPTED, "property_values": ["true"], "source": "custom"}], "exclude": []}} +def property_condition(exclude=()): + return {"repository_property": {"include": [{"name": ADOPTED, "property_values": ["true"], "source": "custom"}], "exclude": list(exclude)}} + + +def test_exclusion(): + """The repository_property exclusion of the test repository (#748).""" + return {"name": TEST_PROPERTY, "property_values": ["true"], "source": "custom"} + + +def excludes_test(ruleset): + """Whether an org ruleset leaves out the repositories with the test + property (#748).""" + cond = (ruleset.get("conditions") or {}).get("repository_property") or {} + return any(p.get("name") == TEST_PROPERTY and "true" in (p.get("property_values") or []) for p in cond.get("exclude") or []) def by_property(ruleset): @@ -418,16 +455,87 @@ def by_property(ruleset): for p in ((cond.get("repository_property") or {}).get("include") or [])) and "repository_name" not in cond -def with_property_condition(ruleset): +def complete(body): + """A ruleset's PUT or POST body, once its conditions are complete + (#748): both `ref_name` and `repository_property`, since GitHub reads a + ruleset with no `ref_name` as covering no branch. Raises naming the + ruleset, never writes one that would cover nothing.""" + cond = body.get("conditions") or {} + for key in ("ref_name", "repository_property"): + if not cond.get(key): + raise RuntimeError(f"the ruleset \"{body.get('name')}\" would have no `{key}` condition: nothing was written") + return body + + +def with_property_condition(ruleset, exclude=None): """The ruleset as the PUT takes it, its every field as the GET gave it - but its repositories, now those with the property.""" + but its repositories, now those with the property, less what it + excluded by property (`exclude`, when given, in its place).""" body = {k: v for k, v in ruleset.items() if k not in ("id", "source", "source_type", "node_id", "_links", "created_at", "updated_at", "current_user_can_bypass")} cond = {k: v for k, v in (ruleset.get("conditions") or {}).items() if k not in ("repository_name", "repository_id")} - body["conditions"] = {**cond, **property_condition()} + kept = ((ruleset.get("conditions") or {}).get("repository_property") or {}).get("exclude") or [] + body["conditions"] = {**cond, **property_condition(kept if exclude is None else exclude)} + return complete(body) + + +def workflows_rules(ruleset): + """The ruleset's `workflows` rules: the required workflow.""" + return [r for r in (ruleset or {}).get("rules") or [] if r.get("type") == "workflows"] + + +def required_ruleset(main, exclude_test): + """The body of "org: required workflow" (#748), from "org: main": its + `workflows` rule unchanged, its target, enforcement, bypass actors and + `ref_name`, over the repositories with the property, less the test + repository's when `exclude_test`. Pure.""" + return complete({"name": REQUIRED_RULESET, "target": main.get("target") or "branch", "enforcement": main.get("enforcement") or "active", + "bypass_actors": main.get("bypass_actors") or [], + "conditions": {"ref_name": (main.get("conditions") or {}).get("ref_name"), + **property_condition([test_exclusion()] if exclude_test else [])}, + "rules": workflows_rules(main)}) + + +def without_workflows(main): + """The body of "org: main" with its `workflows` rule taken out (#748): + its pull request, deletion and signed commits rules stay, on the test + repository too. Pure.""" + body = with_property_condition(main) + body["rules"] = [r for r in body.get("rules") or [] if r.get("type") != "workflows"] return body +def ref_of(workflow): + """A required workflow's ref, without refs/tags/ or refs/heads/.""" + return re.sub(r"^refs/(tags|heads)/", "", str((workflow or {}).get("ref") or "")) + + +def required_drift(org_rs, major): + """What is wrong with the required workflow's rulesets (#748), as + lines: a `workflows` rule still in "org: main", "org: required + workflow" covering the test repository, an organisation ruleset with no + `ref_name`, and a required workflow at a ref other than `v`. + Pure.""" + found = [] + for r in org_rs or []: + if r.get("name") not in ORG_RULESET_NAMES: + continue + if r.get("name") == MAIN_RULESET and workflows_rules(r): + found.append(f"the ruleset \"{MAIN_RULESET}\" still holds the `workflows` rule (the required workflow): it belongs to \"{REQUIRED_RULESET}\", " + f"which leaves the test repository {TEST_REPO} out (`sync-repos.py rulesets`)") + if r.get("name") == REQUIRED_RULESET and not excludes_test(r): + found.append(f"the ruleset \"{REQUIRED_RULESET}\" covers the test repository {TEST_REPO}: it does not exclude `{TEST_PROPERTY}=true`, so " + f"its pull requests run the pipeline at `v{major}`, not the candidate's (`sync-repos.py rulesets`, once it has {TEST_CALLER})") + if not ((r.get("conditions") or {}).get("ref_name") or {}).get("include"): + found.append(f"the ruleset \"{r['name']}\" has no `ref_name` condition: it covers no branch or tag (`sync-repos.py rulesets`)") + for rule in workflows_rules(r): + for w in (rule.get("parameters") or {}).get("workflows") or []: + if ref_of(w) != f"v{major}": + found.append(f"the ruleset \"{r['name']}\" runs `{w.get('path')}` at `{ref_of(w) or 'no ref'}`, not `v{major}`: every repository but the test " + f"repository keeps the required workflow at `v{major}`") + return found + + def org_repo_names(): """The names of the organisation's repositories that are not archived, forks and those repos.yml excludes included.""" @@ -458,9 +566,10 @@ def would_drop(moving, every): return {r["name"]: sorted(named_coverage(r, names) - adopted) for r in moving if named_coverage(r, names) - adopted} -def property_values(names): - """{repository: its value of the property}, for these names; None when - they could not be read.""" +def property_values(names, prop=None): + """{repository: its value of the property (`prop`, else the adopted + one)}, for these names; None when they could not be read.""" + prop = prop or ADOPTED got = gh("api", "--paginate", "--slurp", f"orgs/{ORG}/properties/values?per_page=100", check=False) if got.returncode != 0: return None @@ -469,7 +578,7 @@ def property_values(names): for page in rows if rows and isinstance(rows[0], list) else [rows]: for r in page: if r.get("repository_name") in names: - out[r["repository_name"]] = next((str(p.get("value")) for p in r.get("properties") or [] if p.get("property_name") == ADOPTED), None) + out[r["repository_name"]] = next((str(p.get("value")) for p in r.get("properties") or [] if p.get("property_name") == prop), None) return out @@ -517,6 +626,128 @@ def sync_rulesets(repos, every, dry): print(f" \"{r['name']}\" covers the repositories with {ADOPTED}=true") if moving and not dry: ORG_RULESETS = None + # The rulesets as read before the move: it changed their repositories + # alone, which the split writes again. + split_required(every, dry, org_rs) + + +def has_test_caller(name): + """Whether the test repository has its pull-request caller on its + default branch (#748).""" + return gh("api", f"repos/{ORG}/{name}/contents/{TEST_CALLER}", check=False).returncode == 0 + + +def sync_test_property(name, dry): + """The test property (org.yml `test-property`, a true_false custom + property of the organisation) created, and set to true on the test + repository (#748).""" + if gh("api", f"orgs/{ORG}/properties/schema/{TEST_PROPERTY}", check=False).returncode != 0: + if dry: + print(f" would create the custom property `{TEST_PROPERTY}`") + else: + gh("api", "-X", "PUT", f"orgs/{ORG}/properties/schema/{TEST_PROPERTY}", "--input", "-", input_text=json.dumps( + {"value_type": "true_false", "required": False, + "description": "The organisation's test repository (DiluxOne/.github org.yml test-repo): the required workflow's ruleset leaves it out."})) + have = property_values({name}, TEST_PROPERTY) + if have is None: + raise RuntimeError(f"the values of {TEST_PROPERTY} could not be read") + if have.get(name) == "true": + return + if dry: + print(f" would set {TEST_PROPERTY}=true on {name}") + return + gh("api", "-X", "PATCH", f"orgs/{ORG}/properties/values", "--input", "-", input_text=json.dumps( + {"repository_names": [name], "properties": [{"property_name": TEST_PROPERTY, "value": "true"}]})) + print(f" {TEST_PROPERTY}=true on {name}") + + +REQUIRED_PIPELINE = "required pipeline" + + +def required_pipeline_ruleset(): + """The test repository's own ruleset "required pipeline" (#748): the + `pipeline` check of its pull-request caller required on its default + branch, the gate the required workflow gives everywhere else (a required + workflow requires its whole run to succeed; `pipeline` is that run as one + check, on the pull request and in the merge queue). Pure.""" + return {"name": REQUIRED_PIPELINE, "target": "branch", "enforcement": "active", "bypass_actors": [], + "conditions": {"ref_name": {"include": ["~DEFAULT_BRANCH"], "exclude": []}}, + "rules": [{"type": "required_status_checks", "parameters": { + "strict_required_status_checks_policy": False, "do_not_enforce_on_create": False, + "required_status_checks": [{"context": "pipeline"}]}}]} + + +def sync_required_pipeline(name, dry): + """"required pipeline" written in the test repository when it has none + (#748); one that exists is left as it is.""" + full = f"{ORG}/{name}" + if any(r.get("name") == REQUIRED_PIPELINE for r in rulesets(full)): + return + if dry: + print(f" would create the ruleset \"{REQUIRED_PIPELINE}\" in {name}, requiring the `pipeline` check") + return + gh("api", "-X", "POST", f"repos/{full}/rulesets", "--input", "-", input_text=json.dumps(required_pipeline_ruleset())) + print(f" \"{REQUIRED_PIPELINE}\" in {name} requires the `pipeline` check") + + +def split_required(every, dry, org_rs=None): + """The required workflow in a ruleset of its own (#748): "org: required + workflow" created first, with "org: main"'s `workflows` rule as it is, + and only then that rule taken out of "org: main", so no repository is + ever without it. It leaves out the test repository (the test property, + set on it first) once that repository has its pull-request caller on its + default branch, and its own ruleset "required pipeline" requires that + caller's `pipeline` check; until then it covers it, and says so, for + otherwise its pull requests would run no pipeline.""" + global ORG_RULESETS + org_rs = org_rulesets() or [] if org_rs is None else org_rs + main = next((r for r in org_rs if r.get("name") == MAIN_RULESET), None) + required = next((r for r in org_rs if r.get("name") == REQUIRED_RULESET), None) + rules = workflows_rules(main) + if required is None and not rules: + return + test = TEST_REPO if TEST_REPO in {r["name"] for r in every} else None + ready = False + if test: + sync_test_property(test, dry) + ready = has_test_caller(test) + if ready: + # Its gate first, then the required workflow leaves it. + sync_required_pipeline(test, dry) + else: + print(f" {test} has no {TEST_CALLER} on its default branch yet: \"{REQUIRED_RULESET}\" still covers it, " + "so its pull requests run a pipeline (`sync-repos.py callers` writes it)") + changed = False + if required is None: + body = required_ruleset(main, ready) + if dry: + print(f" would create \"{REQUIRED_RULESET}\" with the `workflows` rule of \"{MAIN_RULESET}\"" + (f", leaving out {test}" if ready else "")) + else: + gh("api", "-X", "POST", f"orgs/{ORG}/rulesets", "--input", "-", input_text=json.dumps(body)) + print(f" \"{REQUIRED_RULESET}\" created" + (f", leaving out {test}" if ready else "")) + changed = True + carries = bool(body["rules"]) + else: + carries = bool(workflows_rules(required)) + if ready and not excludes_test(required): + if dry: + print(f" would make \"{REQUIRED_RULESET}\" leave out {test} ({TEST_PROPERTY}=true)") + else: + exclude = [p for p in ((required.get("conditions") or {}).get("repository_property") or {}).get("exclude") or []] + [test_exclusion()] + gh("api", "-X", "PUT", f"orgs/{ORG}/rulesets/{required['id']}", "--input", "-", input_text=json.dumps(with_property_condition(required, exclude))) + print(f" \"{REQUIRED_RULESET}\" leaves out {test}") + changed = True + if rules and carries: + if dry: + print(f" would take the `workflows` rule out of \"{MAIN_RULESET}\"") + else: + gh("api", "-X", "PUT", f"orgs/{ORG}/rulesets/{main['id']}", "--input", "-", input_text=json.dumps(without_workflows(main))) + print(f" \"{MAIN_RULESET}\" keeps its other rules; the required workflow is \"{REQUIRED_RULESET}\"'s") + changed = True + elif rules: + print(f" \"{REQUIRED_RULESET}\" has no `workflows` rule: \"{MAIN_RULESET}\" keeps it") + if changed: + ORG_RULESETS = None def not_adopted(every): @@ -1007,14 +1238,14 @@ def sync_callers(repo, conf, dry): behind = old_versions(added.values(), current) added = {p: to_major(t, behind, current) for p, t in added.items()} if behind else added parsed_moved = {p: (t, parse_yaml(t)) for p, t in moved.items()} - shared = shared_callers([(n, d, caller_for(dict(d, _name=n), parsed_moved, False)) for n, (_, d) in templates(major).items()]) + shared = shared_callers([(n, d, caller_for(dict(d, _name=n), parsed_moved, False)) for n, (_, d) in templates_for(kind, conf, major).items()]) open_prs = json.loads(gh("pr", "list", "--repo", full, "--state", "open", "--limit", "500", "--json", "headRefName").stdout) if any(p["headRefName"].endswith("-org-workflow-callers") for p in open_prs): print(" a pull request for the workflow callers is already open; skipped") return owned = (conf.get("callers") or {}).get("repository-owned") or {} wanted, notes, stuck = {}, list(held), set() - for name, path, found in callers_drift(moved, conf, major=major, default=branch): + for name, path, found in callers_drift(moved, conf, major=major, default=branch, kind=kind): if not path: if f".github/workflows/{name}" not in added: notes.append(f"no caller of workflow-templates/{name}: add it from the template by hand (docs/adopting.md)") @@ -1042,7 +1273,7 @@ def sync_callers(repo, conf, dry): stuck.add(path) continue new = granted - left = [f for n, p, f in callers_drift({**moved, path: new}, conf, major=major, default=branch) if p == path and n == name] if new else [] + left = [f for n, p, f in callers_drift({**moved, path: new}, conf, major=major, default=branch, kind=kind) if p == path and n == name] if new else [] if new is None: notes.append(f"{path}: no job of the repository gives {', '.join(missing)}; bring it in step by hand") stuck.add(path) @@ -1064,7 +1295,7 @@ def sync_callers(repo, conf, dry): wanted[path] = text # A caller in step but for a template's placeholder (#562): the value # repos.yml gives it, its entry's or the organisation's default. - for tname, (ttext, data) in templates(major).items(): + for tname, (ttext, data) in templates_for(kind, conf, major).items(): path = caller_for(dict(data, _name=tname), parsed_moved, False) if path is None or path in stuck: continue @@ -1081,7 +1312,7 @@ def sync_callers(repo, conf, dry): print(f" {note}") # Nothing still holding a template's placeholder is written (#671): that # file stays out, the others go in. - refused = refuse_placeholders(full, wanted, added, parsed_moved, owned, values, major) + refused = refuse_placeholders(full, wanted, added, parsed_moved, owned, values, major, kind, conf) if old and refused: print(f" the move to @v{current} waits: {', '.join(refused)} must be brought in step first") return @@ -1096,7 +1327,7 @@ def sync_callers(repo, conf, dry): sys.stdout.writelines(" " + l for l in difflib.unified_diff(files.get(path, "").splitlines(True), new.splitlines(True), path, path)) return default = (repo.get("defaultBranchRef") or {}).get("name") or "main" - templates_named = ", ".join(sorted({f"workflow-templates/{n}" for n, p, _ in callers_drift(moved, conf, major=major, default=branch) if p in wanted})) or "the templates" + templates_named = ", ".join(sorted({f"workflow-templates/{n}" for n, p, _ in callers_drift(moved, conf, major=major, default=branch, kind=kind) if p in wanted})) or "the templates" to = f"@v{current}" move = (f" It also moves every call of the shared workflows from {', '.join('@v' + str(v) for v in old)} to {to}, the repository's own jobs too: " f"that major asks a repository for nothing but its callers (docs/migrating.md)." if old else "") @@ -1116,12 +1347,12 @@ def sync_callers(repo, conf, dry): assisted_form(CALLERS_KEEP), mode="callers") -def refuse_placeholders(full, wanted, added, parsed, owned, values, major): +def refuse_placeholders(full, wanted, added, parsed, owned, values, major, kind=None, conf=None): """Takes out of `wanted` each caller placeholder_problems() refuses, saying the repository, the path and why (#671): a file in `added` is new, from its template; any other, the caller (in `parsed`) of the templates it calls. The paths it took out, sorted.""" - tpls = templates(major) + tpls = templates_for(kind, conf, major) if conf is not None else templates(major) of = {} for tname, (_, data) in tpls.items(): path = caller_for(dict(data, _name=tname), parsed, False) @@ -1236,6 +1467,32 @@ def templates(major=None): return found +def kinds_only(conf, root=None): + """{template: the kinds whose packs list it}, for the templates a pack's + `callers` lists that `callers: every-repository` does not (#748): those + belong to the repositories of those kinds alone (test-pull-request.yml to + the test repository), and are never compared with, nor written to, any + other.""" + every = set((conf.get("callers") or {}).get("every-repository") or []) + root = root or CENTRAL + out = {} + for kind in sorted(os.listdir(os.path.join(root, "kinds"))): + for name in pack_of(kind, root).get("callers") or []: + if name not in every: + out.setdefault(str(name), set()).add(kind) + return out + + +def templates_for(kind, conf, major=None): + """templates(major), less those of other kinds alone (kinds_only()). One + of this kind alone is marked `_own`: its caller is the file of its name, + never another that calls some of the same workflows, and no other + template's caller is that file (`_skip`, caller_for()).""" + only = kinds_only(conf) + return {n: (t, dict(d, _own=True) if n in only else dict(d, _skip=set(only))) for n, (t, d) in templates(major).items() + if n not in only or kind in only[n]} + + def split_uses(uses, central=False): """(shared workflow, ref) for a call of a shared workflow, else None. In DiluxOne/.github itself a local call (./.github/workflows/x.yml) is the @@ -1262,10 +1519,13 @@ def shared_calls(data, central=False): def caller_for(template, files, central=False): """The repository's workflow file that calls this template's shared workflows: the one calling most of them, the template's own name first - on a tie. None when no file calls any.""" + on a tie. None when no file calls any. A template of some kinds alone + (`_own`, templates_for()) has only the file of its own name (#748).""" + if template.get("_own"): + return next((p for p in sorted(files) if os.path.basename(p) == template.get("_name")), None) wanted = shared_calls(template) best = None - for path in sorted(files): + for path in sorted(p for p in files if os.path.basename(p) not in template.get("_skip", ())): overlap = len(shared_calls(files[path][1], central) & wanted) if overlap: key = (overlap, os.path.basename(path) == template.get("_name")) @@ -1328,16 +1588,18 @@ def shared_callers(found_for): return {p for p in paths if paths.count(p) > 1} -def callers_drift(files, conf, central=False, major=None, default="main"): +def callers_drift(files, conf, central=False, major=None, default="main", kind=None): """What differs in a repository's callers from the templates that apply to it, as [(template, caller path or None, [phrases])], only those that differ. `files` is {path: text}; major, as templates() takes it; default, - the repository's default branch (a template's `$default-branch`).""" + the repository's default branch (a template's `$default-branch`); kind, + the repository's (a template of a kind alone applies to that kind only, + kinds_only()).""" spec = conf.get("callers") or {} every, owned = set(spec.get("every-repository") or []), spec.get("repository-owned") or {} parsed = {p: (t, parse_yaml(t)) for p, t in files.items()} out = [] - found_for = [(name, data, caller_for(dict(data, _name=name), parsed, central)) for name, (_, data) in templates(major).items()] + found_for = [(name, data, caller_for(dict(data, _name=name), parsed, central)) for name, (_, data) in templates_for(kind, conf, major).items()] shared = shared_callers(found_for) for name, data, path in found_for: data = dict(data, _name=name) @@ -1500,12 +1762,12 @@ def placeholder_problems(path, text, template_text, owned, values, whole): return reasons -def placeholder_lines(files, conf, name, central=False, major=None): +def placeholder_lines(files, conf, name, central=False, major=None, kind=None): """The drift lines of the callers that hold a template's placeholder.""" owned, values = (conf.get("callers") or {}).get("repository-owned") or {}, repos_conf.caller_values(conf, name) parsed = {p: (t, parse_yaml(t)) for p, t in files.items()} out = [] - for tname, (ttext, data) in templates(major).items(): + for tname, (ttext, data) in templates_for(kind, conf, major).items(): path = caller_for(dict(data, _name=tname), parsed, central) if path is None: continue @@ -1556,7 +1818,8 @@ def new_callers(files, conf, kind, default="main", major=None, name=None, notes= """{path: text} of the callers a repository lacks, written whole from their templates (#552): those of `every-repository` (the triage and the autopilot start from issues.yml) and those that call a workflow of its - kind's battery, each at .github/workflows/