[quality] Import PRs opened with GITHUB_TOKEN never trigger CI — #681 has no Validate repository run #714

Finding

Pull requests opened by .github/workflows/import-architectures.yml are never
validated by CI. Open PR #681 (chore: import latest reference architectures,
author app/github-actions, head automation/import-architectures) has exactly one
check:

$ gh pr checks 681
DCO   pass   https://probot.github.io/apps/dco/

$ gh pr view 681 --json mergeable,mergeStateStatus
MERGEABLE   BLOCKED

Validate repository (.github/workflows/ci.yml) never ran on it, and never will.
The cause is a documented GitHub behaviour rather than a misconfiguration of
ci.yml: events raised using the automatic GITHUB_TOKEN do not trigger further
workflow runs, precisely to prevent recursive workflow loops. The import job calls

      - uses: peter-evans/create-pull-request@5f6978faf089d4d20b00c7766989d076bb2fc7f1 # v8.1.1
        with:
          branch: automation/import-architectures

with no token: input, so the action falls back to github.token. Every PR it opens
is therefore invisible to on: pull_request. DCO shows up only because it is a
separate GitHub App, not an Actions workflow.

The same applies to any other workflow in this repository that opens a PR with the
default token.

Why the job's inline validation is not a substitute

The import job does run validators before opening the PR (test:unit,
validate:architectures, validate:architecture-assets, validate:awards,
build). That is not the same as CI on the PR:

• It is a strict subset. It runs npm run test:unit, not npm test, so the entire
  check group — check:format, check:spelling, check:markdown,
  check:links — never runs, and neither does ci.yml's separate lint job. This
  is not hypothetical: chore: import latest reference architectures #681 carries 16 files that fail prettier --check while the
  import job reported success (tracked separately in [quality] Import job never runs check:format, so every architecture import PR carries 16 unformatted files #713).
• It validates the importer's working tree, not the merge result. Anything that
  landed on main between the import run and the eventual merge is never
  revalidated against the imported data.
• npm run test:unit:coverage:check, the coverage gate ci.yml enforces, is not run
  at all.

So the one category of pull request that changes bulk generated data with no human
authoring it is also the one category that receives the least verification.

Consequence

mergeStateStatus: BLOCKED on #681 is the visible symptom: required checks can never
report, so the PR cannot satisfy branch protection and sits indefinitely. The likely
resolutions are both bad — an administrator merges it unvalidated, or the nightly
import silently stops producing mergeable output.

Recommendation

Open the PR with a credential that is not the automatic GITHUB_TOKEN, so that
on: pull_request fires normally. Using a GitHub App installation token keeps this
secret-light and avoids a long-lived PAT. In
.github/workflows/import-architectures.yml, replace this step:

      - uses: peter-evans/create-pull-request@5f6978faf089d4d20b00c7766989d076bb2fc7f1 # v8.1.1
        with:
          branch: automation/import-architectures
          delete-branch: true
          commit-message: 'chore: import reference architectures'
          title: 'chore: import latest reference architectures'
          body: |
            Automated metrics refresh and architecture import.
            Automated import from https://github.com/cncf/architecture.

            The production build and architecture validation passed.
          labels: automated
          signoff: true

with:

      # A PR opened with the automatic GITHUB_TOKEN does not trigger `on:
      # pull_request`, so ci.yml never runs on the import PR and required checks
      # can never report (it sits at mergeStateStatus BLOCKED). Opening it with an
      # App installation token makes the PR trigger CI like any other.
      - name: Mint an App token for the pull request
        id: app-token
        uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
        with:
          app-id: ${{ vars.AUTOMATION_APP_ID }}
          private-key: ${{ secrets.AUTOMATION_APP_PRIVATE_KEY }}
      - uses: peter-evans/create-pull-request@5f6978faf089d4d20b00c7766989d076bb2fc7f1 # v8.1.1
        with:
          token: ${{ steps.app-token.outputs.token }}
          branch: automation/import-architectures
          delete-branch: true
          commit-message: 'chore: import reference architectures'
          title: 'chore: import latest reference architectures'
          body: |
            Automated metrics refresh and architecture import.
            Automated import from https://github.com/cncf/architecture.

            The production build and architecture validation passed.
          labels: automated
          signoff: true

This requires a repository or organization variable AUTOMATION_APP_ID and secret
AUTOMATION_APP_PRIVATE_KEY for a GitHub App installed on this repository with
Contents: write and Pull requests: write. A classic PAT in a secret works the same
way (token: ${{ secrets.AUTOMATION_PAT }}) if an App is not available, at the cost
of a long-lived credential.

If neither credential can be provisioned, the fallback is to make the inline
validation actually equivalent: change - run: npm run test:unit to
- run: npm test and add - run: npm run test:unit:coverage:check. That closes the
verification gap without closing the trigger gap — the PR still shows no checks and
still needs an administrator to merge — so it is strictly second best.

Scope note

This is the root cause; #713 is one concrete consequence of it (unformatted output
reaching an unvalidated PR) and has its own narrower fix, which is worth applying
regardless of what happens here because it makes the import job self-checking.

This needs a human or an ISSUES_PRS_MERGE agent to land

The change is entirely inside .github/workflows/, and it additionally requires
provisioning a repository secret and variable, which no agent can do. An agent at the
contributor token tier holds no workflows permission, so a push carrying this
diff is rejected by GitHub before review. No pull request accompanies this
issue — a hard ceiling, not a judgement call. The replacement text above is given
in full so applying it is mechanical once the credential exists. No part of this fix
lives outside .github/workflows/, so nothing is being PR'd separately.

Priority

• Impact: high — the repository's bulk-data automation merges, or stalls, with no CI verdict at all; chore: import latest reference architectures #681 is blocked on it right now.
• Effort: medium — the workflow edit is small, but it depends on provisioning an App or PAT credential.