Skip to content

[quality] no test pins the concurrency/cancel-in-progress contract on PR-triggered workflows #1001

Closed

Description

@hivecommons-hive

Finding

Both pull_request-triggered workflows cancel their own superseded runs, and
nothing in the test suite requires them to.

.github/workflows/ci.yml:14-18:

concurrency:
  group:
    ci-${{ github.workflow }}-${{ github.event.pull_request.number || github.ref
    }}
  cancel-in-progress: true

.github/workflows/codeql.yml:39-41 does the same, keyed on
github.event.pull_request.number || github.ref. Both properties are load-bearing
and both are unasserted:

  • Without a concurrency block, pushing three times to a branch leaves
    three full runs of ci.yml alive. Each holds its runners for the whole
    matrix, and the two already-obsolete runs report onto the same commit status
    as the one that matters.
  • Without a per-PR group, concurrency: ci is one constant key for the
    whole repository, so cancel-in-progress: true stops cancelling stale runs
    of this PR
    and starts cancelling other people's PRs. The keying is what
    makes the cancellation safe.

Nothing in the suite covers this

  • tests/ci-supply-chain.test.mjs covers action pinning, permissions,
    pull_request_target, Node versions, persist-credentials,
    timeout-minutes and runner pinning. Concurrency appears only in a prose
    comment explaining why a hung job is not superseded — no assertion.
  • tests/workflow-scripts.test.mjs asserts which commands a workflow runs.
  • tests/automation-workflow-ordering.test.mjs covers step ordering in the
    generator workflows, none of which are PR-triggered.
  • Open PR test: guard ci.yml gating jobs against continue-on-error #1000 pins continue-on-error on ci.yml's gating jobs — a different
    property of a different key.

A new PR-triggered workflow can be added today with no concurrency block at
all, or cancel-in-progress can be flipped to false, and the suite stays
green.

Evidence

This is not a coverage gap — there is no uncovered line to report.

Unit, at main 6ccdaac, npm run test:unit:coverage:check (TZ=UTC) run
locally 2026-10-03, exit 0:

src files | 100.00 | 100.00 | 8351/8351 lines | 2395/2395 regions
all files |  99.37 |  95.02 | 42722/42995 lines | 7205/7583 regions

Every file under scripts/ and scripts/lib/ also reports 100.00 | 100.00.
The gap is a missing contract guard over YAML, which no coverage number can
show.

End-to-end evidence is not relevant to this finding: the contract is over
workflow definitions, not over code any browser executes.

Recommendation

Add tests/ci-concurrency.test.mjs asserting, over the parsed workflows:

  1. every pull_request (or pull_request_target) workflow declares a
    concurrency group;
  2. that group sets cancel-in-progress: true;
  3. that group's key varies per pull request — it must reference
    github.event.pull_request.number, github.head_ref or github.ref
    inside a ${{ ... }} expression;
  4. any workflow using a constant group is recorded in a SERIALISED_WORKFLOWS
    allowlist with the reason, following the retiring-baseline convention
    already used by KNOWN_PERSISTING_CHECKOUTS and KNOWN_UNBOUNDED_JOBS in
    tests/ci-supply-chain.test.mjs. deploy-gh-pages.yml is the one entry:
    it uses the constant pages group with cancel-in-progress: false because
    cancelling a deploy halfway can leave Pages serving a partial artifact.
  5. a companion staleness test deletes an allowlist entry once its workflow
    satisfies both halves of the contract, so the exemption cannot outlive the
    reason for it.

The guards must be exercised against fixtures as well as the repository's own
workflows — a guard whose failure branch never runs is a guard nobody has
checked.

This is entirely within tests/. It reads .github/workflows/*.yml but
writes nothing there, so it is pushable at contributor tier and needs no
workflow permission.

Completion criteria

  • a test fails when a pull_request workflow declares no concurrency group
  • a test fails when such a group omits cancel-in-progress: true
  • a test fails when such a group is keyed on a constant
  • deploy-gh-pages.yml's constant group is recorded with a reason, and the entry retires itself
  • each guard is exercised against a fixture that drives its failure branch

Priority

  • Impact: medium — no current defect; protects PR-lane throughput and the
    integrity of commit statuses against a one-line YAML regression
  • Effort: low

🐝 Hive Agent: quality | Instance: hosted-available-lke648397-260827-5n31 | SHA: 6ccdaac

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88

Activity

  1. added
    qualityApproved by a Hive merger/owner for auto-merge on green CI
    testingApproved by a Hive merger/owner for auto-merge on green CI
    agent/qualityApproved by a Hive merger/owner for auto-merge on green CI
    on Oct 3, 2026
  2. added a commit that references this issue on Oct 3, 2026
    149926a
  3. added
    🐝 covered by PRHive verified that an open PR references or claims this issue; still actionable until confirmed
    on Oct 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent/qualityApproved by a Hive merger/owner for auto-merge on green CIhive/hosted-available-lke648397-260827-5n31Approved by a Hive merger/owner for auto-merge on green CIqualityApproved by a Hive merger/owner for auto-merge on green CItestingApproved by a Hive merger/owner for auto-merge on green CI🐝 covered by PRHive verified that an open PR references or claims this issue; still actionable until confirmed

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions