Skip to content

test(ci-supply-chain): guard that every CI job bounds its runtime with timeout-minutes - #744

Closed
hivecommons-hive[bot] wants to merge 1 commit into
mainfrom
quality/test-workflow-job-timeouts
Closed

hivecommons-hive[bot] wants to merge 1 commit into
mainfrom
quality/test-workflow-job-timeouts

Conversation

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Test Improvement

Adds a timeout-minutes contract to tests/ci-supply-chain.test.mjs, alongside the action-pinning, permissions and persist-credentials contracts already in that file. No other file is touched.

No job in .github/workflows/ declares timeout-minutes, so all eleven inherit GitHub's 6-hour default:

$ grep -c timeout-minutes .github/workflows/*.yml
ci.yml:0
codeql.yml:0
create-milestones.yml:0
deploy-gh-pages.yml:0
import-architectures.yml:0
pr-queue-hygiene.yml:0
refresh-community-people.yml:0
refresh-radar-reports.yml:0

That matters because the work these jobs do is network-bound. ci.yml's validate job runs npm run check, which includes check:links — markdown-link-check against every link in every root Markdown file — plus npx-resolved check:spelling and check:markdown. A black-holed host makes those commands hang rather than fail, and the job then holds a runner for six hours on every affected pull request. The scheduled workflows call the GitHub API and would stall the same way. Only deploy-gh-pages.yml declares a concurrency group, so a stuck run in the other seven is not superseded by the next one.

What the three tests assert

  • every workflow job bounds its runtime with timeout-minutes — every job that can take a timeout declares one. Jobs that delegate to a reusable workflow via uses: are excluded, because timeout-minutes is not accepted on those.
  • every declared timeout-minutes is a positive number below the 6-hour default — values are checked through a new timeoutProblem() helper, which rejects non-numbers (a quoted YAML scalar parses as a string), non-integers and non-positives (a fraction is silently floored by the runner), and anything at or above 360 (which restates the default the setting exists to replace). A companion unit test exercises timeoutProblem() directly against usable values, quoted scalars, fractions, zero, negatives and 360+, so the rule is tested rather than merely applied to whatever the workflows happen to contain today.
  • the timeout-minutes baseline retires itself — fails as soon as a baselined job gains a timeout, or names a workflow or job that no longer exists.

Why the suite is green today

KNOWN_UNBOUNDED_JOBS records the eleven currently unbounded jobs. It is a retiring baseline, not an allowance — the same shape as the existing KNOWN_PERSISTING_CHECKOUTS at tests/ci-supply-chain.test.mjs:186-215. Bounding a job turns the suite red until its entry is removed in the same change, so the fix cannot land while quietly leaving the exception behind, and a bounded job cannot later regress unnoticed.

Both failure directions were verified locally: adding timeout-minutes: 30 to ci.yml's validate job fails the retirement test with ci.yml: validate now declares timeout-minutes; remove it from KNOWN_UNBOUNDED_JOBS so the gap cannot reopen, and dropping ci.yml: lint from the baseline fails the main test.

Verification

  • node --test tests/ci-supply-chain.test.mjs — 13 pass, 0 fail
  • node tests/tools/coverage-report.mjs --check 97 --check-source 99 — passes; src files stays at 100.00 lines / 97.95 regions, all files 99.08 / 93.35, and this file rises from 89.24 to 91.23 line coverage
  • npx prettier --check tests/ci-supply-chain.test.mjs — clean

Related Issue

Refs #743 (needs-human: this PR adds only the guard; the fix itself edits .github/workflows/**, which the App token used for automated pull requests cannot push — a push carrying that diff is rejected server-side because the token is minted without the Workflows permission. A maintainer, or an agent whose token carries the workflows scope, has to apply the YAML and empty KNOWN_UNBOUNDED_JOBS. Issue #743 carries the exact replacement text for all ten job blocks and stays open until then.)


Filed by quality agent (hold-gated mode). Human review required.

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88

No job in .github/workflows/ declares timeout-minutes, so all eleven
inherit GitHub's 6-hour default. The commands these workflows run are
network-bound -- markdown-link-check walks every link in every root
Markdown file, and the scheduled jobs call the GitHub API -- so an
unreachable host makes a step hang rather than fail and the job holds a
runner for six hours. Only deploy-gh-pages declares a concurrency group,
so the next run does not supersede a stuck one either.

Add three contracts to tests/ci-supply-chain.test.mjs, alongside the
existing action-pinning, permissions and persist-credentials contracts:

- every boundable job declares timeout-minutes, with the eleven current
  gaps recorded in KNOWN_UNBOUNDED_JOBS
- every declared value is a positive whole number below 360, checked
  through timeoutProblem(), which is exercised directly against usable
  values, quoted scalars, fractions, zero, negatives and 360+
- the baseline retires itself, failing the moment a listed job gains a
  timeout without its entry being removed

KNOWN_UNBOUNDED_JOBS is a retiring baseline, not an allowance: the suite
is green today and turns red the moment a job is bounded, which couples
the fix to emptying the list. Jobs that delegate to a reusable workflow
are excluded, since timeout-minutes is not accepted there.

Signed-off-by: hivecommons-hive[bot] <hivecommons-hive@hive.kubestellar.io>
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

Important

Held for human review by the hive's ACMM level gate.

This PR was opened by the "quality" agent while Hive policy required a human checkpoint for that agent. Non-outreach agents are held at ACMM L3–L5; the outreach agent is always held because it publishes project-facing communication.

Hive will automatically remove the hold label once current policy no longer requires a level hold for "quality". If this is an outreach PR, a human must review it and remove the label.

@mrbobbytables

Copy link
Copy Markdown
Member

Superseded by #750, which consolidates this and 14 other test-only coverage PRs into a single reviewable change (commit cherry-picked unmodified, authorship and DCO preserved).

@mrbobbytables
mrbobbytables deleted the quality/test-workflow-job-timeouts branch September 28, 2026 14:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant