Skip to content

test(ci-gates): assert every validate:*/check:* gate is actually run in CI - #742

Closed
hivecommons-hive[bot] wants to merge 1 commit into
mainfrom
quality/test-ci-gate-wiring
Closed

hivecommons-hive[bot] wants to merge 1 commit into
mainfrom
quality/test-ci-gate-wiring

Conversation

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Test Improvement

Adds five guards to tests/workflow-scripts.test.mjs. No production code, no
workflow file is touched.

That suite already checks that package.json scripts, the files under
scripts/, the linter configs those scripts pass, and the npm run calls
inside .github/workflows/ all name something real. It never checks that the
repository's gates — the validate:* and check:* scripts — are invoked by
a workflow at all. A gate nobody runs is inert: it passes when a contributor
runs it by hand, the unit suite stays green, and the regression it exists to
catch reaches main.

New tests:

  • every validate:*/check:* gate is run by a workflow or recorded as exempt —
    the guard itself, resolving npm run targets across every workflow's run:
    steps.
  • every gate this suite recognises is a real package.json script — pins the
    validate:|check: filter to evidence so the guard above cannot pass
    vacuously if the filter stops matching.
  • every recorded CI exemption still names a defined script
  • every recorded CI exemption carries a reason
  • no recorded CI exemption names a gate a workflow already runs

The exemption map GATES_NOT_RUN_BY_CI is seeded with the two gates CI does not
run today, each with its reason on the record: check:links resolves outbound
URLs against the live internet, and check:community-group-links calls the
GitHub API, needs GH_TOKEN, and rewrites data/community-groups.json (a
refresh job rather than a pass/fail check). The three guards over the map keep
it from rotting into a standing untruth.

Verification

  • TZ=UTC node --test tests/workflow-scripts.test.mjs — 13 pass, 0 fail.
  • Mutation-checked: removing npm run validate:awards from all three workflows
    that invoke it (ci.yml, deploy-gh-pages.yml, import-architectures.yml)
    makes the new guard fail naming validate:awards. Removing it from ci.yml
    alone correctly does not fail, since the gate still runs elsewhere.
  • npx prettier --check tests/workflow-scripts.test.mjs — clean.
  • TZ=UTC node tests/tools/coverage-report.mjs --check 97 --check-source 99 —
    passes; src files stay at 100.00% lines / 97.95% regions, and this file's
    region coverage rises from 87.23% to 88.79%. Every added line is executed.

Scope

Touches only tests/workflow-scripts.test.mjs, which no other open PR modifies.
This is deliberately not a coverage-gap fix: every source file currently below
100% region coverage is already claimed by an open hold-gated PR.

Heads-up for review ordering: open PR #684 adds a validate:projects-born
script with no CI step. If it merges before this, the new guard will fail naming
validate:projects-born until .github/workflows/ci.yml gains a step for it.
That is the guard working as intended, but landing that step needs a human —
this lane cannot push changes under .github/workflows/.

Related Issue

Closes #741


Filed by quality agent (hold-gated mode). Human review required.

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88

tests/workflow-scripts.test.mjs guards that package.json scripts, the files
under scripts/, the linter configs they reference, and the npm run calls in
.github/workflows/ all name something real, but never asserts that the
repository's gates are actually invoked by a workflow. A gate nobody runs is
inert: it passes when run by hand, the unit suite stays green, and the
breakage it was written to catch reaches main.

Adds a guard over the validate:* and check:* scripts, with an explicit
exemption map recording the two gates CI deliberately does not run
(check:links reaches the live internet, check:community-group-links needs
GH_TOKEN and rewrites data). The map is itself guarded against rot: entries
must name a defined script, carry a reason, and not name a gate a workflow
already runs.

Signed-off-by: hivecommons-hive[bot] <hivecommons-hive@hive.kubestellar.io>
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

Important

Held for human review by the hive's ACMM level gate.

This PR was opened by the "quality" agent while Hive policy required a human checkpoint for that agent. Non-outreach agents are held at ACMM L3–L5; the outreach agent is always held because it publishes project-facing communication.

Hive will automatically remove the hold label once current policy no longer requires a level hold for "quality". If this is an outreach PR, a human must review it and remove the label.

@hivecommons-hive hivecommons-hive Bot added quality Approved by a Hive merger/owner for auto-merge on green CI testing Approved by a Hive merger/owner for auto-merge on green CI agent/quality Approved by a Hive merger/owner for auto-merge on green CI hive/hosted-available-lke648397-260827-5n31 Approved by a Hive merger/owner for auto-merge on green CI labels Sep 27, 2026
@mrbobbytables

Copy link
Copy Markdown
Member

Superseded by #750, which consolidates this and 14 other test-only coverage PRs into a single reviewable change (commit cherry-picked unmodified, authorship and DCO preserved).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agent/quality Approved by a Hive merger/owner for auto-merge on green CI hive/hosted-available-lke648397-260827-5n31 Approved by a Hive merger/owner for auto-merge on green CI hold quality Approved by a Hive merger/owner for auto-merge on green CI testing Approved by a Hive merger/owner for auto-merge on green CI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[quality] Nothing asserts that validate:*/check:* gates are actually run by CI

1 participant