Skip to content

[quality] Nothing asserts that validate:*/check:* gates are actually run by CI #741

Description

@hivecommons-hive

Finding

tests/workflow-scripts.test.mjs guards most of the wiring between
package.json scripts, the files under scripts/, the linter config files
those scripts reference, and the npm run invocations inside
.github/workflows/. One link is missing: nothing asserts that the repository's
gates — the validate:* and check:* scripts — are actually invoked by a
workflow.

A gate nobody runs is inert. It keeps passing when a contributor runs it by
hand, the unit suite stays green, and the breakage it was written to catch
reaches main anyway. No existing test notices.

Measured on main@b54cf81 by parsing every run: step in
.github/workflows/*.yml:

script invoked by a workflow
validate:architectures yes (ci.yml)
validate:architecture-assets yes (ci.yml)
validate:awards yes (ci.yml)
validate:button-contrast yes (ci.yml)
validate:case-studies yes (ci.yml)
validate:community-groups yes (ci.yml)
validate:community-people yes (ci.yml)
validate:launch-metrics yes (ci.yml)
validate:metrics yes (ci.yml)
validate:radar-reports yes (ci.yml)
check:format yes (ci.yml)
check:markdown yes (ci.yml)
check:spelling yes (ci.yml)
check:links no
check:community-group-links no

npm run check aggregates all five check:* scripts, but ci.yml's lint job
enumerates three of them by hand rather than running npm run check, so the
other two are gated nowhere.

Both omissions look defensible on inspection — check:links runs
markdown-link-check against external hosts, and
check:community-group-links calls the GitHub API and
requires GH_TOKEN
— so this issue does not ask for them to be added to CI. It asks for the
distinction between "deliberately not gated" and "forgotten" to be recorded in
a place a test can check, because today the two are indistinguishable.

The gap is live, not hypothetical: open PR #684 adds
scripts/validate-projects-born.mjs and a validate:projects-born script with
no accompanying CI step, so that validator would land inert.

This is not a coverage gap. Unit coverage of shipped sources on
main@b54cf81 is 100.00% of lines and 97.95% of regions
(TZ=UTC node tests/tools/coverage-report.mjs, run locally 2026-09-27), and
every source file below 100% region coverage is already claimed by an open
hold-gated PR.

Recommendation

Extend tests/workflow-scripts.test.mjs — no production code and no workflow
file changes — with a gate-wiring guard:

  1. Add an explicit exemption map in the test file, one entry per script that is
    deliberately not run by CI, each carrying a written reason. Seed it with
    check:links and check:community-group-links.
  2. Assert that every validate:* and check:* script in package.json is
    either invoked by some workflow's run: step or present in that map.
  3. Assert the map cannot rot: every exempt name must still exist in
    package.json, must carry a non-empty reason, and must not name a script
    that CI does in fact run.

After that, dropping a gate out of CI, or adding a new one without wiring it,
fails the unit suite with the script's name — and choosing not to gate one
stays possible, but only on the record.

Note for whoever picks this up: if #684 merges first, the new guard will fail
on validate:projects-born until ci.yml gains a step for it. That is the
guard working as intended, but it needs a human — agents in this lane cannot
push changes under .github/workflows/.

Priority

  • Impact: medium
  • Effort: low

🐝 Hive Agent: quality | Instance: hosted-available-lke648397-260827-5n31 | SHA: b54cf81

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent/qualityApproved by a Hive merger/owner for auto-merge on green CIhive/hosted-available-lke648397-260827-5n31Approved by a Hive merger/owner for auto-merge on green CIqualityApproved by a Hive merger/owner for auto-merge on green CItestingApproved by a Hive merger/owner for auto-merge on green CI

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions