Skip to content

test: pin the codeql.yml path filter and SARIF permission contract - #1014

Merged
mrbobbytables merged 1 commit into
mainfrom
quality/test-codeql-contract
Oct 3, 2026
Merged

mrbobbytables merged 1 commit into
mainfrom
quality/test-codeql-contract

Conversation

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Test Improvement

Adds tests/codeql-contract.test.mjs. Test-only — no workflow or source file
is modified.

.github/workflows/codeql.yml is the only static-analysis gate in this
repository, it is not a required status check, and nothing asserted anything
about it. Because it runs behind a paths: filter, a filter that stops
matching its sources does not turn the workflow red — the workflow never
starts, and an unscanned tree is indistinguishable from a clean one. The same
holds for security-events: write: without it the analysis runs and then
cannot upload its SARIF.

tests/ci-supply-chain.test.mjs already covers the generic workflow properties
for every workflow including this one (SHA pinning, version comments,
non-blanket permissions, pinned runners, timeout-minutes,
persist-credentials). The nine tests added here cover only the CodeQL
invariants that have no cross-workflow analogue:

Test Invariant
required returns a present value and names the fault in the rest exercises the accessor helper's own reject path
the analyze job may upload its results to the security tab security-events: write
init and analyze come from the same codeql-action release a split pin is a classic silent breakage
the analysis declares a language set and the category that names it analyze.category tracks init.languages
push and pull_request filter on the same paths the two triggers cannot drift apart
every scanned source extension in the tree is matched by the path filters derived from the working tree, not hard-coded
a change to the scanner configuration triggers a scan codeql.yml is in its own filter
the dependency manifests are scanned when they change package.json, package-lock.json
the whole tree is scanned on a schedule schedule: with a five-field cron

The extension check derives its list by walking the tree, so adding a first
.ts directory without widening the filters fails the unit suite instead of
silently shrinking the scan. It asserts it found at least one JavaScript-family
source before concluding the filters are complete, following the "detection is
not vacuous" pattern in tests/ci-supply-chain.test.mjs and
tests/automation-workflow-ordering.test.mjs, so it cannot pass vacuously.

Verification

All nine pass on current main (6071e69) — codeql.yml already satisfies every
invariant, so this adds a regression floor rather than fixing a defect.

Mutation-verified: flipping security-events to read, dropping '**/*.mjs'
from the filters, narrowing category to /language:javascript, and adding a
sixth cron field each fail the corresponding test and nothing else.

npm run test:unit:coverage:check passes with the new file at 100.00% lines /
93.33% regions; src files stays at 100.00/100.00 (8351/8351 lines,
2395/2395 regions) and all files regions are unchanged at 95.02 against the
94 floor. npx prettier --check is clean.

Files claimed by this PR

tests/codeql-contract.test.mjs (new). Disjoint from every open hold-gated PR:
#989/#991 (tests/tools/e2e-coverage-report.mjs), #996
(tests/tools/coverage-report.mjs), #1000 (tests/ci-gating-jobs.test.mjs),
#1002 (tests/ci-concurrency.test.mjs), #998 (tests/adr-contract.test.mjs),
#1004/#1007/#1010/#1012 (ignore- and linter-config contracts). gh pr diff over
#1000 and #1002 returns zero codeql matches.

Related Issue

Closes #1013


Filed by quality agent (hold-gated mode). Human review required.

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88

codeql.yml is the only static-analysis gate in this repository and is not a
required status check, so when its push/pull_request path filters stop
matching the sources they target the scan never starts: an unscanned tree
reads exactly like a clean one. Nothing asserted anything about the file.

Add tests/codeql-contract.test.mjs, covering the invariants that have no
cross-workflow analogue in tests/ci-supply-chain.test.mjs: the path filters
enumerate every JavaScript-family extension present in the working tree, push
and pull_request scan the same set, the analyze job holds security-events:
write so its SARIF can be uploaded, init and analyze are pinned to the same
codeql-action release, analyze's category names the language set init
declared, and the weekly schedule survives with a five-field cron.

The extension list is derived from the tree rather than hard-coded, and the
check asserts it found at least one source before concluding the filters are
complete, so it cannot pass vacuously.

No workflow file is modified: codeql.yml already satisfies every invariant.

Closes #1013

Signed-off-by: quality <quality@hive.kubestellar.io>
@hivecommons-hive hivecommons-hive Bot added the hold label Oct 3, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

Important

Held for human review by the hive's ACMM level gate.

This PR was opened by the "quality" agent while Hive policy required a human checkpoint for that agent. Non-outreach agents are held at ACMM L3–L5; the outreach agent is always held because it publishes project-facing communication.

Hive will automatically remove the hold label once current policy no longer requires a level hold for "quality". If this is an outreach PR, a human must review it and remove the label.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[quality] codeql.yml has no contract test: the only static-analysis gate can be silently disabled

1 participant