test: consolidated coverage suite for test tooling, validators, collectors, and CI contracts - #750
Merged
mrbobbytables merged 15 commits intoSep 27, 2026
Conversation
![hivecommons-hive[bot]](https://github.githubassets.com/images/gravatars/gravatar-user-420.png){kind=avatar}
The sidebar-title contract in tests/docs-contract.test.mjs restated the autogenerated sidebar directories by hand as ['architectures', 'community'], but sidebars.js declares a third one, resourcesSidebar, from docs/resources. The three docs under it were never checked for a frontmatter title, so losing one would silently degrade the sidebar label to the filename. Derive the list from sidebars.js, the way tests/site-config.test.mjs already does for its directory-existence assertion, and assert the derived list is non-empty so the contract cannot become a no-op. Signed-off-by: hivecommons-hive[bot] <hivecommons-hive@hive.kubestellar.io>
No job in .github/workflows/ declares timeout-minutes, so all eleven inherit GitHub's 6-hour default. The commands these workflows run are network-bound -- markdown-link-check walks every link in every root Markdown file, and the scheduled jobs call the GitHub API -- so an unreachable host makes a step hang rather than fail and the job holds a runner for six hours. Only deploy-gh-pages declares a concurrency group, so the next run does not supersede a stuck one either. Add three contracts to tests/ci-supply-chain.test.mjs, alongside the existing action-pinning, permissions and persist-credentials contracts: - every boundable job declares timeout-minutes, with the eleven current gaps recorded in KNOWN_UNBOUNDED_JOBS - every declared value is a positive whole number below 360, checked through timeoutProblem(), which is exercised directly against usable values, quoted scalars, fractions, zero, negatives and 360+ - the baseline retires itself, failing the moment a listed job gains a timeout without its entry being removed KNOWN_UNBOUNDED_JOBS is a retiring baseline, not an allowance: the suite is green today and turns red the moment a job is bounded, which couples the fix to emptying the list. Jobs that delegate to a reusable workflow are excluded, since timeout-minutes is not accepted there. Signed-off-by: hivecommons-hive[bot] <hivecommons-hive@hive.kubestellar.io>
tests/workflow-scripts.test.mjs guards that package.json scripts, the files under scripts/, the linter configs they reference, and the npm run calls in .github/workflows/ all name something real, but never asserts that the repository's gates are actually invoked by a workflow. A gate nobody runs is inert: it passes when run by hand, the unit suite stays green, and the breakage it was written to catch reaches main. Adds a guard over the validate:* and check:* scripts, with an explicit exemption map recording the two gates CI deliberately does not run (check:links reaches the live internet, check:community-group-links needs GH_TOKEN and rewrites data). The map is itself guarded against rot: entries must name a defined script, carry a reason, and not name a gate a workflow already runs. Signed-off-by: hivecommons-hive[bot] <hivecommons-hive@hive.kubestellar.io>
tests/static-assets.test.mjs asserted six static-asset contracts but never exercised the branches that report a violation: describe(), the missing, not-a-file, zero-byte, case-mismatch, traversal and CNCFProjectCard logo push sites were all uncovered (89.06% lines / 85.71% regions). Its only guard against becoming vacuous was REFERENCES.size > 20, which catches a scan that finds nothing but not one that quietly stops finding the broken references. Move the scan and the five checkers into tests/tools/static-references.mjs, parameterised by root, so the contract test keeps asserting them against the real repository while tests/static-references.test.mjs drives each one against fixture trees that really do contain a missing, zero-byte, case-mismatched, escaping and unresolvable reference. All three files now report 100% line and region coverage. Signed-off-by: hivecommons-hive[bot] <hivecommons-hive@hive.kubestellar.io>
… report playwright.config.js was the only executable JavaScript file at the repository root absent from the test:unit:coverage report. Nothing imported it, so Node never instrumented it and the coverage gate could not see it at all. Cover the environment-dependent branches it evaluates at module scope: E2E_PORT threading into use.baseURL, webServer.url and webServer.command; the CI-conditional forbidOnly, retries, reporter and webServer.reuseExistingServer; and testDir agreeing with the directory that actually holds the spec files. The config now reports 100.00 / 100.00. Closes cncf#732 Signed-off-by: hivecommons-hive[bot] <hivecommons-hive@hive.kubestellar.io>
runScriptWithFixtures, runScriptWithFetchMock and runWithGhStub each normalise their spawnSync result before returning it (status ?? 1, stdout ?? '', stderr ?? '') and runWithGhStub falls back to an empty PATH tail when PATH is unset. spawnSync reports a failed spawn as status null with stdout/stderr undefined rather than throwing, so those eleven sub-line regions decide what a harness failure looks like to every fixture-driven suite -- and none of them had ever executed. The new file breaks the bare 'node' lookup by removing PATH from the test process, restoring it in a finally. It also pins readBack's documented contract that a path the script never wrote comes back as null. All three helpers reach 100% region coverage; overall region coverage moves 93.34% to 93.54%. Closes cncf#726 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: hivecommons-hive[bot] <hivecommons-hive@hive.kubestellar.io>
Adds tests/coverage-report-cli.test.mjs, covering the reporter CLI paths that tests/coverage-report.test.mjs cannot reach: it deletes NODE_V8_COVERAGE from the reporter subprocess, so the reporter's own execution is never recorded and only paths with an exit code of their own can be asserted on. Covers parseArgs' passthrough of a bare node --test argument (76-78), the "Not reported" notice for records made against loader-generated text (570-578), and the non-zero exit when the spawned suite fails (580-583). Line coverage of tests/tools/coverage-report.mjs moves 96.21 -> 98.35 and region coverage 93.10 -> 94.40. Closes cncf#718 Signed-off-by: hivecommons-hive[bot] <hivecommons-hive@hive.kubestellar.io>
tests/tools/fake-dom.mjs is the stand-in document every focus-trap test mounts against, and four of its sub-line regions were unexercised: the removeEventListener path for a type that was never registered, the empty fallbacks in keydownListenerCount and dispatchKeyDown, and the restore arm that puts back a document which existed before the install. The focus-trap suite never reaches them because it registers exactly one keydown listener and only runs where no global document exists. A regression there surfaces as a confusing failure in an unrelated suite, or as a leaked global document that quietly changes how a later test behaves. Region coverage for tests/tools/fake-dom.mjs goes from 92.00% to 100%. Signed-off-by: hivecommons-hive[bot] <hivecommons-hive@hive.kubestellar.io>
tests/tools/react-hook-driver.mjs sits under every component suite, yet six of its sub-line regions were unexercised: the lazy useState initialiser, the functional updater, the Object.is bail-out, the undefined-deps arm of sameDeps, the unchanged-deps effect skip, and the repeat-unmount guard. A regression in any of them surfaces as an unexplained component failure, or as a suite that keeps passing while asserting nothing. This adds six tests to tests/react-harness.test.mjs, the file that already exists to pin harness branches no component suite reaches. Region coverage for tests/tools/react-hook-driver.mjs goes from 87.23% to 100%. Signed-off-by: hivecommons-hive[bot] <hivecommons-hive@hive.kubestellar.io>
collect() reads the raw NODE_V8_COVERAGE dump, folds each sandbox URL back onto the real source tree, and merges every process's record for a file into one per-offset count map. Every coverage gate scores what it returns. It had no direct test, and its own execution is invisible to the report it produces: the reporter runs collect() in the parent process and sets NODE_V8_COVERAGE only for the node --test child it spawns. Add tests/coverage-report-collect.test.mjs, driving collect() against a hand-built coverage directory and a fake repository root so the merge arithmetic and each skip-this-record fallback are pinned independently of whatever the real suite happens to execute. Closes cncf#701 Signed-off-by: hivecommons-hive[bot] <hivecommons-hive@hive.kubestellar.io>
The three data collectors each report 100% line coverage while six sub-line regions never execute: line coverage marks a line covered when any character on it ran, so a || default or a ternary arm that never executed stays invisible whenever the rest of its line did run. Covered here: - collect-metrics.mjs:39 source.landscape || [] - collect-metrics.mjs:41 subcategory.items || [] - collect-metrics.mjs:45 category.name || 'Other' - collect-radar-reports.mjs:52 existing.radarReports || [] - collect-launch-metrics.mjs:142 githubAll Authorization ternary, both arms The Authorization pair needed a new capability rather than a new fixture: the sandbox pins GH_TOKEN='' for reproducibility and had no way to observe outbound request headers, so no test could assert what a collector sends. runScriptInSandbox now logs every stubbed fetch and returns it as `requests`, which lets the token and no-token cases both be asserted directly instead of inferred from the response handling. Source region coverage rises from 97.95% to 98.26%; all 1186 unit tests pass. Closes cncf#692 Signed-off-by: hivecommons-hive[bot] <hivecommons-hive@hive.kubestellar.io>
Line coverage is 100% across this repository's source files, so the remaining unit gaps are sub-line regions. Seven of them form one cluster: the fallback labels the validator scripts attach to an error when the record that caused it has no natural identifier, plus the two || [] guards that keep a missing array from becoming an iteration TypeError. Drive each arm from the four existing validator test files: - validate-architectures: an id-less catalog record, asserting both 'path: record.id || "<unknown>"' sites label the error <unknown> - validate-case-studies: an entry with a slug but no id reaches the slug arm; one with neither reaches the 'unknown' literal - validate-launch-metrics: a file with no signals key is reported as a count failure rather than a crash, and an id-less signal is filed under '(missing id)' - validate-metrics: a file with no metrics key is reported by its real failure rather than an iteration TypeError All four files now report 100% region coverage. Closes cncf#687 Signed-off-by: hivecommons-hive[bot] <hivecommons-hive@hive.kubestellar.io>
scripts/fetch-community-people.mjs reported 100.00% line but 94.20% region coverage. Four fallback regions never executed: - lastSegment()'s '|| null' for a URL whose path strips to no segment - the '|| previous.name' and "|| ''" arms of the name fallback chain - the ": 's'" plural suffix on the run's fallback count Add one case per region. The file now reports 100.00% region coverage and the repo-wide src region figure moves from 97.95% to 98.19%. Closes cncf#685 Signed-off-by: hivecommons-hive[bot] <hivecommons-hive@hive.kubestellar.io>
…values ATTRIBUTE_PATTERN in scripts/lib/svg-active-content.mjs accepts three attribute value forms -- double-quoted, single-quoted and unquoted -- and findActiveContent and stripActiveContent read them through the fallback chains 'match[2] ?? match[3] ?? match[4]' and 'dq ?? sq ?? uq'. Every existing case wrote double-quoted markup, so only the first alternative of either chain ran: a regression that dropped an alternative from the pattern, or read the wrong capture group, would leave the suite green while onclick='alert(1)' and href=javascript:alert(1) passed the sanitizer unseen. Add quoting variants for the three attribute classes the scanner recognises -- event handler, script URI, embedded document -- plus an inert-value case proving the pattern does not over-match. Region coverage for scripts/lib/svg-active-content.mjs rises 93.18% -> 97.73% and src files 97.95% -> 98.20%. The two remaining '?? ""' tails are unreachable, since the regex cannot match with all three capture groups undefined. Signed-off-by: hivecommons-hive[bot] <hivecommons-hive@hive.kubestellar.io>
scripts/import-architectures.mjs scored 100.00% line coverage but only
92.22% region coverage -- the lowest region score of any shipped source
file. Seven executable regions never ran, three of them fail-closed
paths whose regression would leave the line score untouched.
Adds tests/import-architectures-fallbacks.test.mjs covering:
L183 the '?? {}' fallback in splitFrontmatter for an empty block
L234 both arms of cleanMarkdown's image rewrite -- a non-artwork
remote image demoted to a plain link, and an artwork image
rewritten to its mirrored /img/cncf-projects path
L249 the 'continue' in mirrorProjectAssets for an artwork URL with
no derivable mirror name, asserted to skip before any fetch
L280 the "?? ''" fallback in firstParagraph for a body with no prose
L305 the false arm of walkFiles' 'entry.isFile() ? [path] : []',
reached with a FIFO in images/
L338 both dimension ternaries in sanitizeArchitectureAssets -- an SVG
L339 with no viewBox and no width/height gains no NaN viewBox, while
a sized SVG still gains a real one
Adds an upstreamFifos option to runImportArchitectures; a named pipe is
the only entry kind that is neither symlink, directory nor regular file.
scripts/import-architectures.mjs now reports 100.00% region coverage and
source region coverage rises from 97.95% to 98.39%.
Closes cncf#677
Signed-off-by: hivecommons-hive[bot] <hivecommons-hive@hive.kubestellar.io>
This was referenced Sep 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Consolidates 15 open test-only PRs into a single reviewable change. Every commit is an unmodified cherry-pick from its source PR (original authorship and DCO sign-offs preserved), and all 15 applied cleanly with no conflicts.
npm run test:unit: 1276 pass, 0 fail.No source, script, or workflow files are modified — this PR only adds/extends test files and test helpers.
Consolidated PRs
The source PRs above can be closed in favor of this one.
Closes #746, closes #743, closes #741, closes #734, closes #732, closes #726, closes #718, closes #707, closes #705, closes #701, closes #692, closes #687, closes #685, closes #679, closes #677