You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
tests/tools/coverage-report.mjs produces the only trustworthy coverage number this repository has, and nothing tests it.
node --test --experimental-test-coverage discards every sandboxed script run: tests/helpers.mjs copies a script out of scripts/ into a temp directory so it reads fixture data through its own import.meta.url, and V8 records that execution under a file:///tmp/<sandbox>/scripts/<name> URL outside the project directory. The reporter's whole purpose is to rewrite those URLs back onto the real source tree and merge the runs. The difference is not marginal — the built-in reporter puts scripts/validate-metrics.mjs at 57.58% while the sandbox-aware reporter puts it at 100.00%.
A silent regression in that rewrite does not fail anything. It just makes the published percentage wrong, in whichever direction the bug happens to point.
Evidence
grep -rl coverage-report tests/ matches only tests/tools/coverage-report.mjs itself. No test imports it and no test executes it.
The file is absent from the coverage table it prints, i.e. it is never loaded in-process by any test.
Its exports toRepoRelativePath, countsForScript and summarizeLines exist solely for testability and have no importer anywhere in the repository.
Unit evidence:TZ=UTC node --test --experimental-test-coverage on node v26.8.2, run locally 2026-09-24 at rev a994a2e in a fresh clone after npm ci — 1036 tests, 0 failures, 3 todo. tests/tools/coverage-report.mjs does not appear in the report.
End-to-end evidence:.github/workflows/ci.yml:28 runs npm run test:unit:coverage on every pull_request, so main(), collect() and report() are executed end to end. Latest green run: 36009167129 at headSha a994a2e (2026-09-24T13:56:26Z). That exercises the happy path and asserts nothing about it.
The paths covered by neither source are the ones that decide whether the number is right and whether it can ever fail a build:
parseArgs--check parsing, including its --check requires a numeric percentage throw
the --check threshold comparison and its non-zero exit — the gate itself, never once run
the No coverage data was recorded. exit, which is what stops an empty merge being reported as a pass
the non-file:, unparseable, node_modules and out-of-repo rejection branches of toRepoRelativePath
the outermost-first nested range override in countsForScript, which is the V8 semantics the whole percentage rests on
the whitespace and executability rules in summarizeLines
Provenance note: the two evidence sources are not combinable here. The CI job publishes a text summary artifact, not raw V8 coverage data, so each source is analysed separately.
Recommendation
Add tests/coverage-report.test.mjs pinning the three exported helpers in-process and driving the CLI as a subprocess for the --check gate and the empty-merge exit.
toRepoRelativePath maps in-repo, sandbox-copy, node_modules, non-file:, unparseable and out-of-repo URLs
countsForScript nested-range override, arrival-order independence, clamping, and missing functions/ranges
summarizeLines whitespace-only lines, CRLF, final line without a trailing newline, and regions outside any range
the --check gate passes, fails, and rejects a non-numeric or missing threshold
an empty merge exits non-zero rather than reporting a total
Priority
Impact: high — this is the measurement everything else is judged by, and it is unverified
Effort: low — the exports are pure functions and the CLI runs in about a second against a single narrow suite
Finding
tests/tools/coverage-report.mjsproduces the only trustworthy coverage number this repository has, and nothing tests it.node --test --experimental-test-coveragediscards every sandboxed script run:tests/helpers.mjscopies a script out ofscripts/into a temp directory so it reads fixture data through its ownimport.meta.url, and V8 records that execution under afile:///tmp/<sandbox>/scripts/<name>URL outside the project directory. The reporter's whole purpose is to rewrite those URLs back onto the real source tree and merge the runs. The difference is not marginal — the built-in reporter putsscripts/validate-metrics.mjsat 57.58% while the sandbox-aware reporter puts it at 100.00%.A silent regression in that rewrite does not fail anything. It just makes the published percentage wrong, in whichever direction the bug happens to point.
Evidence
grep -rl coverage-report tests/matches onlytests/tools/coverage-report.mjsitself. No test imports it and no test executes it.toRepoRelativePath,countsForScriptandsummarizeLinesexist solely for testability and have no importer anywhere in the repository.Unit evidence:
TZ=UTC node --test --experimental-test-coverageon node v26.8.2, run locally 2026-09-24 at reva994a2ein a fresh clone afternpm ci— 1036 tests, 0 failures, 3 todo.tests/tools/coverage-report.mjsdoes not appear in the report.End-to-end evidence:
.github/workflows/ci.yml:28runsnpm run test:unit:coverageon everypull_request, somain(),collect()andreport()are executed end to end. Latest green run: 36009167129 at headShaa994a2e(2026-09-24T13:56:26Z). That exercises the happy path and asserts nothing about it.The paths covered by neither source are the ones that decide whether the number is right and whether it can ever fail a build:
parseArgs--checkparsing, including its--check requires a numeric percentagethrow--checkthreshold comparison and its non-zero exit — the gate itself, never once runNo coverage data was recorded.exit, which is what stops an empty merge being reported as a passfile:, unparseable,node_modulesand out-of-repo rejection branches oftoRepoRelativePathcountsForScript, which is the V8 semantics the whole percentage rests onsummarizeLinesProvenance note: the two evidence sources are not combinable here. The CI job publishes a text summary artifact, not raw V8 coverage data, so each source is analysed separately.
Recommendation
Add
tests/coverage-report.test.mjspinning the three exported helpers in-process and driving the CLI as a subprocess for the--checkgate and the empty-merge exit.toRepoRelativePathmaps in-repo, sandbox-copy,node_modules, non-file:, unparseable and out-of-repo URLscountsForScriptnested-range override, arrival-order independence, clamping, and missingfunctions/rangessummarizeLineswhitespace-only lines, CRLF, final line without a trailing newline, and regions outside any range--checkgate passes, fails, and rejects a non-numeric or missing thresholdPriority
Filed by quality agent (hold-gated mode)
🐝 Hive Agent:
quality| Instance:hosted-available-lke648397-260827-5n31| SHA:unknown— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88