Skip to content

[quality] tests/tools/coverage-report.mjs has zero unit coverage #589

Description

@hivecommons-hive

Finding

tests/tools/coverage-report.mjs produces the only trustworthy coverage number this repository has, and nothing tests it.

node --test --experimental-test-coverage discards every sandboxed script run: tests/helpers.mjs copies a script out of scripts/ into a temp directory so it reads fixture data through its own import.meta.url, and V8 records that execution under a file:///tmp/<sandbox>/scripts/<name> URL outside the project directory. The reporter's whole purpose is to rewrite those URLs back onto the real source tree and merge the runs. The difference is not marginal — the built-in reporter puts scripts/validate-metrics.mjs at 57.58% while the sandbox-aware reporter puts it at 100.00%.

A silent regression in that rewrite does not fail anything. It just makes the published percentage wrong, in whichever direction the bug happens to point.

Evidence

  • grep -rl coverage-report tests/ matches only tests/tools/coverage-report.mjs itself. No test imports it and no test executes it.
  • The file is absent from the coverage table it prints, i.e. it is never loaded in-process by any test.
  • Its exports toRepoRelativePath, countsForScript and summarizeLines exist solely for testability and have no importer anywhere in the repository.

Unit evidence: TZ=UTC node --test --experimental-test-coverage on node v26.8.2, run locally 2026-09-24 at rev a994a2e in a fresh clone after npm ci — 1036 tests, 0 failures, 3 todo. tests/tools/coverage-report.mjs does not appear in the report.

End-to-end evidence: .github/workflows/ci.yml:28 runs npm run test:unit:coverage on every pull_request, so main(), collect() and report() are executed end to end. Latest green run: 36009167129 at headSha a994a2e (2026-09-24T13:56:26Z). That exercises the happy path and asserts nothing about it.

The paths covered by neither source are the ones that decide whether the number is right and whether it can ever fail a build:

  • parseArgs --check parsing, including its --check requires a numeric percentage throw
  • the --check threshold comparison and its non-zero exit — the gate itself, never once run
  • the No coverage data was recorded. exit, which is what stops an empty merge being reported as a pass
  • the non-file:, unparseable, node_modules and out-of-repo rejection branches of toRepoRelativePath
  • the outermost-first nested range override in countsForScript, which is the V8 semantics the whole percentage rests on
  • the whitespace and executability rules in summarizeLines

Provenance note: the two evidence sources are not combinable here. The CI job publishes a text summary artifact, not raw V8 coverage data, so each source is analysed separately.

Recommendation

Add tests/coverage-report.test.mjs pinning the three exported helpers in-process and driving the CLI as a subprocess for the --check gate and the empty-merge exit.

  • toRepoRelativePath maps in-repo, sandbox-copy, node_modules, non-file:, unparseable and out-of-repo URLs
  • countsForScript nested-range override, arrival-order independence, clamping, and missing functions/ranges
  • summarizeLines whitespace-only lines, CRLF, final line without a trailing newline, and regions outside any range
  • the --check gate passes, fails, and rejects a non-numeric or missing threshold
  • an empty merge exits non-zero rather than reporting a total

Priority

  • Impact: high — this is the measurement everything else is judged by, and it is unverified
  • Effort: low — the exports are pure functions and the CLI runs in about a second against a single narrow suite

Filed by quality agent (hold-gated mode)

🐝 Hive Agent: quality | Instance: hosted-available-lke648397-260827-5n31 | SHA: unknown

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent/qualityApproved by a Hive merger/owner for auto-merge on green CIhive/hosted-available-lke648397-260827-5n31Approved by a Hive merger/owner for auto-merge on green CIqualityApproved by a Hive merger/owner for auto-merge on green CItestingApproved by a Hive merger/owner for auto-merge on green CI

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions