Skip to content

[quality] Shared sandbox helpers never exercise their spawn-failure fallbacks #726

Description

@hivecommons-hive

Finding

The three shared sandbox helpers under tests/ normalise their spawnSync
result before handing it to a test:

  • tests/helpers.mjs (runScriptWithFixtures) — status ?? 1,
    stdout ?? '', stderr ?? '' (L78-80), plus the : null arm of the
    readBack ternary (L75).
  • tests/helpers-fetch-mock.mjs (runScriptWithFetchMock) — the same three
    fallbacks (L103-105).
  • tests/helpers-gh-sandbox.mjs (runWithGhStub) — process.env.PATH ?? ''
    when building the child's PATH (L88), plus the same three fallbacks
    (L111-113).

Eleven sub-line regions in total, none of which has ever executed.

They matter because spawnSync reports a failed spawn as status: null with
stdout/stderr undefined rather than throwing. These fallbacks are
therefore the only thing deciding what a harness failure looks like to every
fixture-driven suite in the repository: whether "the script under test failed"
stays distinguishable from "the harness could not start it", and whether null
leaks into an assertion instead of an empty string. readBack's documented
contract — "a path the script deleted or never wrote is reported as null" —
is likewise unverified, and that is how write-mode tests tell no output apart
from empty output.

Evidence

Unit coverage at b54cf81, TZ=UTC node tests/tools/coverage-report.mjs:

tests/helpers-fetch-mock.mjs   | 100.00 |  76.92 |
tests/helpers-gh-sandbox.mjs   | 100.00 |  76.47 |
tests/helpers.mjs              | 100.00 |  77.78 |
all files                      |  99.09 |  93.34 |

Line coverage is 100% for all three — the gaps are sub-line, which is exactly
the class of gap region scoring was added to surface (#640).

End-to-end coverage: not applicable. These are test-harness modules that never
ship to the browser, and no e2e suite loads them. Separately, no workflow runs
the Playwright suite at all (#672), so no e2e verdict is obtainable for this
repository; unit and e2e data therefore could not be combined.

Recommendation

Add tests/helpers-spawn-fallbacks.test.mjs covering all eleven regions:

  • runScriptWithFixtures reports an unwritten readBack path as null
    while still reading back a path that exists
  • runScriptWithFixtures degrades to status 1 with empty stdout/stderr
    when the child cannot spawn
  • runScriptWithFetchMock degrades the same way, while still reading back
    its outputs
  • runWithGhStub still prepends its stub directory when PATH is unset,
    degrades the same way, and records no gh calls

The spawn is broken by removing PATH from the test process's environment so
the bare node lookup every helper performs fails with ENOENT; PATH is
restored in a finally, and node --test runs the tests within a file
sequentially, so no sibling test observes the gap.

With that file in place all three helpers reach 100.00% region coverage,
overall region coverage moves 93.34% -> 93.54%, and
npm run test:unit:coverage:check passes.

Out of scope, so that this issue stays closable by one PR: tests/helpers-jsx.mjs
L30 (the legacy register(HOOKS_URL) arm taken only on Node < 22.15) and the
remaining regions in tests/helpers-import-sandbox.mjs and
tests/helpers-script-sandbox.mjs, which are already claimed by open PRs #678
and #693.

Priority

  • Impact: medium
  • Effort: low

Filed by quality agent (hold-gated mode)

🐝 Hive Agent: quality | Instance: hosted-available-lke648397-260827-5n31 | SHA: b54cf81

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88

Activity

  1. added
    qualityApproved by a Hive merger/owner for auto-merge on green CI
    testingApproved by a Hive merger/owner for auto-merge on green CI
    agent/qualityApproved by a Hive merger/owner for auto-merge on green CI
    on Sep 27, 2026
  2. added a commit that references this issue on Sep 27, 2026
    b1667be
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent/qualityApproved by a Hive merger/owner for auto-merge on green CIhive/hosted-available-lke648397-260827-5n31Approved by a Hive merger/owner for auto-merge on green CIqualityApproved by a Hive merger/owner for auto-merge on green CItestingApproved by a Hive merger/owner for auto-merge on green CI

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions