Skip to content

test: make the e2e coverage artifact renderable without its build - #1040

Merged
castrojo merged 3 commits into
mainfrom
quality/e2e-coverage-self-contained-artifact
Oct 6, 2026
Merged

castrojo merged 3 commits into
mainfrom
quality/e2e-coverage-self-contained-artifact

Conversation

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Test Improvement

Rendering the published e2e-coverage artifact required a byte-identical
build/, and that build does not reproduce outside the runner — so the number
behind --check-source-regions 80 could only be audited by pushing to CI.

Confirmed before changing anything. At main 900592b, with the real
artifact's chunk and a real npm run build:e2e:coverage, the reporter aborts
on the first unresolvable script:

coverage script not found in <run>/scripts, <repo>/build: http://localhost:3000/assets/js/3fa7bede.e4867f23.js

What this changes

  • tests/tools/e2e-coverage-scripts.mjs (new) — owns
    isEligibleScript / scriptPathname (moved out of the reporter so capture
    and report can never disagree about which scripts matter), plus
    resolveScriptInRoot and captureRunScripts.
  • tests/tools/e2e-coverage-run.mjs — sealCoverageRun copies every
    referenced script and its source map into <run>/scripts/ before the
    manifest flips to a sealed status, and records capturedScripts /
    uncapturedScripts as provenance. Capture is opt-in at the library level and
    defaults to build in the CLI (new optional --build-dir).
  • tests/tools/e2e-coverage-report.mjs — convertScript resolves each
    script against an ordered root list, <run>/scripts first and build/
    second, and loads the source map from whichever root matched.

No workflow edit is needed: ci.yml already runs the seal step and already
uploads path: coverage/e2e/, so the copies ship with the existing artifact.

Design points

  • A script is captured only together with the map it names. Copying it
    alone would shadow a usable build/ copy with one the reporter cannot map,
    so a script with no map is left out and listed in uncapturedScripts.
  • Fail-closed is preserved, as the issue asked. A script resolvable from no
    root still aborts the whole report; nothing is skipped to make a number
    appear.
  • sourceLength / sourceSha256 still apply to the captured copy, which is
    what makes a self-contained artifact trustworthy rather than merely
    convenient. Files are copied via a temporary name and renamed, so a crashed
    seal cannot leave a half-written script.
  • Capture declines a path-escaping URL instead of throwing, keeping the
    reporter the single place that rejects one — the seal step runs with
    if: always() and must not fail on a hostile URL.

Verification

  • TZ=UTC node --test — 1921 passed, 0 failed (1912 before; 9 new tests).
  • npm run test:unit:coverage:check — exit 0. all files regions
    94.79% → 94.89%, lines 99.39% → 99.41%; src files unchanged at 100.00%
    lines / 99.84% regions. The new module is 100.00% lines / 94.67% regions.
  • npx prettier --check on all five changed files — clean.
  • Real-bundle check, not just fixtures: sealed a run against a real
    npm run build:e2e:coverage chunk, deleted build/ outright, then rendered.
    The real chunk and its real source map resolved, passed the length and
    sha256 checks, and converted through v8-to-istanbul with no build present
    — reaching attribution, which a hand-written single whole-file range cannot
    satisfy. Deleting <run>/scripts as well restores the original
    coverage script not found abort, so the fail-closed path is intact.

Playwright cannot launch in the environment this was written in
(libglib-2.0.so.0 missing, no root), so no browser capture was executed
here; the End-to-end coverage job on this PR exercises the real capture path.

This unblocks #1035, which could not be diagnosed locally for exactly this
reason.

Related Issue

Closes #1036


Filed by quality agent (hold-gated mode). Human review required.

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88

Rendering the published e2e-coverage artifact required the exact build/ that
produced it, and that build does not reproduce outside the runner, so the
number behind --check-source-regions could not be audited anywhere else.

Seal now copies each referenced script and its source map into the run
directory, and the reporter prefers those copies over build/. The artifact
travels with its own inputs; no workflow change is needed because ci.yml
already uploads coverage/e2e/ and already runs the seal step.

A script is only captured together with the map it names, so a partial copy
can never shadow a usable build/ copy. The sourceLength/sourceSha256 checks
still apply to the captured copy, and a script resolvable from no root still
fails the report closed rather than rendering on partial data.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: quality <quality@hive.kubestellar.io>
@hivecommons-hive hivecommons-hive Bot added the hold label Oct 4, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

Important

Held for human review by the hive's ACMM level gate.

This PR was opened by the "quality" agent while Hive policy required a human checkpoint for that agent. Non-outreach agents are held at ACMM L3–L5; the outreach agent is always held because it publishes project-facing communication.

Hive will automatically remove the hold label once current policy no longer requires a level hold for "quality". If this is an outreach PR, a human must review it and remove the label.

…self-contained-artifact

Signed-off-by: quality <quality@hive.kubestellar.io>

# Conflicts:
#	tests/e2e-coverage-report.test.mjs
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

⚠️ Hold-gate mergeability: this PR has a real conflict with its base

This PR is hold-gated and has fallen behind its base branch. The hive tried to merge the base in to keep it mergeable while it waits on review, but that merge conflicts and the hive cannot resolve it automatically.

It will stay held and will NOT merge on its own even once reviewed — removing hold alone will not be enough. Someone will need to merge or rebase the base branch in by hand (or resolve the conflict another way) before this PR can merge.

…self-contained-artifact

# Conflicts:
#	tests/e2e-coverage-report.test.mjs
@castrojo
castrojo added this pull request to the merge queue Oct 6, 2026
Merged via the queue into main with commit 7ab301e Oct 6, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[quality] e2e coverage artifact is not reproducible outside CI: no inline script source, and the build does not reproduce byte-for-byte

1 participant