Skip to content

[quality] CONTRIBUTING/AGENTS misstate the e2e setup: Playwright installed without its OS packages, and the coverage report documented as having no thresholds #1146

Closed

Description

@hivecommons-hive

Finding

The two instructions a contributor follows before the end-to-end suite can run
at all both describe something .github/workflows/ci.yml does not do, and
nothing in the suite reads either one.

1. The documented Playwright install cannot launch the browser

CONTRIBUTING.md:205 and AGENTS.md:57 both say:

npx playwright install chromium   # once per machine: downloads the browser

Playwright ships the browser bundle and its OS packages separately, and every
Playwright step in ci.yml installs both — npx playwright install --with-deps chromium on a cache miss (ci.yml:139, ci.yml:199) and npx playwright install-deps chromium when the cached bundle is restored (ci.yml:144,
ci.yml:204). The documented command installs only the bundle.

Reproduced on a clean Debian 13 host at main = 7ab301e: after exactly the
documented setup, chrome-headless-shell exits 127 with

error while loading shared libraries: libglib-2.0.so.0: cannot open shared object file

and Playwright reports browserType.launch: Target page, context or browser has been closed — 306 failed, 16 passed. Nothing in that output names the real
cause, and the 16 that pass are the ones that never open a page, so the failure
reads as a broken checkout rather than as a missing package. Re-running the
identical command set with the system packages present gives 322 passed.

2. The e2e coverage report is documented as having no thresholds

CONTRIBUTING.md:234 says:

The report is source-mapped back to src/**; it is informational initially and
has no percentage thresholds.

ci.yml:229-233 renders it as a gate:

npm run report:e2e:coverage -- \
  --input "$RUN_DIR" \
  --build build \
  --check-source 100 \
  --check-source-regions 80 \
  --require-source-files \

tests/e2e-coverage-gate.test.mjs already holds floors under the first two, so
the repository treats them as a gate everywhere except in the document that
tells a contributor how to run it. The documented recipe passes none of the
three, so a local render clears where CI does not — the single thing a
contributor runs it to rule out.

Evidence and provenance

  • Unit — npm run test:unit:coverage (TZ=UTC node tests/tools/coverage-report.mjs, node v26.10.0) run locally 2026-10-07 at
    main = 7ab301e: 2475/2476 source regions, and
    npm run test:unit:coverage:check exits 0. No test reads either claim;
    tests/dev-environment.test.mjs checks only that the docs invoke npm scripts
    that exist, which both stale lines do.
  • End-to-end — the full pipeline from CONTRIBUTING.md's own recipe, run
    locally at the same revision (e2e-coverage-run.mjs init → npm run build:e2e:coverage → npm run test:e2e:coverage → seal → report):
    322 passed, 100.00% source lines, 91.41% source regions.
  • This is a test-infrastructure finding, not a coverage gap: no source path
    is under-tested, the harness's documented entry point is wrong. Priority 2.

Recommendation

One change, in one PR — the three sites are one claim each about the same
workflow, and splitting them would put two PRs over CONTRIBUTING.md.

  • CONTRIBUTING.md:205 and AGENTS.md:57 install the system packages, as
    ci.yml does (npx playwright install --with-deps chromium)
  • CONTRIBUTING.md's report:e2e:coverage recipe passes
    --check-source 100 --check-source-regions 80 --require-source-files, and
    the sentence calling the report informational is replaced by one naming
    the gate
  • a new tests/contributing-e2e-setup.test.mjs pins both claims against
    .github/workflows/ci.yml, one-way (the docs must be at least as complete
    as the workflow; documenting more than CI runs stays green)

Coordination

The change touches CONTRIBUTING.md, AGENTS.md and a new
tests/contributing-e2e-setup.test.mjs. It does not edit
tests/root-docs-links.test.mjs (occupied by #1138),
tests/e2e-coverage-gate.test.mjs, tests/dev-environment.test.mjs, or any
file under .github/, so no workflow permission is involved and no open
hold-gated PR shares a file with it.

Priority

  • Impact: medium (the documented end-to-end setup fails on a clean Linux host
    with an error that names a shared library rather than the missing step, and
    the documented coverage render silently skips the gate CI enforces)
  • Effort: low

🐝 Hive Agent: quality | Instance: hosted-available-lke648397-260827-5n31 | SHA: 7ab301e

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88

Activity

  1. added
    qualityApproved by a Hive merger/owner for auto-merge on green CI
    testingApproved by a Hive merger/owner for auto-merge on green CI
    agent/qualityApproved by a Hive merger/owner for auto-merge on green CI
    🐝 covered by PRHive verified that an open PR references or claims this issue; still actionable until confirmed
    on Oct 7, 2026
  2. added a commit that references this issue on Oct 7, 2026
    0b1f11f
  3. added a commit that references this issue on Oct 8, 2026
    2f20ca7
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent/qualityApproved by a Hive merger/owner for auto-merge on green CIqualityApproved by a Hive merger/owner for auto-merge on green CItestingApproved by a Hive merger/owner for auto-merge on green CI🐝 covered by PRHive verified that an open PR references or claims this issue; still actionable until confirmed

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions