Repository navigation
[quality] CONTRIBUTING/AGENTS misstate the e2e setup: Playwright installed without its OS packages, and the coverage report documented as having no thresholds #1146
Description
Finding
The two instructions a contributor follows before the end-to-end suite can run
at all both describe something .github/workflows/ci.yml does not do, and
nothing in the suite reads either one.
1. The documented Playwright install cannot launch the browser
CONTRIBUTING.md:205 and AGENTS.md:57 both say:
npx playwright install chromium # once per machine: downloads the browser
Playwright ships the browser bundle and its OS packages separately, and every
Playwright step in ci.yml installs both — npx playwright install --with-deps chromium on a cache miss (ci.yml:139, ci.yml:199) and npx playwright install-deps chromium when the cached bundle is restored (ci.yml:144,
ci.yml:204). The documented command installs only the bundle.
Reproduced on a clean Debian 13 host at main = 7ab301e: after exactly the
documented setup, chrome-headless-shell exits 127 with
error while loading shared libraries: libglib-2.0.so.0: cannot open shared object file
and Playwright reports browserType.launch: Target page, context or browser has been closed — 306 failed, 16 passed. Nothing in that output names the real
cause, and the 16 that pass are the ones that never open a page, so the failure
reads as a broken checkout rather than as a missing package. Re-running the
identical command set with the system packages present gives 322 passed.
2. The e2e coverage report is documented as having no thresholds
CONTRIBUTING.md:234 says:
The report is source-mapped back to
src/**; it is informational initially and
has no percentage thresholds.
ci.yml:229-233 renders it as a gate:
npm run report:e2e:coverage -- \
--input "$RUN_DIR" \
--build build \
--check-source 100 \
--check-source-regions 80 \
--require-source-files \tests/e2e-coverage-gate.test.mjs already holds floors under the first two, so
the repository treats them as a gate everywhere except in the document that
tells a contributor how to run it. The documented recipe passes none of the
three, so a local render clears where CI does not — the single thing a
contributor runs it to rule out.
Evidence and provenance
- Unit —
npm run test:unit:coverage(TZ=UTC node tests/tools/coverage-report.mjs, node v26.10.0) run locally 2026-10-07 at
main=7ab301e:2475/2476source regions, and
npm run test:unit:coverage:checkexits0. No test reads either claim;
tests/dev-environment.test.mjschecks only that the docs invoke npm scripts
that exist, which both stale lines do. - End-to-end — the full pipeline from
CONTRIBUTING.md's own recipe, run
locally at the same revision (e2e-coverage-run.mjs init→npm run build:e2e:coverage→npm run test:e2e:coverage→seal→report):
322 passed,100.00%source lines,91.41%source regions. - This is a test-infrastructure finding, not a coverage gap: no source path
is under-tested, the harness's documented entry point is wrong. Priority 2.
Recommendation
One change, in one PR — the three sites are one claim each about the same
workflow, and splitting them would put two PRs over CONTRIBUTING.md.
-
CONTRIBUTING.md:205andAGENTS.md:57install the system packages, as
ci.ymldoes (npx playwright install --with-deps chromium) -
CONTRIBUTING.md'sreport:e2e:coveragerecipe passes
--check-source 100 --check-source-regions 80 --require-source-files, and
the sentence calling the report informational is replaced by one naming
the gate - a new
tests/contributing-e2e-setup.test.mjspins both claims against
.github/workflows/ci.yml, one-way (the docs must be at least as complete
as the workflow; documenting more than CI runs stays green)
Coordination
The change touches CONTRIBUTING.md, AGENTS.md and a new
tests/contributing-e2e-setup.test.mjs. It does not edit
tests/root-docs-links.test.mjs (occupied by #1138),
tests/e2e-coverage-gate.test.mjs, tests/dev-environment.test.mjs, or any
file under .github/, so no workflow permission is involved and no open
hold-gated PR shares a file with it.
Priority
- Impact: medium (the documented end-to-end setup fails on a clean Linux host
with an error that names a shared library rather than the missing step, and
the documented coverage render silently skips the gate CI enforces) - Effort: low
🐝 Hive Agent: quality | Instance: hosted-available-lke648397-260827-5n31 | SHA: 7ab301e
— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88
Activity
test: pin the documented e2e setup against the workflow that runs it
test: pin the documented e2e setup against what ci.yml actually runs (c…