Repository navigation
[quality] data/metrics.json render contract is unenforced: MetricsDashboard crashes the build on input validate-metrics accepts #286
Description
Finding
scripts/validate-metrics.mjs and src/components/MetricsDashboard/index.js
disagree about the shape of data/metrics.json. The validator is tolerant
exactly where the component is not, so a data/metrics.json that passes
npm run validate:metrics can still crash npm run build.
The component dereferences without a guard:
metricsData.omitted.map(...)— the validator readsdata.omitted || [],
so an absent key validates clean.chart.values[0].valueas the bar-width divisor — the validator only checks
Array.isArray(chart.values), sovalues: []validates clean.metric.value.toLocaleString()andcard.value.toLocaleString()as numbers
— the validator rejects onlyundefined/null/'', so the string
"1,234"validates clean and renders unformatted.referenceArchitectureLifecycle.trends— the validator checks
.cardsand.omittedbut never inspects.trendsat all, so the values
Sparklinemaps over are entirely unvalidated.
Bar width is item.value / chart.values[0].value * 100, so the component also
assumes each breakdown is sorted descending and that values[0].value is
greater than zero. Neither is checked anywhere; a zero divisor is finite, so
it passes validation and makes every bar width NaN.
Verified
Against a local clone at 00b44df. Seven mutations of data/metrics.json
were applied one at a time; every one passes npm run validate:metrics:
| mutation | validate:metrics |
|---|---|
delete omitted |
PASSES |
breakdowns.projectMaturity.values = [] |
PASSES |
breakdowns.projectMaturity.values[0].value = 0 |
PASSES |
breakdowns.projectCategories.values.reverse() |
PASSES |
metrics[0].value = '1,234' |
PASSES |
trends.submissions.values = 'x' |
PASSES |
delete trends.publications.values[0].date |
PASSES |
The first mutation was then carried through a full npm run build:
$ npm run validate:metrics
Validated 2 metrics
$ npm run build
build exit=1
[cause]: TypeError: Cannot read properties of undefined (reading 'map')
So the guard that is supposed to catch bad generated metrics reports success
on input that breaks the site build.
Recommendation
Add a single test-only file, tests/metrics-data.test.mjs, asserting the real
data/metrics.json against the render contract MetricsDashboard relies on —
the invariants the table above proves the validator does not enforce. All of
these hold on main today, so the file is green on merge and fails only on
regression.
This is deliberately a data-contract test over the real file, matching the
approach already used for the other data files, rather than a change to
scripts/validate-metrics.mjs (production code) or another validator-fixture
test.
Completion criteria
-
tests/metrics-data.test.mjsexists and runs undernpm run test:unit - Each assertion is mutation-checked, so none can silently pass
- No production code, workflow,
package.jsonor dependency is modified
Priority
- Impact: medium — a bad generated
metrics.jsonreaches a red build instead
of a red validator, andcollect:metricsregenerates this file - Effort: low — one test file, no new dependency
Coverage evidence
- Unit:
npm run test:unitandnode --test --experimental-test-coverage,
local clone at00b44df, 2026-09-18, node v26.8.2 — 55 tests pass. These
invariants are covered by no unit test:data/metrics.jsonis data rather
than an executable module, so it appears in no coverage report at any
percentage, and no existing test reads the real file except the smoke test,
which only asserts the validator exits zero. - End-to-end: unavailable — this repository has no end-to-end or browser
suite and publishes no coverage artifact from any suite (tracked in [quality] CI publishes no coverage evidence, so coverage findings cannot be verified #186).
No claim is made that these paths lack end-to-end coverage.
Disjointness
tests/metrics-data.test.mjs is touched by no open PR. Distinct from #263,
which covers scripts/validate-metrics.mjs branch behaviour using synthetic
fixtures and asserts nothing about the real data/metrics.json — the table
above is precisely the set of shapes #263 proves the validator accepts.
Distinct from the other data-contract tests (#235, #239, #241, #253, #257,
#260, #270), from the navigation contract (#275) and script wiring contract
(#278), and from #229/#268, which add JSX/hook import infrastructure; this
needs none of it and adds no dependency.
Filed by quality agent (hold-gated mode)
— hive: agent=quality backend=copilot model=claude-opus-5
Activity