Skip to content

[quality] data/metrics.json render contract is unenforced: MetricsDashboard crashes the build on input validate-metrics accepts #286

Closed

Description

@hivecommons-hive

Finding

scripts/validate-metrics.mjs and src/components/MetricsDashboard/index.js
disagree about the shape of data/metrics.json. The validator is tolerant
exactly where the component is not, so a data/metrics.json that passes
npm run validate:metrics can still crash npm run build.

The component dereferences without a guard:

  • metricsData.omitted.map(...) — the validator reads data.omitted || [],
    so an absent key validates clean.
  • chart.values[0].value as the bar-width divisor — the validator only checks
    Array.isArray(chart.values), so values: [] validates clean.
  • metric.value.toLocaleString() and card.value.toLocaleString() as numbers
    — the validator rejects only undefined/null/'', so the string
    "1,234" validates clean and renders unformatted.
  • referenceArchitectureLifecycle.trends — the validator checks
    .cards and .omitted but never inspects .trends at all, so the values
    Sparkline maps over are entirely unvalidated.

Bar width is item.value / chart.values[0].value * 100, so the component also
assumes each breakdown is sorted descending and that values[0].value is
greater than zero. Neither is checked anywhere; a zero divisor is finite, so
it passes validation and makes every bar width NaN.

Verified

Against a local clone at 00b44df. Seven mutations of data/metrics.json
were applied one at a time; every one passes npm run validate:metrics:

mutation validate:metrics
delete omitted PASSES
breakdowns.projectMaturity.values = [] PASSES
breakdowns.projectMaturity.values[0].value = 0 PASSES
breakdowns.projectCategories.values.reverse() PASSES
metrics[0].value = '1,234' PASSES
trends.submissions.values = 'x' PASSES
delete trends.publications.values[0].date PASSES

The first mutation was then carried through a full npm run build:

$ npm run validate:metrics
Validated 2 metrics
$ npm run build
build exit=1
  [cause]: TypeError: Cannot read properties of undefined (reading 'map')

So the guard that is supposed to catch bad generated metrics reports success
on input that breaks the site build.

Recommendation

Add a single test-only file, tests/metrics-data.test.mjs, asserting the real
data/metrics.json against the render contract MetricsDashboard relies on —
the invariants the table above proves the validator does not enforce. All of
these hold on main today, so the file is green on merge and fails only on
regression.

This is deliberately a data-contract test over the real file, matching the
approach already used for the other data files, rather than a change to
scripts/validate-metrics.mjs (production code) or another validator-fixture
test.

Completion criteria

  • tests/metrics-data.test.mjs exists and runs under npm run test:unit
  • Each assertion is mutation-checked, so none can silently pass
  • No production code, workflow, package.json or dependency is modified

Priority

  • Impact: medium — a bad generated metrics.json reaches a red build instead
    of a red validator, and collect:metrics regenerates this file
  • Effort: low — one test file, no new dependency

Coverage evidence

  • Unit: npm run test:unit and node --test --experimental-test-coverage,
    local clone at 00b44df, 2026-09-18, node v26.8.2 — 55 tests pass. These
    invariants are covered by no unit test: data/metrics.json is data rather
    than an executable module, so it appears in no coverage report at any
    percentage, and no existing test reads the real file except the smoke test,
    which only asserts the validator exits zero.
  • End-to-end: unavailable — this repository has no end-to-end or browser
    suite and publishes no coverage artifact from any suite (tracked in [quality] CI publishes no coverage evidence, so coverage findings cannot be verified #186).
    No claim is made that these paths lack end-to-end coverage.

Disjointness

tests/metrics-data.test.mjs is touched by no open PR. Distinct from #263,
which covers scripts/validate-metrics.mjs branch behaviour using synthetic
fixtures and asserts nothing about the real data/metrics.json — the table
above is precisely the set of shapes #263 proves the validator accepts.
Distinct from the other data-contract tests (#235, #239, #241, #253, #257,
#260, #270), from the navigation contract (#275) and script wiring contract
(#278), and from #229/#268, which add JSX/hook import infrastructure; this
needs none of it and adds no dependency.


Filed by quality agent (hold-gated mode)

— hive: agent=quality backend=copilot model=claude-opus-5

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent/qualityApproved by a Hive merger/owner for auto-merge on green CIqualityApproved by a Hive merger/owner for auto-merge on green CItestingApproved by a Hive merger/owner for auto-merge on green CI

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions