Skip to content

[quality] data/members.json has no contract test or validator #252

Description

@hivecommons-hive

Finding

data/members.json is the largest committed data file (17 organisations, ~21 KB) and the sole input to src/components/MemberDirectory/index.js, but nothing validates it: there is no validate:members script in package.json and no test file references it.

MemberDirectory dereferences the entry arrays with no guards:

member.industries.forEach((i) => industries.add(i));
member.projects.forEach((p) => projects.add(p));
member.architectures.length > 0 || member.awards.length > 0
member.sourceAttribution.map((url) => ...)

The file is generated by scripts/generate-members.mjs from data/architectures/catalog.json and data/awards.json. If a regeneration drops one of those arrays from an entry, the Docusaurus production build crashes; if it emits a logo path with no file behind it, the member directory renders a broken image. Neither failure is caught before deploy.

Sibling data files already have (or are getting) contract tests — data/milestones.json (#234), data/projects-born.json (#238), data/community-roster.json (#240). data/members.json is the remaining unvalidated one.

Coverage evidence

  • Unit: node --test --experimental-test-coverage at main 00b44df — 55 tests pass. The coverage table lists only scripts/lib/validate-utils.mjs, scripts/validate-{architecture-assets,architectures,awards,button-contrast,metrics}.mjs and tests/helpers.mjs. data/members.json appears in no test. Zero unit coverage.
  • End-to-end: the repository has no end-to-end or browser suite — no Playwright/Cypress config, dependency or job (matches in package-lock.json are transitive only), and none of the six workflows in .github/workflows/ runs one. No end-to-end coverage exists to measure.
  • Limitation: the two sources cannot be merged at statement granularity because only one exists. The closest end-to-end signal is the Docusaurus production build in deploy-gh-pages.yml, which would fail on a crash-inducing entry but is not a test and emits no coverage data — tracked separately by [quality] CI publishes no coverage evidence, so coverage findings cannot be verified #186.

Recommendation

Add tests/members-data.test.mjs asserting the contract that MemberDirectory and the generator depend on:

  • Envelope keys (description, generatedFrom, schema) present, and generatedFrom still names data/architectures/catalog.json and data/awards.json
  • Every member carries industries, projects, architectures, awards and sourceAttribution as arrays (the unguarded dereferences above)
  • id/slug are unique kebab-case and equal to each other, so profile links resolve
  • Members stay sorted by display name, matching generator output
  • Every logo, when set, is a /img/ path whose file exists under static/ (the check validate-awards.mjs already performs for awards.json)
  • Nested architecture entries carry id, title, an https sourceUrl and a 40-hex sourceCommit
  • Nested award entries carry year, award, awardLabel, citation, event, with https URLs where present
  • Cross-file parity: every awards.json winner slug has a matching member id, and every member award matches an awards.json entry

Note that parity must key on slug, not display name: awards.json lists Mercedes-Benz Tech Innovation while the member entry's display name is Mercedes-Benz, and both share the slug mercedes-benz-tech-innovation.

Priority

  • Impact: high — an unguarded regeneration breaks the production build or ships broken logos
  • Effort: low — data-only assertions, no new test infrastructure

Filed by quality agent (hold-gated mode)

— hive: agent=quality backend=copilot model=claude-opus-5

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent/qualityApproved by a Hive merger/owner for auto-merge on green CIhive/hosted-available-lke648397-260827-5n31Approved by a Hive merger/owner for auto-merge on green CIqualityApproved by a Hive merger/owner for auto-merge on green CItestingApproved by a Hive merger/owner for auto-merge on green CI

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions