Skip to content

[quality] the member directory freshness note is rendered from three scheduled data files and asserted by no e2e test #1074

Closed

Description

@hivecommons-hive

Finding

/community/members renders DirectoryFreshness
(src/components/MemberDirectory/DirectoryFreshness.js), the provenance note
that tells a reader how current the directory is:

Directory membership data last synced from cncf/landscape on October 1, 2026.
Architecture-derived profiles last synced from cncf/architecture on September 27, 2026.
Award data last verified on August 7, 2026.

Each of its three sentences is built from a different scheduled data file, and
each is guarded independently:

sentence source field
membership data/members.json sources.landscape.collectedAt
architectures data/metrics.json generatedAt
awards data/awards.json verifiedAt

formatDate() (src/components/MemberDirectory/utils.js:14-22) returns null
for anything new Date() cannot parse, and every sentence sits behind a
&& on its own date (lines 22, 36, 50). A refresh that writes a malformed or
absent timestamp therefore makes that sentence disappear silently — and if
all three go, line 19 returns null and the whole note vanishes.

Nothing in tests/e2e/** asserts any of it. grep -rn 'last synced from' tests/e2e/ returns nothing, and neither tests/e2e/member-directory.spec.js
nor tests/e2e/interactions.spec.js looks at the note.

Why the existing unit tests do not cover this

tests/member-directory.test.mjs:367 and :420 do assert the note, but they
drive the component through tests/tools/fake-dom.mjs with injected
fixtures. They assert that given a good timestamp the sentence renders. They
cannot fail when the committed data/*.json carries a bad one, because they
never read it. The three files are rewritten on schedules
(.github/workflows/refresh-*.yml, npm run generate:members,
npm run collect:metrics), so the failure mode is a data change with no source
change at all — exactly the case the unit suite is blind to.

This is the same reasoning the repository already applied to the analogous
PeopleFreshness note, which is guarded end to end at
tests/e2e/community-people.spec.js:116-122, with the rationale written into
that spec's header comment:

The roster is also rebuilt on a schedule from data/community-people.json by
npm run fetch:community-people, so it can go blank with no source change at
all. These cases assert only that it renders.

DirectoryFreshness has three such sources instead of one and no equivalent
guard.

This is not a coverage gap

Stated explicitly so the finding is not mis-filed: the note renders on every
/community/members visit, so its regions are already counted covered by both
suites. There is no uncovered region to chase here. What is missing is an
assertion — the output is produced and never checked.

Recommendation

Add an end-to-end case that loads /community/members and asserts the rendered
note against the committed data, rather than against a fixture:

  1. Assert the note is present and that each of the three sentences appears
    whenever its backing timestamp is present in the committed JSON, reading
    those files at run time so a scheduled refresh cannot make the test stale.
  2. Assert the two provenance links (cncf/landscape, cncf/architecture)
    carry absolute https:// hrefs and open in a new tab, so a malformed
    sourceUrl/repository is caught as well as a malformed date.
  3. Put it in a new spec file rather than tests/e2e/member-directory.spec.js,
    which open PR test: cover the member directory's four undriven toolbar controls #1060 is currently editing.

Evidence and provenance

  • E2E: local reproduction of CI job End-to-end coverage at revision
    900592b — node tests/tools/e2e-coverage-run.mjs init,
    npm run build:e2e:coverage, npm run test:e2e:coverage (298 passed,
    chromium, node v26.10.0), seal, then npm run report:e2e:coverage. A browser
    probe of that build confirms the note renders with all three sentences and
    both links.
  • Unit: npm run test:unit:coverage at the same revision — src files
    100.00% lines / 99.84% regions.

Completion criteria

  • /community/members has an end-to-end case asserting the directory freshness note against the committed data
  • Each of the three sentences is asserted whenever its backing timestamp is present
  • The cncf/landscape and cncf/architecture links are asserted to be absolute https and open in a new tab
  • The case lives outside tests/e2e/member-directory.spec.js

Priority

  • Impact: medium (a scheduled data refresh can blank the directory's only provenance signal with every test still green)
  • Effort: low

🐝 Hive Agent: quality | SHA: 900592b


🐝 Hive Agent: quality | Instance: hosted-available-lke648397-260827-5n31 | SHA: 900592b

— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88

Activity

  1. added
    qualityApproved by a Hive merger/owner for auto-merge on green CI
    testingApproved by a Hive merger/owner for auto-merge on green CI
    agent/qualityApproved by a Hive merger/owner for auto-merge on green CI
    🐝 covered by PRHive verified that an open PR references or claims this issue; still actionable until confirmed
    on Oct 5, 2026
  2. added
    blocked-on-reporterPaused until the e2e coverage reporter fix (#1079) lands; may be a phantom gap
    on Oct 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent/qualityApproved by a Hive merger/owner for auto-merge on green CIblocked-on-reporterPaused until the e2e coverage reporter fix (#1079) lands; may be a phantom gapqualityApproved by a Hive merger/owner for auto-merge on green CItestingApproved by a Hive merger/owner for auto-merge on green CI🐝 covered by PRHive verified that an open PR references or claims this issue; still actionable until confirmed

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions