Repository navigation
[quality] the member directory freshness note is rendered from three scheduled data files and asserted by no e2e test #1074
Description
Finding
/community/members renders DirectoryFreshness
(src/components/MemberDirectory/DirectoryFreshness.js), the provenance note
that tells a reader how current the directory is:
Directory membership data last synced from cncf/landscape on October 1, 2026.
Architecture-derived profiles last synced from cncf/architecture on September 27, 2026.
Award data last verified on August 7, 2026.
Each of its three sentences is built from a different scheduled data file, and
each is guarded independently:
| sentence | source | field |
|---|---|---|
| membership | data/members.json |
sources.landscape.collectedAt |
| architectures | data/metrics.json |
generatedAt |
| awards | data/awards.json |
verifiedAt |
formatDate() (src/components/MemberDirectory/utils.js:14-22) returns null
for anything new Date() cannot parse, and every sentence sits behind a
&& on its own date (lines 22, 36, 50). A refresh that writes a malformed or
absent timestamp therefore makes that sentence disappear silently — and if
all three go, line 19 returns null and the whole note vanishes.
Nothing in tests/e2e/** asserts any of it. grep -rn 'last synced from' tests/e2e/ returns nothing, and neither tests/e2e/member-directory.spec.js
nor tests/e2e/interactions.spec.js looks at the note.
Why the existing unit tests do not cover this
tests/member-directory.test.mjs:367 and :420 do assert the note, but they
drive the component through tests/tools/fake-dom.mjs with injected
fixtures. They assert that given a good timestamp the sentence renders. They
cannot fail when the committed data/*.json carries a bad one, because they
never read it. The three files are rewritten on schedules
(.github/workflows/refresh-*.yml, npm run generate:members,
npm run collect:metrics), so the failure mode is a data change with no source
change at all — exactly the case the unit suite is blind to.
This is the same reasoning the repository already applied to the analogous
PeopleFreshness note, which is guarded end to end at
tests/e2e/community-people.spec.js:116-122, with the rationale written into
that spec's header comment:
The roster is also rebuilt on a schedule from
data/community-people.jsonby
npm run fetch:community-people, so it can go blank with no source change at
all. These cases assert only that it renders.
DirectoryFreshness has three such sources instead of one and no equivalent
guard.
This is not a coverage gap
Stated explicitly so the finding is not mis-filed: the note renders on every
/community/members visit, so its regions are already counted covered by both
suites. There is no uncovered region to chase here. What is missing is an
assertion — the output is produced and never checked.
Recommendation
Add an end-to-end case that loads /community/members and asserts the rendered
note against the committed data, rather than against a fixture:
- Assert the note is present and that each of the three sentences appears
whenever its backing timestamp is present in the committed JSON, reading
those files at run time so a scheduled refresh cannot make the test stale. - Assert the two provenance links (
cncf/landscape,cncf/architecture)
carry absolutehttps://hrefs and open in a new tab, so a malformed
sourceUrl/repositoryis caught as well as a malformed date. - Put it in a new spec file rather than
tests/e2e/member-directory.spec.js,
which open PR test: cover the member directory's four undriven toolbar controls #1060 is currently editing.
Evidence and provenance
- E2E: local reproduction of CI job
End-to-end coverageat revision
900592b—node tests/tools/e2e-coverage-run.mjs init,
npm run build:e2e:coverage,npm run test:e2e:coverage(298 passed,
chromium, node v26.10.0), seal, thennpm run report:e2e:coverage. A browser
probe of that build confirms the note renders with all three sentences and
both links. - Unit:
npm run test:unit:coverageat the same revision —src files
100.00% lines / 99.84% regions.
Completion criteria
-
/community/membershas an end-to-end case asserting the directory freshness note against the committed data - Each of the three sentences is asserted whenever its backing timestamp is present
- The
cncf/landscapeandcncf/architecturelinks are asserted to be absolute https and open in a new tab - The case lives outside
tests/e2e/member-directory.spec.js
Priority
- Impact: medium (a scheduled data refresh can blank the directory's only provenance signal with every test still green)
- Effort: low
🐝 Hive Agent: quality | SHA: 900592b
🐝 Hive Agent: quality | Instance: hosted-available-lke648397-260827-5n31 | SHA: 900592b
— hive: agent=quality backend=copilot model=claude-opus-5 copilot=1.0.88
Activity