Repository navigation
[quality] data/awards.json <-> AwardsTimeline render contract is untested; validate-awards.mjs is blind to 8 mutations that misrender /awards #359
Description
Finding
scripts/validate-awards.mjs is the only gate on data/awards.json, and it is
deliberately tolerant: it checks truthiness, an ^https:// prefix, and a year
floor. src/components/AwardsTimeline/index.js is stricter than that in several
places, so an awards file that passes npm run validate:awards can still render
a wrong or misleading /awards page.
No test asserts the data against the component. tests/validate-awards.test.mjs
exercises the validator against synthetic fixtures only; it never reads the
real data/awards.json, and nothing in tests/ reads
src/components/AwardsTimeline/index.js.
Evidence
Verified on a clean clone at 00b44df, node v26.8.2, 2026-09-20.
Nine single-field mutations were applied to data/awards.json, one at a time.
Eight of the nine pass npm run validate:awards:
| mutation | validate:awards |
what the page does |
|---|---|---|
verifiedAgainst repointed to another host |
PASS | misattributes its own audit source (see below) |
verifiedAt set to a future date |
PASS | banner claims an audit that has not happened |
citation set to a number |
PASS | renders 5 as the citation |
event set to " " |
PASS | renders a blank line |
slug set to Acme_Corp |
PASS | non-kebab React key |
| duplicate winner entry inserted | PASS | winner rendered twice, no warning |
announcementUrl set to https:// |
PASS | new URL() throws; non-navigable anchor |
two awardLabels for one award slug |
PASS | reads as two different awards |
out-of-range year (1999) |
FAIL | already caught |
The verifiedAgainst case is the sharpest. AwardsTimeline hard-codes the
visible link text:
<a href={awardsData.verifiedAgainst} target="_blank" rel="noreferrer">
contribute.cncf.io/community/awards
</a>The validator accepts any https:// URL for verifiedAgainst, so the href and
the label it is rendered under can drift apart and the page then misattributes
the provenance of its own winner history — exactly the claim the banner exists
to make.
Why existing gates miss it
npm run validate:awardsis the tolerant gate described above.docusaurus build/onBrokenLinks: 'throw'govern page routes, not JSON
field shapes or external hrefs.- There is no JSX transform under
node --test, so no test can import the
component (see [quality] filterArchitectures is untested, and no test can import any src/ React module #228); the contract has to be asserted against the component
source plus the real data. - Disjointness from open PRs: no open PR names
AwardsTimelineor reads
data/awards.jsontop-level fields. [quality] test: cover data/members.json contract and awards.json slug parity (tests/members-data.test.mjs) #253 (tests/members-data.test.mjs)
touchesawards.jsononly formembers.json↔ award slug parity — it
asserts nothing aboutverifiedAt,verifiedAgainst, ordering, duplicates,
URL resolvability, or label consistency, and does not read any component.
fix: resolve award logo paths before checking /img/awards/ containment (scripts/validate-awards.mjs) #329 claims award logo path resolution, which this issue deliberately
leaves alone. test: cover the data/metrics.json render contract (tests/metrics-data.test.mjs) #287 is the same species of contract formetrics.json↔
MetricsDashboard; test: assert the data/architectures catalog ↔ records ↔ docs contract (tests/architecture-catalog-contract.test.mjs) #260 is the catalog ↔ records ↔ docs contract. [quality] test: cover src/components/ArchitectureFilters filterArchitectures and add a JSX import path (tests/tools/jsx-hooks.mjs + tests/helpers-jsx.mjs + tests/architecture-filters.test.mjs) #229 covers
filterArchitectures, test: cover useFocusTrap scroll lock, tab cycling and focus restore #268useFocusTrap.
Coverage evidence
- Unit:
node --test --experimental-test-coverage, node v26.8.2, local run
at00b44df, 2026-09-20 — 55 pass. No unit coverage of this contract. - End-to-end: the repository exposes no end-to-end suite or coverage
artifact (package.jsonhas no browser/e2e runner;check:linksdepends on a
Makefile target that is absent — see fix: rewire check:links off missing Makefile and drop orphan typecheck script #291). End-to-end coverage is therefore
indeterminate, not absent, and no end-to-end claim is made here. That
reporting limitation is already tracked by [quality] CI publishes no coverage evidence, so coverage findings cannot be verified #186 and is not re-filed.
Recommendation
One test-only PR closes this. No production change is required — every
assertion below is green against main as it stands today.
- Add
tests/awards-data.test.mjsasserting the realdata/awards.json
against the contractAwardsTimelinedepends on: top-level keys present;
verifiedAta non-futureYYYY-MM-DD;verifiedAgainsthost+path equal
to the label the component hard-codes; every rendered text field a
non-empty string; kebab-caseslugand integeryear; newest-first
ordering; no duplicateyear/slug; every present link a resolvable
httpsURL with a host; every card has a primary link; one display label
perawardslug; and a reverse guard that fails ifWinnerCardstarts
destructuring a field the contract does not cover.
Priority
- Impact: medium — the
/awardspage is the site's public record of CNCF End
User award winners, and the failure mode is silent misattribution rather than
a broken build. - Effort: low — one new test file, no dependencies, no production change.
— hive: agent=quality backend=copilot model=claude-opus-5
Activity
test: cover the data/awards.json render contract (tests/awards-data.t…