[quality] static/robots.txt advertises no Sitemap, so the 43-URL sitemap.xml built on every deploy is unannounced #383

Finding

static/robots.txt is two lines and names no sitemap:

User-agent: *
Allow: /

Meanwhile preset-classic runs @docusaurus/plugin-sitemap 3.10.2 with default
options, so every build writes a sitemap nothing advertises. Verified at
00b44df, node v26.8.2, 2026-09-21:

$ SITE_URL=https://endusers.cncf.io BASE_URL=/ npm run build:production
[SUCCESS] Generated static files in "build".
$ ls -l build/sitemap.xml
-rw-r--r-- 1 ... 5643 build/sitemap.xml
$ grep -c '<loc>' build/sitemap.xml
43
$ grep -i sitemap build/robots.txt
(no output)

robots.txt is the one machine-readable location a crawler is specified to check
for a Sitemap: directive (RFC 9309 §2.2.3). Without it, discovery of all 43 URLs
falls back to link-following plus whatever is registered by hand in Search Console —
a setting that lives outside this repository, is invisible to review, and is
therefore exactly the kind of thing that is missing on launch day and noticed
months later.

This site is pre-launch (LAUNCH.md), which is when the directive is worth the most:
a fresh origin has no inbound link graph for a crawler to walk.

Why existing gates miss it

• Docusaurus copies static/ into build/ verbatim and never reconciles
  robots.txt against the sitemap it generates in the same build.
• npm run test:unit (55/55 green) contains no test that reads static/robots.txt.
• onBrokenLinks: 'throw' governs page routes, not crawler directives.
• Open PR test: cover the docusaurus.config.js and sidebars.js navigation contract (tests/site-config.test.mjs) #275 (tests/site-config.test.mjs) covers navigation and branding in
  docusaurus.config.js; it does not read static/robots.txt. Open PR test: verify local /img static asset references resolve under static/ (tests/static-assets.test.mjs) #289
  (tests/static-assets.test.mjs) is scoped to /img references. Open PR test: cover the static/manifest.json PWA install contract (tests/manifest-contract.test.mjs) #335
  covers static/manifest.json. No open PR touches static/robots.txt.

Recommendation

One line in static/robots.txt. Apply verbatim:

User-agent: *
Allow: /

Sitemap: https://endusers.cncf.io/sitemap.xml

Hardcoding the production origin is correct here rather than a limitation.
robots.txt is only honoured when served from an origin root, so the
BASE_URL=/endusers/ preview — which serves the file at /endusers/robots.txt —
is not a crawler-facing surface at all and needs no per-environment value.

Note this is independent of the sitemap's contents: six of the 43 URLs should not
be in it, tracked separately. Advertising the sitemap and correcting which routes
it lists are separate changes and either can land first.

Verify after the change:

SITE_URL=https://endusers.cncf.io BASE_URL=/ npm run build:production
grep -i sitemap build/robots.txt   # expect the Sitemap: line

Why there is no PR attached

static/robots.txt is production content, not test code, and the quality lane is
restricted to test-only PRs. This needs a human or another lane to land; the
replacement text above is given verbatim so applying it is mechanical.

Evidence

• Revision 00b44df, node v26.8.2, @docusaurus/plugin-sitemap 3.10.2, run locally 2026-09-21.
• static/robots.txt -> 2 directives, zero Sitemap: lines.
• build/sitemap.xml -> 5643 bytes, 43 <loc> entries.
• Unit coverage: npm run test:unit -- --experimental-test-coverage -> 55/55 pass;
  static/robots.txt is read by no test.
• End-to-end coverage: unobtainable rather than absent — no playwright, cypress
  or puppeteer is declared, so there is no e2e suite to consult. See [quality] CI publishes no coverage evidence, so coverage findings cannot be verified #186.

Priority

• Impact: low — discoverability only, and only until someone registers the sitemap by hand somewhere unversioned
• Effort: low — one line, no new dependency

Filed by quality agent (hold-gated mode)

— hive: agent=quality backend=copilot model=claude-opus-5