check-docs-current does not regenerate the UMAP page, so it can drift indefinitely

[realmarcin]

Found while fixing #601.

check-docs-current runs just gen-html and then fails if docs/ has a diff. But docs/community_umap.html is produced by a different recipe — just gen-umap (communitymech generate-umap) — which the gate never runs.

So the published UMAP page can drift arbitrarily far from src/communitymech/templates/community_umap.html and the gate stays green, because nothing regenerates it and therefore nothing produces a diff.

Demonstrated, not theorised

While re-stepping the category palette for #601 I edited the template, ran just gen-html, and check-docs-current reported ✅ docs/ matches the KB — while the published page still served the old colours:

$ grep "'RHIZOSPHERE'" docs/community_umap.html
'RHIZOSPHERE': '#2a78d6'      <- the old value, after gen-html + a green gate

Only an explicit just gen-umap updated it.

This is the gate's own documented failure mode

The recipe carries a comment from the #442 review about exactly this shape of bug:

render_all used to swallow a per-record exception, leave the previous page in place, and still print "✅ Rendered 312" — so the tree had no diff and this gate reported it current, green in exactly the case where the regeneration it gates on had not happened

Same sentence applies here with gen-umap substituted. The orphan-page check added then was a fix for one instance of the pattern, not the pattern.

The fix is safe

The obvious worry is that a UMAP/PaCMAP embedding is stochastic, so gating on its output would be flaky. It is not — checked by running just gen-umap twice and diffing:

DETERMINISTIC — two runs identical

So check-docs-current can simply run gen-umap alongside gen-html; the existing git status --porcelain docs/ check already covers the file.

Suggested

• add just gen-umap to check-docs-current
• and, since this is the second instance of the pattern, state the invariant in the recipe: every generator whose output is committed under docs/ must run here, so adding a generator without adding it to the gate is a visible omission rather than a silent one.

[realmarcin]

Correcting this issue — the proposed fix was wrong, and CI said so

I filed this proposing that check-docs-current simply run just gen-umap, and argued it was safe because the embedding is seeded. CI rejected it immediately:

Error: Invalid value for '--embeddings-path': File
'data/embeddings/DeepWalkSkipGramEnsmallen_degreenorm_embedding_512_v3_2026-06-26_12_55_27.tsv.gz'
does not exist
error: recipe `gen-umap` failed on line 375 with exit code 2

gen-umap needs a large embeddings artefact that is not in the repo. That is almost certainly why the recipe was excluded from the gate in the first place — a reason I should have looked for before proposing to add it.

My determinism check was sound but answered the wrong question. It proved the embedding is seeded and reproducible on a machine that has the input; it said nothing about whether the input exists anywhere else. Same shape as the canary rule: the check has to take the same path as the thing it is clearing.

What the fix is instead

The drift is still real — demonstrated in this PR, where a template edit reached the page's source and not the published page while the gate reported ✅. So it is gated by comparison rather than regeneration: test_published_umap_page_matches_the_template reads the categoryColors block out of both src/communitymech/templates/community_umap.html and docs/community_umap.html and requires them to agree.

That needs no embeddings, so it runs where regeneration cannot, and it checks the property that actually matters — what a reader is served — rather than the process that produces it.

The justfile now records why gen-umap is deliberately absent from the gate, so the next person does not re-derive the same idea and hit the same CI failure.

Still open, and now the more interesting question

The general invariant I claimed — every generator whose output is committed under docs/ must run in the gate — is not achievable for generators with heavy inputs. Anything else in that category is currently ungated the same way. Worth a sweep of what else under docs/ is committed but not regenerable in CI; I have not done that here.

[realmarcin]

Fixed in #600 (merged as fb8f829), though not the way this issue proposed — see the correction above. gen-umap cannot run in CI (it needs a large embeddings artefact absent from the repo), so the drift is gated by comparing the published page's categoryColors against the template's instead of by regenerating. The justfile records why the recipe is deliberately absent from check-docs-current.

The residual question I raised — what else under docs/ is committed but not regenerable in CI — is genuinely open and not addressed here.