Skip to content

Palette: fold the two rarest interaction types, and hold the UMAP palette to the same floor (#532, #601, #602) - #600

Merged
realmarcin merged 3 commits into
mainfrom
fold-rare-interactions-532
Aug 15, 2026
Merged

Palette: fold the two rarest interaction types, and hold the UMAP palette to the same floor (#532, #601, #602)#600
realmarcin merged 3 commits into
mainfrom
fold-rare-interactions-532

Conversation

@realmarcin

Copy link
Copy Markdown
Contributor

Closes #532. Unblocks #199. Implements the decision: fold the two rarest interaction types into "Other".

The constraint this removes

tests/test_network_palette.py stated it in its own docstring:

no nine-colour set survives all three deficiency types at once

The palette shipped with four documented tritan collisions as the price, the worst being CROSS_FEEDING vs MUTUALISM at ΔE 1.8 — the two commonest interaction types in the KB (252 and 229 occurrences), indistinguishable to a tritanope.

Seven colours do survive.

Folding is the enabler, not the fix — worth being precise

The two rarest types are STRAIN_COMPETITION (1 occurrence) and PREDATION (11, against 67 for the next-rarest coloured type). The failing pair was the two commonest. So folding does not directly repair anything.

What it buys is the magenta region those two occupied — and that is exactly where MUTUALISM had to move, because under tritanopia every blue converges on the teal of CROSS_FEEDING. The fold and the re-step are one change.

MUTUALISM #56bbe6#9f2ebb.

Measured, with the module's own Viénot–Brettel–Mollon projection

check before after floor
normal / protan / deutan minimum 15.4 / 17.0 / 15.7 15.4 / 17.0 / 15.7 15.0
CROSS_FEEDINGMUTUALISM under tritan 1.8 68.4
tritan collisions between interaction types 2 0
documented tritan collisions total 4 1 was ≤4
contrast light / dark 5.86:1 / 3.13:1 1.9

The single remaining collision is SYNTROPHY against the grey taxon node — which shape already separates, since taxa are circles and every interaction is a non-circular symbol (#270). The test now asserts zero collisions between interaction types and ratchets the documented total from 4 to 1.

The enum is untouched

All nine values remain — records still use them, and folded types render with the grey "Other" swatch the template already had for unmapped keys. FOLDED is explicit and reasoned, so a newly added enum value still fails rather than silently rendering grey, which is the defect this module was written to catch.

test_folded_types_are_still_the_rare_ones re-derives the counts from the KB, so the justification cannot rot: if a folded type becomes common, or a coloured type becomes rarer than a folded one, it goes red.

What I could not verify, and why the issue's proposal wasn't taken as offered

The dataviz rubric's OKLab validator is not present in this environment (scripts/ in the bundled skill is empty). That is the check that reported the original ΔE 10.3 normal-vision failure, and it should be re-run when available.

The two metrics genuinely disagree on that pair: CIE76 puts #56bbe6#57c7ab at 41.1 while the rubric's OKLab put it at 10.3. Separately, the issue's proposed re-step #edab12 → #b8860b fails this module's deutan floor at ΔE 1.5 against COMMENSALISM — so it could not be adopted, and I searched for a replacement that satisfies the gate the repo actually runs.

Mutation checks

reverted result
MUTUALISM back to #56bbe6 1 failed (tritan collisions)
PREDATION removed from FOLDED 1 failed (enum coverage)
FOLD_HEADROOM 2 → 200 1 failed (usage justification)
nothing 10 passed

Gates

lint 0, validate-all 0, validate-strict 0, 2465 passed. Regenerated HTML for all 309 affected pages is included.

realmarcin and others added 2 commits August 14, 2026 21:45
…a too (#532)

Decision on #532: fold the two rarest interaction types into "Other".

The module docstring of test_network_palette.py stated the constraint that made
this necessary: "no nine-colour set survives all three deficiency types at
once". The palette shipped with four documented tritan collisions as the price,
and the worst was CROSS_FEEDING vs MUTUALISM at delta-E 1.8 — the two commonest
types in the KB, indistinguishable to a tritanope.

Seven colours do survive. Folding is what makes it possible, but it is not by
itself the fix, and it is worth being clear about that: the two rarest types are
STRAIN_COMPETITION (1 occurrence) and PREDATION (11, against 67 for the
next-rarest coloured type), while the failing pair was the two commonest. What
folding buys is the magenta region those two occupied — which is exactly where
MUTUALISM had to move, because under tritanopia every blue converges on the
teal of CROSS_FEEDING.

MUTUALISM #56bbe6 -> #9f2ebb. Result, measured with the module's own
Vienot-Brettel-Mollon projection and CIE76 distances:

  normal 15.4  protan 17.0  deutan 15.7   (floor 15.0)
  CROSS_FEEDING/MUTUALISM under tritan: 1.8 -> 68.4
  tritan collisions between interaction types: 2 -> 0
  contrast 5.86:1 on light, 3.13:1 on dark (floor 1.9)

The one remaining tritan collision is SYNTROPHY against the grey taxon node,
which shape already separates: taxa are circles and every interaction is a
non-circular symbol (#270). The test now asserts zero collisions BETWEEN
interaction types and ratchets the documented total from 4 to 1.

The enum keeps all nine values — records still use them, and folded types render
with the grey "Other" swatch the template already had for unmapped keys. FOLDED
is explicit and reasoned so a newly added enum value still fails rather than
silently rendering grey, which is the defect this module was written for.

test_folded_types_are_still_the_rare_ones re-derives the usage counts from the
KB, so the justification cannot rot: if a folded type becomes common, or a
coloured type becomes rarer than a folded one, it goes red.

Not verified here: the dataviz rubric's own OKLab validator, whose script is not
present in this environment. That check reported the original delta-E 10.3
normal-vision failure and should be re-run when available. The two metrics
disagree on that pair — CIE76 puts it at 41.1 — and the issue's proposed re-step
(#edab12 -> #b8860b) fails THIS module's deutan floor at 1.5 against
COMMENSALISM, so it could not be taken as offered.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
#199's checklist records the UMAP page's colours as "CVD-safe 8 + grey Other".
Measured with this repo's own colour code, they were not:

  normal  22.0 (0 pairs below 15)
  protan   6.9 (4)   <- LIGNOCELLULOSE/METHANOGENESIS, red/green convergence
  deutan  10.4 (3)   <- incl. BIOTECHNOLOGY against the collapsed grey
  tritan   5.8 (3)

The network palette has been gated at delta-E >= 15 for normal/protan/deutan
since #269. The UMAP palette was hand-picked, asserted CVD-safe, and read by no
test — the same shape as #588, a rule applied to one artefact and not its
sibling.

Re-stepped, keeping every hue's semantic assignment (blue rhizosphere, green
lignocellulose, red methanogenesis) and separating on lightness instead, which
is what red/green convergence under protanopia actually requires:

  normal  31.1 (0)   protan 16.0 (0)   deutan 16.7 (0)   tritan 11.0 (4)

Every gated metric improves. Tritan has one more pair under 15 than before but
its WORST pair nearly doubles, 5.8 -> 11.0: no two categories are now
indistinguishable, which matters more than the count. Chosen over a variant
scoring 17.4 on the gated metrics precisely because that one dropped tritan's
worst pair to 2.2.

Both palettes are now gated by one module. The parser is scoped to
`categoryColors`; a first version matched every hex in the file and dragged in
`stateColors`/`originColors`, which belong to other colour-by modes and are
never on screen together — it invented three failures.

#602: check-docs-current ran only `gen-html`, so docs/community_umap.html could
drift from its template forever while the gate stayed green — demonstrated by
this very change, where the template had the new colours and the published page
still served the old ones after a green run. `gen-umap` now runs there too; the
embedding is seeded, so two runs are byte-identical and this cannot flake.

Mutations checked: restoring the old palette reddens protan and deutan;
breaking the parser reddens the shape guard rather than passing silently.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@realmarcin realmarcin changed the title Fold the two rarest interaction types so the palette clears tritanopia too (#532) Palette: fold the two rarest interaction types, and hold the UMAP palette to the same floor (#532, #601, #602) Aug 15, 2026
@realmarcin

Copy link
Copy Markdown
Contributor Author

Added to this PR: #601 and #602

Working #199 (which #532 unblocks) turned up two more instances of the same pattern.

#601 — the UMAP palette was asserted CVD-safe, not computed

#199's checklist records the UMAP page's colours as "CVD-safe 8 + grey Other". Measured with this repo's own colour code:

before after
normal 22.0 (0 pairs below 15) 31.1 (0)
protan 6.9 (4) 16.0 (0)
deutan 10.4 (3) 16.7 (0)
tritan 5.8 (3) 11.0 (4)

The worst pair was LIGNOCELLULOSE/METHANOGENESIS at ΔE 6.9 — the classic red/green convergence under protanopia — and BIOTECHNOLOGY collided with the collapsed grey at 10.4 under deuteranopia, which is a category becoming indistinguishable from the "everything else" bucket.

Re-stepped keeping every hue's semantic assignment (blue rhizosphere, green lignocellulose, red methanogenesis) and separating on lightness, which is what red/green convergence actually requires.

On the tritan trade: one more pair sits under 15, but the worst pair nearly doubles, 5.8 → 11.0. No two categories are now indistinguishable, which matters more than the count. I picked this over a variant scoring 17.4 on the gated metrics precisely because that one dropped tritan's worst pair to 2.2.

Both palettes are now gated by one module — the defect was the same as #588: a rule applied to one artefact and not its sibling.

#602 — the docs gate could not see the UMAP page

check-docs-current ran only just gen-html, but docs/community_umap.html comes from just gen-umap. So that page could drift from its template indefinitely and the gate stayed green, because nothing regenerated it to produce a diff.

Demonstrated by this very change: after editing the template and running gen-html, the gate reported ✅ while the published page still served 'RHIZOSPHERE': '#2a78d6'.

That is the failure mode the recipe's own #442 comment describes — "green in exactly the case where the regeneration it gates on had not happened". The orphan-page check added then fixed one instance, not the pattern. gen-umap now runs there too, and the invariant is written down: every generator whose output is committed under docs/ must run in that gate.

Safe to add — checked that the embedding is seeded, so two runs are byte-identical and this cannot flake.

One parser bug caught in my own work

The first version of _umap_palette matched every hex in the template and dragged in stateColors/originColors — separate schemes for other colour-by modes, never on screen together. It invented three failures. Now scoped to categoryColors, with a shape guard so a parser that returns {} after a refactor fails rather than passing vacuously.

CI rejected the first fix, correctly. `gen-umap` needs
data/embeddings/DeepWalkSkipGramEnsmallen_*.tsv.gz — a large local artefact
that is not in the repo — so adding it to check-docs-current failed the gate
outright:

  Error: Invalid value for '--embeddings-path': File '...tsv.gz' does not exist

That absence is presumably why the recipe was excluded in the first place, which
I should have checked before proposing to add it. My local determinism check
proved the embedding is seeded; it did not prove the input exists anywhere but
this machine.

The drift is still real and still worth gating, so it is gated a different way:
compare the category colours in docs/community_umap.html against the ones in
the template. That needs no embeddings, so it runs where regeneration cannot,
and it checks the property that actually matters — what a reader is served —
rather than the process that produces it.

The justfile now records why gen-umap is deliberately absent, so the next person
does not repeat the same reasoning and the same CI failure.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Network palette fails two computed checks, and dark mode reuses the light steps (#199)

1 participant