Skip to content

CI runs the whole suite with zero renderer plugins loaded, and stays green #22

Description

@n1ckyb

What CI prints

WARNING intentumdiff.plugins.registry: Failed to load renderer plugin 'html': Failed to load plugin from '...\src\intentumdiff\wasm\html_renderer.wasm': file not found
WARNING intentumdiff.plugins.registry: Failed to load renderer plugin 'llm': ...
WARNING intentumdiff.plugins.registry: Failed to load renderer plugin 'patch': ...
WARNING intentumdiff.plugins.registry: Failed to load renderer plugin 'terminal': ...

All four renderers. Every run. And the suite passes.

Why

src/intentumdiff/wasm/*.wasm is gitignored (.gitignore:56) because the components are build artifacts, not source. A fresh clone therefore has none, and CI does not stage them.

When a renderer fails to load, plugins/registry.py:1058 logs a warning and simply omits the adapter from the returned list:

except Exception as exc:
    logger.warning("Failed to load renderer plugin %r: %s", ep.name, exc)
return adapters

No fallback, no failure, no record that the set is incomplete. So the whole suite runs against a build with zero renderer plugins, and reports green.

Locally there are 4 renderer .wasm files present, so this reproduces only in CI or a fresh clone — which is exactly the environment a user gets.

Why this matters more than the warning suggests

This is a coverage-integrity problem, not log noise. Every test touching --format html|patch|llm|terminal is, in CI, either skipped, exercising a Python fallback, or asserting on degraded output — and nothing distinguishes those cases from real coverage.

The failure shape is familiar: green CI that proves less than it appears to. It is what pulled 0.0.1 (every check green, four defects obvious on install), and the same shape as the parser estate not compiling for days behind CodeQL checks that were never builds.

A test suite that silently loses a whole plugin class is worse than one that fails, because the second is fixable in minutes and the first is invisible for months.

What to do

  1. Stage the wasm components in CI before running the suite — the same provisioning the wheel build performs. That is the actual fix.
  2. Fail loudly when the expected component set is incomplete. A count assertion at collection time (N renderers expected, M loaded) turns an invisible gap into a red build. Optional escape hatch (INTENTUMDIFF_ALLOW_MISSING_COMPONENTS=1) for genuinely component-free runs, so the default is safe and the exception is explicit.
  3. Mark tests that require a renderer so they skip visibly rather than passing against nothing.

Related: intentumdiff-core#23 (parity gate) hits the same staging problem from the Rust side — a harness that covers 5 of 365 cells while looking complete. Both want the same fix: CI stages the components, and a missing component is an error rather than a shrug.

Not release-blocking for 0.0.2 (the shipped wheel bundles the components; this is a CI-environment gap), but it should not survive another release.


Root cause found (2026-08-09) — it is a three-part cross-repo fix

Traced end to end. Confirmed still reproducing on the most recent successful run (31333276808, 20:03): 4 renderer-load failures, 0 parser-catalog failures. The parser half is fixed; the renderer half is not.

Why renderers are never staged:

  1. scripts/provision_build_inputs.py --from-parser-artifacts calls stage_wasm_from_artifacts(), which fetches components from the parser repos.
  2. Renderers are not built in parser repos. They are built in intentumdiff-corecrates/{html,llm,patch,terminal}-renderer — and build.py maps those crates to {html,llm,patch,terminal}_renderer.wasm.
  3. So there is no code path that fetches renderer components at all.
  4. The existing fail-closed check counts parser repos with no artifact. Renderers are not in that set, so zero missing parsers reads as a successful provisioning while four renderers are silently absent.

Why core cannot supply them today: core's Wasm components build (index engine + renderers) job runs cargo build --release --target wasm32-wasip2 and never uploads the result. There is no upload-artifact step, so the components exist only inside that job and are discarded.

The fix, in order

  1. intentumdiff-core — upload the built renderer components as a CI artifact from the existing components job. Small, additive, breaks nothing.
  2. intentumdiff-python — extend stage_wasm_from_artifacts() to fetch that artifact alongside the parser components.
  3. intentumdiff-python — extend the fail-closed check to cover the expected component set, renderers included, so a missing renderer stops provisioning with a legible message instead of surfacing an hour later as absent functionality.

Step 3 must come last. Adding it first would turn CI red immediately with no way to satisfy it, which blocks the release for a gap that is a CI-environment problem rather than a product defect — the shipped wheel bundles its components.

Not release-blocking for 0.0.2

The published wheel bundles the components; this affects what CI proves, not what users receive. But it means every renderer-touching test currently proves nothing, so it should not survive another release.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions