Skip to content

Trust the package's own distribution name, and add the artefact smoke test - #15

Closed
n1ckyb wants to merge 1 commit into
release/v0.0.2-rcfrom
fix/first-party-trust
Closed

Trust the package's own distribution name, and add the artefact smoke test#15
n1ckyb wants to merge 1 commit into
release/v0.0.2-rcfrom
fix/first-party-trust

Conversation

@n1ckyb

@n1ckyb n1ckyb commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

The defect that forced 0.0.1 to be pulled from PyPI.

The package omitted its own distribution name from its first-party trust list, so all 78 bundled parsers were refused as untrusted third-party code and every invocation printed ~69 Failed to catalog parser plugin errors. Results were still correct, so the exit code said everything was fine.

No test in the repo could have caught it. In a source checkout the distribution resolves differently and the check passes. The bug exists only in an installed wheel, and nothing in CI ever installed one.

Also in this change:

  • src/intentumdiff/__main__.pypython -m intentumdiff failed outright; nothing invoked it
  • The error message pointed at docs.intentumdiff.dev, an unregistered domain. Now points at real documentation
  • scripts/smoke_published_wheel.py — installs the artefact into a clean venv and does what a user does in their first five minutes: install, import, console script, python -m, a real diff, clean stderr, and every URL resolves

That last script is the durable part. It closes the gap this PR exists because of: CI proves the code builds; only this proves the artefact works.

🤖 Generated with Claude Code

Every one of these was invisible to CI and obvious within thirty seconds of
installing the published wheel. Nothing in the pipeline ever did that.

1. THE ONE THAT BROKE THE PRODUCT. _TRUSTED_PLUGIN_PACKAGE_NAMES listed
   "intentumdiff" — the IMPORT name — but entry points report the DISTRIBUTION,
   which is "intentumdiff-python". So the package failed its own first-party
   check, treated all 69 built-in parsers as untrusted third-party plugins, and
   refused them with a security warning naming itself as the culprit. Every
   invocation printed ~69 error lines about "arbitrary code execution" before
   returning a result.

   No repo test could see it: in a source checkout the distribution resolves
   differently and the check passes. It exists only in an installed wheel.

2. `python -m intentumdiff` failed with "cannot be directly executed" — there
   was no __main__.py. READMEs reach for `python -m` constantly, and it is what
   people use when the console script is not on PATH.

3. The plugin error pointed at https://docs.intentumdiff.dev/plugins/metadata,
   a domain that is not registered — so it failed DNS, not merely 404. Now
   points at docs/PLUGIN_GUIDE.md, which exists.

4. The README's headline example could not run: `old` and `new` were never
   defined, so a user copying it got NameError on line 3. It is now a complete,
   runnable program, and the output shown is its real output.

Adds scripts/smoke_published_wheel.py, which installs the published artefact
into a clean venv and uses it as a user would: install, import, console script,
python -m, a real diff, CLEAN STDERR, and every URL in the output resolving.

Run against 0.0.1 it fails three checks — exactly the ones above. That is the
point: it reproduces the shipped bugs rather than asserting they are absent.

Verified: the README example was extracted from the file and run verbatim
against a patched install; it prints the documented output.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@n1ckyb

n1ckyb commented Aug 8, 2026

Copy link
Copy Markdown
Contributor Author

Do not merge yet — this fix exposes a deeper problem

CI: 20 failed, 2194 passed. The failures are caused by this change, and they are not
stale test data. The RC branch is green (the docs-only #11 passed the same suite on the same
base), so these are new.

Two representative failures:

test_certified_python_change_path_uses_rust_finalizer_not_python_engine
  AssertionError: assert 'rust_finalize_review_v1' == 'rust_finalized_v1'

test_intent_facts_are_sufficient[python-validator]
  facts['behavior_category'] = None, expected 'validator'
  Full facts: {'param_count': 1, 'returns': 'value', 'return_kind': None,
               'body': 'substantive', 'side_effects': None, ...}

What this means

Until now every parser was refused as untrusted, so every diff took the fallback path
and that is the path the suite has been validating. Trusting the parsers moves execution onto
the certified path for the first time, and on that path the intent facts come back
less populated: behavior_category, return_kind and side_effects are all None
where the fallback produced real values.

So the 78 catalogue errors were not merely noise. They were routing every user away from the
certified path, and the certified path has a gap in fact derivation that no test could see,
because no test ever reached it.

That reframes 0.0.1: the suite was green because it was exercising the fallback. Fixing the
trust list is necessary but not sufficient — it makes the real path reachable, and the real
path is not yet correct.

What has to happen before this merges

  • Establish why behavior_category, return_kind and side_effects are None on the
    certified path. Facts derivation lives in the Rust core, so this is very likely an
    engine gap rather than a binding one
  • Decide whether rust_finalize_review_v1 is the correct finalizer for this path, or a
    routing mistake. If it is correct, that assertion is stale and should be updated —
    but only once we know, not to make the suite quiet
  • Re-run and confirm all 20 pass for the right reason

Related

@n1ckyb n1ckyb added the bug Something isn't working label Aug 8, 2026
n1ckyb added a commit that referenced this pull request Aug 9, 2026
* fix: land the verified 0.0.1 fixes in the repo that publishes the wheel

This repo publishes intentumdiff-python to PyPI, and it did not have the fixes
for the defects that got 0.0.1 pulled. They were stranded in PR #15 (now
conflicting) and #14. Two were missing outright:

- src/intentumdiff/__main__.py — `python -m intentumdiff` failed in 0.0.1
  because this file did not exist. It still did not exist here.
- scripts/smoke_published_wheel.py — the release gate itself: installs the
  artefact into a clean venv and checks install, import, console script,
  `python -m`, a real diff, clean stderr, and that every URL resolves.

The engine fixes, all verified against a green build in the experiments repo:

- rust_core.py: canonicalise the distribution-qualified parser plugin id. The
  wheel publishes as intentumdiff-python while the import package is
  intentumdiff, so the certified parser arrived as
  'intentumdiff-python:python:python' and failed an allowlist that only knew
  'intentumdiff:python:python'. The certified batch path declined with
  "unsupported parser plugin" and fell through to routed finalize — Rust to
  Rust, so no gate fired and nothing looked wrong, while the certification and
  the facts only that path derives were silently absent.
- registry.py: trust the package's own distribution name. THE 0.0.1 defect.
- loader.py: the OSV advisory notice is debug, not warning. It fired on every
  ordinary run and put alarming text on stderr beside correct results.
- _shared.py: `file a.py b.py` labelled both sides with the NEW filename, and
  claimed a "working tree" scope for diffs with no working tree.
- sources/: a source that compares two differently named files now says so;
  get_content returns one filename because that is what language detection
  needs, and both sides inherited it.
- core/models.py: facts_trace, populated only under INTENTUMDIFF_TRACE_FACTS.
- live_server.py: the perceptual asset_diff op.
- differ.py: display names applied at the boundary that knows both.

Verified before syncing: Rust 266 passed, extension 260 passed, Python suite
running. The experiments repo has no CI (Actions disabled on billing), so this
repo's own CI is the gate — which is the point of syncing here.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(build): resolve the core from the RC, not main

CORE_REF was "main". But main only moves when a release is cut, so a consumer
pinned to it cannot verify against an unreleased engine change — which is
exactly what a release candidate exists to allow.

That gap is not theoretical. The provenance tests in this PR assert facts_trace,
added to the engine, and failed in CI while passing locally: CI was building an
engine that predated the feature under test. The tests were right; the pin was
wrong.

Now follows release/v0.0.2-rc, and reads INTENTUMDIFF_CORE_REF so a one-off
build can point elsewhere without editing the file.

Tracking a branch does make the build unreproducible, so this becomes a TAG the
moment intentumdiff-core cuts one. The comment says so where the next person
will read it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* release: 0.0.2b1

PEP 440 spells the prerelease 0.0.2b1 — no hyphen, no dot — where Cargo and
npm use 0.0.2-beta.1. Same release, three spellings, and the publish workflow
asserts the tag matches this string exactly.

Beta because 0.0.1 was published without anyone installing the artefact and
using it. A version without a beta marker is a claim of stability, and that
claim follows use rather than preceding it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test: derive the version instead of hardcoding it

Three tests asserted the literal "0.0.1", so bumping to 0.0.2b1 failed them —
for a reason that has nothing to do with what any of them checks. Two assert
CLI BRANDING and one asserts IMPORT PURITY; none is about which version is
current.

A hardcoded version in a test is the same defect the release skill names for
workflows: "derive the expected version, never a literal. A hardcoded version
goes stale silently and fails at the very end of a release, blaming the
artefact when the gate was wrong."

They now read __version__. Importing it into the purity test is safe: that
test's subject is a subprocess, so the parent's imports are not what it
measures.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: n1ckyb <nicknuxton@icloud.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
@n1ckyb

n1ckyb commented Aug 9, 2026

Copy link
Copy Markdown
Contributor Author

Closing as superseded — but not because this PR was unnecessary. It was carrying a fix the release branch needed, and the fact that it sat here behind a conflict is the more interesting part.

Measured against release/v0.0.2-rc today

File vs RC
src/intentumdiff/__main__.py identical blob — landed via dfbad9e (#16)
src/intentumdiff/plugins/registry.py identical blob — same
README.md differs — this PR's version runs (exit 0); the RC's was broken
scripts/smoke_published_wheel.py differs — this PR has no README gate; the RC now does

The part worth recording

The RC's README.md had its triple quotes collapsed to single quotes, so the headline example died with SyntaxError: unterminated string literal at parse time — the first code a user copies, rendering on PyPI. That is 0.0.1 defect #4 recurring.

The correct README was in this PR the entire time. It was found and fixed again from scratch in #19, because a conflicting PR on the release branch is invisible to everything: CI does not run it, the release checklist does not read it, and nobody diffs an open PR against the branch it targets.

The likely mechanism is a bad conflict resolution putting a mangled README on the RC while the good copy stayed stranded here.

Why closing rather than rebasing

  • Two of the four files are byte-identical to the RC — no content to land.
  • The README fix is equivalent to what The README example doesn't run, and the gate tests a private copy of it #19 landed; both produce exit 0 and the documented output.
  • This PR's smoke script is now behind the RC. Merging it would remove the gate that executes the README, which is the check that stops this recurring.

#19 carries both the repair and the gate.

Follow-up

The release skill now requires draining open PRs that target an RC — especially conflicting ones — before tagging, since this one proved a stranded PR can hold the fix for a defect that ships.

@n1ckyb n1ckyb closed this Aug 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant