Skip to content

Test on every platform we publish a wheel for (today: build 4, test 1) #9

Description

@n1ckyb

The gap

publish.yml builds wheels on ubuntu-latest, macos-latest, windows-latest and windows-11-arm.
ci.yml runs the test suite on windows-latest, Python 3.12, only.

So three of the four artefacts we publish have never had a test run against them on the
platform they target. They are compiled and uploaded on the strength of a Windows test run.

This is the same shape of gap that produced the 0.0.1 pull: a check that proves the code
builds, standing in for a check that proves the artefact works.

Why it is not theoretical

The parts most likely to diverge by platform are exactly the parts we lean on:

  • ctypes loading of the core: .dll / .so / .dylib resolution differs per platform
  • path handling, which Windows-only testing systematically under-exercises
  • the wasm component cache directory and its permissions
  • console-script shims, which differ between Scripts/ and bin/

What to do

  1. Widen ci.yml to os: [ubuntu-latest, macos-latest, windows-latest] x
    python-version: ["3.10", "3.11", "3.12", "3.13"]
  2. Run scripts/smoke_published_wheel.py --wheel <built> on each matrix leg, against the
    wheel built by that leg — not against a source checkout
  3. Make the smoke run a required check before any tag

Definition of done

No wheel is published for a platform on which the full suite and the artefact smoke have not
both passed. If we cannot test a platform, we do not ship a wheel for it — an untested
artefact is a claim we have not earned.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions