[CalvinAllen]

Description

For Python versions python-build-standalone has no Windows build for (72 windows-amd64, 19 windows-arm64), dtvem served python.org's embeddable zips. Those run in isolated mode because of a pythonXY._pth file: a script can't import modules next to it and PYTHONPATH is ignored. They also lack venv, ensurepip, C headers and tkinter.

This PR makes the mirror assemble a full build of the same release instead, on the existing Linux runner. It also makes the installer and dtvem doctor deal with the embeddable builds that remain.

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Code refactoring (no functional changes)
• Performance improvement
• Test coverage improvement
• CI/CD or build process change

Related Issues

Resolves #293
Relates to #113

Changes Made

Mirror (scripts/mirror-binaries), commit 1

• windows-amd64: python.org's per-component MSIs (core, exe, lib, dev, tcltk):
  • Each MSI is verified against its .asc signature using python.org's Windows signing key. The key is committed in keys/, its fingerprint is pinned in code, and verification happens in Go with ProtonMail/go-crypto, a dependency of the mirror module only.
  • The MSIs are unpacked with msiextract from msitools.
  • Discovery reads python.org's FTP index and emits jobs only for releases from 3.5.0 to 3.13.x whose amd64/ folder has all five MSIs with signatures. 3.14+ uses Sigstore only.
• windows-arm64: python.org's pythonarm64 NuGet packages. Each is verified against the SHA-512 in nuget.org's catalog, and only tools/ is kept.
  • x64 NuGet packages are deliberately not used: 3.5.2, 3.5.3 and 3.6.0 lack venv and ensurepip, while 3.5.4 and 3.6.8 are full, so there's no clean cutoff.
• Archive and validation:
  • Both are packed reproducibly (fixed mtimes, sorted entries) as a .tar.gz with everything under python/. That's python-build-standalone's Windows layout, so the client installs them unchanged.
  • Every archive is validated as a full install before upload: no ._pth, plus venv, ensurepip, headers, the import library, and tkinter for MSI builds.
  • An upstream build that fails validation is skipped, not published, and doesn't fail the run (errNotFullBuild).
• Mirror plumbing:
  • MirrorJob gains an optional Build step that's used in place of a download.
  • The new sources run after python-build-standalone, which still wins wherever it has a build.
  • Archives go to new .tar.gz keys. The old .zip objects aren't overwritten, because they're cached as immutable and released dtvem binaries reference them.
• Sync-only skip: --sync-only used to skip any job whose .meta.json existed, and the old zips' metadata already exists. An assembled job now also stays pending until its archive exists. This applies to Build jobs only, so other runtimes keep today's behaviour.
• Workflows:
  • mirror-sync.yml installs msitools for the Python job.
  • The new mirror-tool.yml vets and race-tests this module on PRs. It's a separate Go module, so Build & Test doesn't cover it.
• CLAUDE.md notes the new sources and workflow.

Client (src/), commit 2

• Installer: renames pythonXY._pth to pythonXY._pth.disabled instead of uncommenting import site. This turns isolated mode off for any embeddable build still installed. The only one left after the mirror runs is 3.5.2: its MSIs are unsigned and its NuGet package is incomplete.
• New dtvem doctor check python-isolated-mode: flags installed versions that still have a ._pth file. --fix applies the rename in place, and the resolution text recommends reinstalling for a full build. The shared logic is in a new internal/pth package.
• getPipURL: now uses the version-specific get-pip.py for 3.9 too. The main script requires 3.10+, so pip setup failed today for any 3.9 install relying on the get-pip.py fallback; released dtvem 0.6.5 fails the same way.

Testing

Test Environment

• OS: Windows 11 (client, end-to-end); Linux: ubuntu:24.04 / golang:1.25 containers (mirror)
• Go Version: 1.25.4 (modules target 1.23)
• dtvem Version: built from this branch; compared against released 0.6.5

Test Steps

1. Main module: gofmt clean, golangci-lint 0 issues, go test ./src/... all 27 packages pass. The new tests cover internal/pth, the doctor check (flag, fix, re-run OK, registered) and the getPipURL 3.9 case.
2. Mirror module:
   • go vet passes on Windows and on Linux (GOOS=linux).
   • go test -race passes in a Linux container.
   • golangci-lint reports 0 issues in new code.
   • Offline unit tests use an in-test OpenPGP key, httptest servers and a fake extractor. They cover signature checks (good, tampered, wrong signer, wrong pinned fingerprint), discovery for both sources, MSI and NuGet assembly (including URI-escaped names and dropped packaging files), reproducibility, rejection of embeddable or incomplete trees, pendingJobs, recordResult, fetchBinary and source order.
3. Real-data tests (MIRROR_NETWORK_TESTS=1, Linux container with msitools):
   • MSI builds: 3.5.0 (14.1 MB), 3.9.11 (18.2 MB) and 3.12.2 (17.4 MB), from real signed MSIs.
   • NuGet builds: ARM64 3.9.7 and 3.11.4.
   • Rejection: x64 NuGet 3.5.2 is correctly rejected.
   • Discovery: 99 signed x64 MSI releases and 66 ARM64 NuGet releases.
4. The real dtvem install, with no client changes, of the Linux-built 3.5.0, 3.9.11 and 3.12.2 archives on Windows 11. They were served locally through a seeded manifest cache in an isolated DTVEM_ROOT.
5. New installer against today's live mirror: installed the embeddable 3.9.9 with this branch and with 0.6.5.
6. dtvem doctor: ran against the 0.6.5 install.

Test Results

Converted archives installed with dtvem install:

Version	isolated	sibling import	PYTHONPATH	tkinter	Scripts\pip.exe / pip	python -m venv
3.5.0	0	✅	✅	✅ Tcl 8.6.4	✅ pip 7.1.2	✅
3.9.11	0	✅	✅	✅ Tcl 8.6.12	✅ pip 22.0.4	✅
3.12.2	0	✅	✅	✅ Tcl 8.6.13	✅ pip 24.0	✅

The embeddable 3.9.9 from today's mirror:

	dtvem 0.6.5	this branch
sys.flags.isolated	1	0 (python39._pth.disabled)
sibling import	❌	✅
PYTHONPATH	❌	✅
pip	❌ setup failed	✅
venv, tkinter	missing	missing (only the mirror conversion adds these)

dtvem doctor on the 0.6.5 install reported ⚠ Python versions run in isolated mode (embeddable builds) [fixable], listing Python 3.9.9: python39._pth, and its fix renamed the file.

Checklist

• My code follows the Google Go Style Guide
• I have run golangci-lint run and fixed all issues (includes gofmt, govet, and 40+ linters)
• I have run go mod tidy to clean up dependencies (mirror module: adds github.com/ProtonMail/go-crypto; main module unchanged)
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes (go test ./...)
• I have commented my code, particularly in hard-to-understand areas
• I have updated the documentation (README.md, CONTRIBUTING.md, etc.) if needed
• My changes generate no new warnings or errors
• I have checked my code works on multiple platforms (Windows, macOS, Linux) if applicable (mirror built and run on Linux; client on Windows; CI covers the rest)
• I have added appropriate error handling
• I have followed conventional commit format in my commit messages

Breaking Changes

None.

Rollout

The mirror change takes effect on the next Mirror Sync run (daily, or dispatch it for python).

• What it builds: about 71 x64 MSI builds (≈15–18 MB each) and the ARM64 NuGet builds python-build-standalone lacks.
• Then: run Generate Manifests from R2 to point the manifests at the new archives.
• Not tested: the R2 upload and the manifest regeneration need the workflow's credentials, so they're untested locally.

Additional Notes

• More Windows ARM64 versions: the mirror doesn't map python-build-standalone's Windows ARM64 builds today, and dtvem has only 10 full ARM64 builds. So the NuGet source also adds about 37 new ARM64 versions beyond the 19 it fixes, all validated full builds. Mapping python-build-standalone's own ARM64 Windows builds could be a follow-up.
• Differences from fix(mirror): replace Windows embeddable Python builds with full CPython builds #293's plan:
  • x64 3.5.2 can't come from NuGet (its package is incomplete), so it stays embeddable with ._pth.disabled.
  • The NuGet source is ARM64-only.
• A bug found while testing, not fixed here: doctor's stale-shims-path Windows fix rewrites HKCU\Environment\Path with SetStringValue (REG_SZ). A REG_EXPAND_SZ user PATH is converted, so entries like %USERPROFILE%\… stop expanding. It should preserve the value kind. Worth its own ticket.