Repository navigation
W11 WP 11.4: photosphere simulator tests run in CI and catch a stale fixture (#1156) - #1329
Merged
Merged
Conversation
…atch a stale fixture (#1156) A new workflow, photosphere-sim, runs tools/photosphere_sim/tests when the simulator or the pano scanner sources change. It installs the versions the fixture manifests record, uses no secrets and has read-only permissions. A new test rebuilds every committed fixture in headless Chromium and compares it with what is committed: frames byte-equal, or within 2 levels per channel; visibility.json byte-equal when the frames are, else the same mask and numbers within 1 percent. A failure names the manifest versions that differ. With CI set, a missing Chromium fails instead of skipping. The workflow is not yet a required check: wiring it into ci.yml's ci-ok gate follows once it is proven on a hosted runner. Mutant: the comparison returns no problems; 5 tests fail. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K7CHb44VFtZLGUDLF4us2p
…r image (#1156) setup-python has no 3.12.10 for the Ubuntu 26.04 image, so the first hosted run failed before any test. The other workflows pin '3.12'; this one now does too. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K7CHb44VFtZLGUDLF4us2p
…res were recorded (#1156) On ubuntu-latest the Chromium fixture rebuild matched, but 13 older tests that byte-compare float JSON failed in the last digits (the platforms' maths libraries round differently). The suite is graded on the recording platform until those comparisons take a tolerance. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K7CHb44VFtZLGUDLF4us2p
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Plan 2, wave 11, WP 11.4.
The photosphere simulator's 607 tests ran in no workflow, and nothing ever rebuilt the committed fixture frames or
visibility.json.photosphere-sim: runs on changes totools/photosphere_sim/**, the pano scanner sources and the workflow itself;permissions: contents: read, no secrets, 30 minute timeout, versions pinned to what the fixture manifests record (Python 3.12.10, numpy 2.5.0, pillow 12.3.0, playwright 1.61.0, which downloads Chromium 149.0.7827.55).visibility.jsonbyte-equal when the frames are, otherwise the same visible mask and numbers within 1 percent. A failure names any manifest version that differs from the rebuild's.CIset, a missing Chromium is a failure, not a skip.This PR is the first run on a hosted Linux runner. The tolerances were measured only on the recording machine, where a rebuild is byte-identical. If this run is red on a rasteriser difference, the message says how many pixels differ and by how much.
Not in this PR: making it block a merge. A path-filtered workflow cannot be a required check, so it has to become a job of
ci.ymlbehind an area filter, reporting intoci-ok. That follows as its own change once this run is green.Local cost: the whole sim suite took 646 s on a shared machine; the fixture rebuild is about 1 minute of that. Estimated 12 to 15 minutes on a cold runner.
Mutant (the comparison returns no problems): 5 tests fail. A real 8 x 8 px edit to one committed frame also turned the rebuild test red.
Part of #1156.
🤖 Generated with Claude Code
https://claude.ai/code/session_01K7CHb44VFtZLGUDLF4us2p