Repository navigation
Leave out images whose file carries an EXIF orientation - #18
Merged
Merged
Conversation
The browser turns an image by its EXIF orientation, and Open Images' boxes and Rotation value do not say whether they allow for it. build_manifest.py now leaves out any image whose file carries an orientation other than 1 and takes the next one in its split's ranking; the runner, render_copies.py and transforms.upright() refuse such a file. The orientation is read from the EXIF data only: Chromium ignores XMP's tiff:Orientation, which Pillow's getexif() also reports. No image of the current draw is affected. build_manifest.py also prints how many drawn files are larger than 1,024 px or carry EXIF data or an ICC profile, and the protocol and eval README no longer say that every CVDF copy is at most 1,024 px with all metadata removed. Follows Pratyush's review of #10. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The runner gives an image stored turned its own EXIF orientation tag in place of any EXIF data the file has, instead of refusing such a file, so every image the builder keeps can be served. The current draw is served byte for byte as before. - EXIF data that Pillow cannot read counts as 'unreadable', which the builder leaves out and the runner refuses, instead of a crash. Like Chromium, only the first EXIF segment counts (checked in Chromium). - check_colour.py runs the orientation check before Chromium starts. - The builder fetches and checks only images it has not checked before. - The refusal message no longer says the page would apply the tag, which is wrong for transforms served as PNGs. - The docs say which counts the builder prints; the made-up Open Images files carry EXIF data only where a test needs it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
06pratyush
approved these changes
Oct 28, 2026
06pratyush
left a comment
Collaborator
There was a problem hiding this comment.
Approving. I checked:
- the exclude-and-replace loop in
build_manifest.py: each image left out for its EXIF orientation is replaced by the next one in its split's ranking; - the refusals in
run_detect.py,render_copies.py,check_colour.pyandtransforms.upright(); - the runner replacing the EXIF data of an image stored turned with its own orientation tag;
- the test that Chromium turns an image by its first EXIF segment only.
The eval tests pass on Windows at the top of the stack (104, none skipped).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follows Pratyush's review of #10. Stacked on #17 (
eval-hashes); once #17 is merged, this PR moves tomain.What changes
eval/build_manifest.pyleaves out any image whose file carries an EXIF orientation other than 1, or EXIF data that Pillow cannot read, and takes the next image in its split's ranking, checking that one in turn. The browser turns an image by its EXIF orientation, and Open Images' boxes and Rotation value do not say whether they allow for it. No image of the current draw is affected:eval/manifest/is unchanged.6ae23606d22bad2d(3780 x 6111) andcce0d9c5636f322e(6000 x 4000), both with EXIF, IPTC, XMP and ICC data, and Rotation 0.run_detect.py,render_copies.pyandcheck_colour.pyrefuse such a file before the browser starts, whatever the transform, andtransforms.upright()refuses it too.transforms.exif_orientation()): from the first EXIF segment only, and not from XMP'stiff:Orientation, which Pillow'sgetexif()also reports. A test checks this in Chromium 156.docs/eval-protocol.md(part B: Data, Exclusions, Orientation and the tiled setting) andeval/README.mdno longer say that every CVDF copy is at most 1,024 px with all metadata removed, and list the new exclusion. No evaluation run has been made yet, so this is not a deviation from the protocol.Checks
.venv/bin/python -m pytest eval/tests: 104 passed.as_storedcondition, the EXIF-only reading, the replacement of EXIF data, theunreadablecase,check_colour.py's check) makes a test fail.eval/build_manifest.pyon the real metadata leaveseval/manifest/byte for byte unchanged, and all 1,000 images passcheck_orientations()./code-review(max): 9 findings, all fixed in the second commit.🤖 Generated with Claude Code