Repository navigation
Eval: per-face luma and the darkest-tenth breakdown - #14
Merged
Merged
Conversation
face_luma.py measures each face's mean luma (BT.601, as JPEG stores it) over the pixels whose centres lie inside its box, on the image as Chromium decodes it, and writes eval/manifest/luma.csv. score.py reads it and breaks the miss rate down into the darkest tenth of the faces scored and the rest. Part B of the protocol now gives the definition. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
score.py stops when luma.csv exists but holds no rows, as it does for any other mismatch with faces.csv, and a test now covers the darkest tenth's tie-break by image and face ID. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
shashwat-deep
marked this pull request as ready for review
October 28, 2026 04:41
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
06pratyush
approved these changes
Oct 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Next step of the evidence half (
docs/plan.md, "Your half", item 4): the measured-light breakdown, "the darkest tenth of faces by mean luma inside the box, against the rest" (protocol part B). Stacked on #13, which is stacked on #12, #10, #9 and #2; merge those first.What is in it
eval/face_luma.pymeasures each face's mean luma and writeseval/manifest/luma.csv, one row per face offaces.csv. Each mean is taken:transforms.upright()from Eval: rotation and darkening transforms in the runner #12;eval/score.pyreadsluma.csvwhen it exists and addsmissRateBy.luma: the darkest tenth (the ⌈n/10⌉ faces with the lowest mean luma among the n faces scored, ties broken by image and face ID) against the rest. Both come with bootstrap intervals, plus the cutoff luma. The per-face CSV gainslumaanddarkestTenth. Aluma.csvthat doesn't matchfaces.csvstops the scoring, with a message to rerunface_luma.py.eval/manifest/luma.csvfor the current draw (1,797 faces, 12 s to make).5 tests:
luma.csvmatchesfaces.csv.77 eval tests pass in total.
On the current draw (before the review)
Among the 1,298 report-split faces that aren't depictions, the darkest tenth is 130 faces with mean luma at or below 56.2. The median face is at 109.5 and the darkest at 8.1. The review will change the draw a little, so rerun
face_luma.pyafter it (the test catches a mismatch).For both of us to agree
Part B of the protocol now defines luma (BT.601 on the decoded image), "inside the box" (pixel centres), and the darkest tenth (⌈n/10⌉ of the faces scored, ties by ID).
🤖 Generated with Claude Code