[shashwat-deep]

Follows Pratyush's review of #13, and takes both of his suggestions. Stacked on #18 (eval-orientation).

What changes

• Crops only from the frozen set. eval/yaw_sheet.py refuses to run until review.csv keeps every image in images.csv, and deletes the crops and page of its last build before making new ones. Other files in the folder are left alone. The check of the review is now one function that the runner uses too (build_manifest.not_kept()). The page I built on the unreviewed draw for Eval: yaw labelling page and Cohen's kappa #13 went to a temporary folder outside the repo, not to data/yaw/. That folder has since been deleted, and no crops of the draw remain on my machine.
• No labels for faces nobody saw. The page ignores a key held down (e.repeat) and keys pressed with Ctrl, Alt or Meta, and a clicked label button gives up the focus, so Space or Enter cannot press it again. Only a copy of the whole box clears the unsaved-labels warning.
• Classes by head turn. Frontal: turned under about 30°; three-quarter: about 30° to 60°; profile: over about 60°; unclear. Sunglasses, hair or a hand over the eyes do not change the label. The page gives a cue for each class (where the nose points, how much of the far cheek shows). This changes part B of docs/eval-protocol.md before any labelling. The 30° and 60° boundaries are my choice, so please check them.
• Label files. Each author's file stays in data/yaw/yaw-<name>.csv (git ignores data/) until both are done. Then both are copied into results/yaw/ and committed together. yaw_sheet.py reads data/yaw/ by default and yaw_agreement.py reads results/yaw/.
• κ inputs recorded (rule 5). yaw_agreement.py writes results/yaw/agreement.json by default (--out beside the labels). It records the SHA-256 of both label files, the disagreements file, and images.csv and faces.csv as read, plus the git commit and the Python and numpy versions. It reads the commit before writing anything, so its own output never shows as an uncommitted change. If a label file is missing from results/yaw/, it says to copy both finished files there. The CLAUDE.md command now writes there without --out.

Checks

• .venv/bin/python -m pytest eval/tests: 104 passed. The yaw tests now drive the page in headless Chromium with a held key, Ctrl/Alt/Meta keys, a clicked button followed by Space and Enter, and a partial and a whole copy. They also check that the sheet refuses a manifest with an image removed or not yet reviewed, that it deletes old crops, and that the hashes are recorded.
• Removing each fix in turn makes a test fail.
• /code-review (max): 7 findings. Six are fixed in the second commit. One is left as it was: yaw_agreement.py imports run_detect only for git_commit(), as render_copies.py already does.

🤖 Generated with Claude Code

[06pratyush]

Approving. These all look right:

• the freeze check;
• the crop cleanup, which deletes only index.html and crop-named files, never a labels file;
• the key handling;
• the whole-box copy rule;
• the provenance in agreement.json, with the commit read before the script writes anything.

One change before labelling starts, which needn't hold up this PR: the page's cues don't match its angles.

• "Both cheeks show about equally" (frontal) fits a turn of about 10° or less.
• "The far cheek shows narrower" (three-quarter) is already true at 10° to 15°.

Take a simple head model: an ellipse 15 cm wide and 19 cm deep, with the face ending 7 cm either side of the midline. In it, the far cheek is 80% as wide as the near one at 7°, 62% at 15°, 36% at 30° and 9% at 60°.

Other head shapes move these figures. At 30°, the far cheek is 24% to 50% of the near one for ellipses 13 to 16 cm wide and 18 to 21 cm deep, and for round heads. But in every shape I tried, flat-fronted ones included, it is under 80% of the near one by 18°.

So a labeller who goes by the cues rather than the number would tend to put the frontal/three-quarter line nearer 10° than 30°. Faces turned 10° to 30°, which the protocol calls frontal, would then be labelled three-quarter. And if one of us goes by the number and the other by the cues, κ drops for a reason that has nothing to do with the faces.

Dropping only the frontal cheek clause isn't enough. A face turned 20° would then fit both "the nose points at or near the camera" and "the far cheek shows narrower". The protocol, the docstring and eval/README.md give only the angles. I'd make the page do the same, anchored by a reference strip:

• Frontal: under about 30°, turned less than in the 30° photo. Even a small turn makes the far cheek look narrower; that alone doesn't make a face three-quarter.
• Three-quarter: about 30° to 60°, between the 30° and 60° photos.
• Profile: over about 60°, turned more than in the 60° photo; the far side of the face is hidden, or nearly so.

For the strip:

• Photograph one of us at 0°, 15° … 90°, the steps of Multi-PIE's cameras and of AFW's hand labels.
• Turn the chair or move the camera, not the neck.
• Look straight ahead rather than at the camera, so the strip shows the turn of the head.

Where to keep it is our choice, and rule 3 allows either: it covers test images and photos with GPS data, and figures may show us.

• In data/, which git ignores, we'd share it by hand, and the repo wouldn't record which strip we labelled against.
• Committed, with its location data removed, it would be public.

Either way, part B should say that the crops are shown beside the strip.

The 30° and 60° boundaries themselves are fine:

• Equal thirds are easy to explain, and 3DDFA splits AFLW and AFLW2000-3D the same way (Zhu et al., CVPR 2016).
• By about 50° to 60° the nose hides most of the far eye, so "profile" keeps roughly the old one-eye meaning for an unobstructed face.
• Judging the head rather than the eyes is right too: sunglasses and hands are occlusion, not turn.