[06pratyush]

Stacked on #20 (the eval page): merge that first. With this, eval/run_detect.py --tiles 1920,0.25 runs instead of stopping at detectFaces, so part B can compare the single and tiled settings for YuNet.

What it does

detectFaces with tiling: { kind: "tiles", longSide, overlap }:

1. shrinks the photo so its long side is at most longSide (tiledSize()), by the same area averaging as the single setting, and never enlarges it;
2. cuts it into 640 × 640 tiles that start every round(640 × (1 − overlap)) px, 480 at 0.25, with the last one moved back to end at the edge, so no tile reaches past the image and neighbours overlap by at least 25%;
3. runs YuNet on each tile, keeping every box that scores 0.05 or more;
4. runs NMS once, over the boxes of all tiles in the shrunk image's pixels, then the score filter, so filtering the runner's boxes at a higher threshold still gives exactly the boxes found at that threshold.

A 4000 × 3000 photo becomes 1920 × 1440 and 12 tiles. An Open Images copy of at most 1,024 px is not shrunk and gives at most 4.

letterbox() is now resizeArea() plus tile(), which the tiled setting shares. The single setting's input is bit for bit what it was.

docs/contracts.md goes to 0.3: section 1 now says where the tiles lie and how their boxes merge. A contract change needs both of us, so approving this PR approves that too.

Tests

• Unit tests (Vitest):
  • tile starts for the protocol's sizes, and for every side from 1 to 2,000 px;
  • tiledSize() for portrait, landscape, rounding and never enlarging;
  • tile()'s padding, rounding in resizeArea(), and portrait letterboxing;
  • a fake detector over four tiles that see one square, two whole and two cut.
• Golden test. tests/reference/opencv_yunet_tiles.py runs OpenCV's FaceDetectorYN on each tile and merges the boxes with one NMS.
  • A separate Claude agent wrote it from a step-by-step description of the setting, without seeing this code. In its environment the existing single-setting reference reproduced byte for byte.
  • The eval page finds all 38 of its boxes, with the largest difference 0.0011 px and 7 × 10⁻⁷ of score in Chromium.
  • The test compares the 16 boxes scoring 0.10 or more (the next scores 0.0939): within 0.5 px in Chromium, and 2 px in Firefox and WebKit.
  • Boxes are now matched by position, not rank, since two of the 16 scores are 0.00046 apart.
• Never enlarged. The test photo drawn at 750 × 1000 gives the very same boxes with longSide 1000 and 4000. Most part-B images take this path, which the 4,000 px golden photo cannot reach.

The full suite passes in Chromium, Firefox and WebKit.

Checked before opening

/code-review, then four more agents: one on conformance with the contract and protocol, one mutation-testing in its own worktree, one on memory and time, and a skeptic for each finding. Fixed in the second commit:

• The never-enlarge rule was untested, and the one golden photo hid another mistake: it is portrait, so taking the height as the long side passed. The size is now tiledSize(), unit-tested on both shapes, and the eval-page test above covers never enlarging. One mistake no test catches: swapping the x and y scales in the final mapping. The test photo's two scales are both exactly 0.48. At 2048 × 1365 the swap would move boxes by under 0.4 px.
• The comment said a face that two tiles find is "kept once". With tiles alone, that holds only when the boxes overlap by more than nmsIoU. The comment and the contract now say so, and a unit test keeps a cut view of a face as a box of its own at IoU 0.3.
• Hardening:
  • unknown tiling kinds, and overlaps that put tiles under 1 px apart, are refused before any work;
  • the photo's full-size pixels are no longer kept during the model runs;
  • the shrunk planes are bytes, not floats.
• resizeArea() matches OpenCV's INTER_AREA except where an average falls exactly halfway. It rounds those up, while OpenCV's float32 sums land on either side.
  • At the test photo's factors (6.25 and 25/12) the two agree on every pixel.
  • At 1.6, which shrinks a 1,024 px copy to 640, 5.9% of pixels differ by 1. Boxes then move by under 0.1 px, and scores by up to 0.003.
  • The comments and tests/README.md now say this rather than implying an exact match.

Left as they are: the resize's float intermediate (69 MB for a 12 MP photo in this setting) and a new tensor per tile. Both matter only if the app or the bench page runs this setting on phones.

For Shashwat

1. Tile layout in the protocol. Part B says the tiles "overlap by 25%" but not where they start. The contract now does. Part B and the paper could say "cut into 640 px tiles that start every 480 px, the last in each row and column moved back to end at the image's edge".

2. Faces larger than the overlap. With tiles alone, a face more than 160 px across at the 1,920 px scale (a twelfth of the long side) may lie whole in no tile, and one more than 640 px (a third) never does. The test photo shows what that does.

   • Its face is 510 px tall at that scale. The tile above finds most of it at 0.909, and the tile below finds the lower part at 0.939.
   • The one NMS drops the 0.909 box, which overlaps the 0.939 one by IoU 0.346, and the face comes back as six boxes scoring 0.10 or more.
   • At the app's provisional settings (padding 0.2, threshold 0.5), the tiled boxes leave 3.6% of the face's core uncovered (15,124 of 422,500 px, a strip across the eyebrows). At padding 0, 17.2% is uncovered.
   • The single setting covers all of it at every threshold up to 0.9.

   The face-size breakdown will show what this costs across the set. A setting that keeps the gain on small faces without this loss would be tiles plus the whole image, merged by the same NMS. That would be one more Tiling kind, so a contract change.

3. Speed. A 12 MP photo takes 12 model runs in this setting, against 1 in the single setting. Measured in Node on my desktop, that is about 0.7 s against 0.13 s per photo; part D will measure the browsers.

🤖 Generated with Claude Code