Keep TTA extras on busy pages; raise CF early-exit to 0.95 - #36
Merged
Merged
Conversation
Unsplash featured-feed false positives at 81-94% were CF center crops scored under center-only load-shed, which disabled view agreement. Adaptive already skips extras on confident reals, so shedding only hurt the band that needs corroboration. A lone view now needs extras through 0.95. Threshold stays raw 0.65. The 893-image fixture is not in git and was not re-scored. Co-authored-by: felirami <felirami@users.noreply.github.com>
A disagreed 0.90 Unsplash center crop fell back to 0.40, then DINO 0.76 re-flagged it. After fallback, DINO must be >= 0.96 to lift. On the runnable stand-in (39 Unsplash CDN + 16 DALL-E 3) fused live FPs go from 3/39 to 1/39 at the same 43.8% TPR. Threshold stays raw 0.65. Co-authored-by: felirami <felirami@users.noreply.github.com>
felirami
marked this pull request as ready for review
August 17, 2026 06:41
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why Unsplash still failed on v1.3.2
A live pass of the public v1.3.2 zip on the Unsplash featured feed scored 15 AI of 26 badges (ordinary editorial photos at 81-94 percent, raw p(AI) equal to neural). Wikipedia photos stayed clean. The published 3.0 percent held-out live-CDN guard did not match that feed.
Two production behaviors caused it. Neither remaps 0.65.
ttaModeForLoadreturnedcenterwhen more than 12 images were pending. A 26-image masonry scored the first ~14 images on the official 440 center crop alone, soagreedMaxnever ran. A lone 0.81-0.94 center crop became the AI verdict. The 132-image guard was scored with harness--tta=adaptive, so it never saw this path.Adaptive already skips extras on confident reals (center below 0.15). Shedding to center-only only changed behavior when the center crop was suspicious: exactly the band that needs extras for agreement.
Change
ttaModeForLoadalways returnsadaptive. Busy pages keep view agreement.TTA_EARLY_EXITmoves from 0.85 to 0.95. A lone 0.81-0.94 crop must gather extras and survive agreement. Two agreeing high views, or a single view at or above 0.95, still carry the verdict.>= 0.96to lift. A 0.76 probe on a 0.40 runner-up (livecdn-121) no longer re-flags the photo.DEFAULT_THRESHOLDstays raw 0.65. No Platt, no logit shift, no score remap.Evidence (run in this checkout)
The published 893 / 240 / 132 / 100 fixtures are not in git.
eval/fetch-live-guard.mjsbuilt a stand-in: 39 held-out unsplash-lite rows (80-149) re-fetched as live imgix grid JPEGs (w=700 q=60 fit=crop) plus 16 DALL-E 3 images. Product and Flux sources returned zero files.Offline comparison at raw 0.65, with harness DINO scores (
eval/compare-tta-policy.mjs):TPR did not move on this 16-image DALL-E 3 set. The remaining production FP is livecdn-145 (CF 0.020, DINO 0.994), the existing strong-tier rescue, not introduced here.
Removing load-shed does not change the harness adaptive path that produced the last published 893-image row (87.9 BA / 76.0 TPR / 99.8 TNR). Raising early-exit may still cost TPR on images with a lone 0.85-0.94 view; that fixture was not re-scored. A full live Unsplash re-smoke of this build has not been published. README and the site say so.
npm testpasses.Docs
STORE.md, eval/README.md, docs/POIDH-323.md, AGENTS.md, and README now describe the 0.95 agreement band, the no-load-shed rule, and the fallback DINO gate. README does not claim a live pass we did not run.