Skip to content

Keep TTA extras on busy pages; raise CF early-exit to 0.95 - #36

Merged
cursor[bot] merged 2 commits into
mainfrom
cursor/live-cdn-tta-agreement-f466
Aug 17, 2026
Merged

cursor[bot] merged 2 commits into
mainfrom
cursor/live-cdn-tta-agreement-f466

Conversation

@felirami

@felirami felirami commented Aug 17, 2026 •

Copy link
Copy Markdown
Collaborator

Why Unsplash still failed on v1.3.2

A live pass of the public v1.3.2 zip on the Unsplash featured feed scored 15 AI of 26 badges (ordinary editorial photos at 81-94 percent, raw p(AI) equal to neural). Wikipedia photos stayed clean. The published 3.0 percent held-out live-CDN guard did not match that feed.

Two production behaviors caused it. Neither remaps 0.65.

  1. Center-only load-shed. ttaModeForLoad returned center when more than 12 images were pending. A 26-image masonry scored the first ~14 images on the official 440 center crop alone, so agreedMax never ran. A lone 0.81-0.94 center crop became the AI verdict. The 132-image guard was scored with harness --tta=adaptive, so it never saw this path.
  2. Early-exit at 0.85. Even when adaptive TTA ran, a center crop of 0.85-0.94 stopped extras and kept single-view authority. The two FPs on the new stand-in were exactly this: livecdn-121 center 0.902 (runner-up 0.399) and livecdn-142 center 0.926 (runner-up 0.588).

Adaptive already skips extras on confident reals (center below 0.15). Shedding to center-only only changed behavior when the center crop was suspicious: exactly the band that needs extras for agreement.

Change

  • ttaModeForLoad always returns adaptive. Busy pages keep view agreement.
  • TTA_EARLY_EXIT moves from 0.85 to 0.95. A lone 0.81-0.94 crop must gather extras and survive agreement. Two agreeing high views, or a single view at or above 0.95, still carry the verdict.
  • After an agreement fallback, DINO must be >= 0.96 to lift. A 0.76 probe on a 0.40 runner-up (livecdn-121) no longer re-flags the photo.
  • DEFAULT_THRESHOLD stays raw 0.65. No Platt, no logit shift, no score remap.

Evidence (run in this checkout)

The published 893 / 240 / 132 / 100 fixtures are not in git. eval/fetch-live-guard.mjs built a stand-in: 39 held-out unsplash-lite rows (80-149) re-fetched as live imgix grid JPEGs (w=700 q=60 fit=crop) plus 16 DALL-E 3 images. Product and Flux sources returned zero files.

Offline comparison at raw 0.65, with harness DINO scores (eval/compare-tta-policy.mjs):

Policy Live-CDN FP TPR (n=16 DALL-E 3) BA (n=55)
Load-shed (center only) 2/39 (5.1%) 25.0% 59.9%
v1.3.2 (early-exit 0.85) 3/39 (7.7%) 43.8% 68.0%
Production 1/39 (2.6%) 43.8% 70.6%

TPR did not move on this 16-image DALL-E 3 set. The remaining production FP is livecdn-145 (CF 0.020, DINO 0.994), the existing strong-tier rescue, not introduced here.

Removing load-shed does not change the harness adaptive path that produced the last published 893-image row (87.9 BA / 76.0 TPR / 99.8 TNR). Raising early-exit may still cost TPR on images with a lone 0.85-0.94 view; that fixture was not re-scored. A full live Unsplash re-smoke of this build has not been published. README and the site say so.

npm test passes.

Docs

STORE.md, eval/README.md, docs/POIDH-323.md, AGENTS.md, and README now describe the 0.95 agreement band, the no-load-shed rule, and the fallback DINO gate. README does not claim a live pass we did not run.

Open in Web Open in Cursor 

cursoragent and others added 2 commits August 17, 2026 06:31
Unsplash featured-feed false positives at 81-94% were CF center crops
scored under center-only load-shed, which disabled view agreement. Adaptive
already skips extras on confident reals, so shedding only hurt the band
that needs corroboration. A lone view now needs extras through 0.95.
Threshold stays raw 0.65. The 893-image fixture is not in git and was
not re-scored.

Co-authored-by: felirami <felirami@users.noreply.github.com>
A disagreed 0.90 Unsplash center crop fell back to 0.40, then DINO 0.76
re-flagged it. After fallback, DINO must be >= 0.96 to lift. On the
runnable stand-in (39 Unsplash CDN + 16 DALL-E 3) fused live FPs go
from 3/39 to 1/39 at the same 43.8% TPR. Threshold stays raw 0.65.

Co-authored-by: felirami <felirami@users.noreply.github.com>
@felirami
felirami marked this pull request as ready for review August 17, 2026 06:41
@cursor
cursor Bot merged commit 18d5d23 into main Aug 17, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants