fix(detection): stop scoring two signals that genuine browsers also produce - #26
Conversation
|
Holding this. The conjunction rule is unsafe and must not merge as written. Tested in a real Chrome 150 (not Playwright-driven, A genuine browser produces both artifacts, so the rule would block every Chrome visitor at 0.9. My reasoning that each signal has a separate false-positive path, and that requiring two closes both off, was wrong — they fire together on ordinary Chrome. Reworking: the underlying issue is that neither signal indicates automation in the first place. |
…roduce
`webdriver_configurable` and `chrome_runtime_missing` were reported as Playwright
artifacts and scored at 0.7 and 0.6 in the headless category. Neither indicates
automation.
Measured in Chrome 150 with navigator.webdriver === false, driven by a person:
signals reported: ["webdriver_configurable", "chrome_runtime_missing"]
WebIDL defines navigator.webdriver as configurable, so every browser reports the
descriptor exactly as a patched one does. chrome.runtime is only exposed to a page
when an extension declares externally_connectable matching it, so on an ordinary
site an ordinary Chrome has none. Both therefore fired on genuine Chrome, together,
giving those visitors a headless category score of 0.771 and roughly +0.116 on the
final score.
The client no longer reports either signal, and all three servers ignore them when
received, since deployed clients will keep sending them for a while.
`webdriver_deleted` and `playwright_globals` are unchanged. Deleting the property
or injecting framework globals is deliberate tampering that no shipped browser does.
Agent detection is unaffected: bench TPR stays at 97.33% with every captured class
at 100%, because automated browsers are identified by navigator.webdriver being
true, which is dispositive on its own. Human panel FPR stays at 0.00%.
Also fixes touch form submissions. The submit classifier consulted only keydown and
mousedown within 100ms of the submit event; touchstart was recorded in the event log
but never read, so a tap whose compatibility mousedown lagged read as a programmatic
submit, as did any click-to-submit slower than 100ms on a loaded device. It now
selects by time rather than by the last five entries, considers touchstart, and
allows 500ms.
Five end-to-end cases cover both directions of each change: a genuine Chrome's
signal set raises no headless detection and is allowed, deleted-webdriver and
injected globals are still detected, a tap-submitted form is not flagged, and a
genuinely programmatic submit still is.
c03d8dc to
cd47a4a
Compare
|
Reworked. The conjunction rule is gone; the change is now to stop scoring the two signals entirely, since a genuine browser produces both. Bench after the change: agent TPR 97.33% with every captured class at 100%, human FPR 0.00%. Removing them costs no detection, because automated browsers are identified by |
…isely
Follow-ups to the two inert signals, both in the harness rather than the product.
## The panel could not have disagreed
normalize.js pinned the navigator.webdriver descriptor non-configurable and added
a chrome.runtime object, on the stated belief that a real Chrome has both.
Measured in Chrome 150 it has neither: WebIDL defines the attribute as
configurable, and chrome.runtime is only exposed to pages an extension declares
externally_connectable for.
So the panel was cleaner than any real browser, and since the client had checks
for exactly those two properties, no human persona could trip them. The bench
reported 0.00% false positives for two signals that fire on every genuine Chrome,
and could not have reported anything else.
Now reproduces measured Chrome, verified property by property:
normalized bench browser: webdriver false, descriptor configurable,
window.chrome present, chrome.runtime absent
real Chrome 150: identical
All 22 personas re-captured against it. The rule is recorded in normalize.js and
bench/README.md: anything pinned there is a signal the panel cannot see, so pin as
little as possible and pin it to measured values.
## The solver tripped both ends of the same check
With honest normalization the gate failed on "PoW timing suggests external
processing" — the mirror image of the "impossibly fast" bug fixed earlier today,
and the same root cause. The final settle used setTimeout, which under concurrency
overshoots by tens of milliseconds; for a short solve that is longer than the work
itself, dragging the implied rate below the server's slow bound.
Now sleeps for the bulk and spins the last few milliseconds. Spinning is wasteful
by design and bounded; the alternative is a duration that misreports how long the
work took. Sixty short solves land between 195k and 407k hashes/sec, inside the
accepted band rather than merely near it.
Human panel FPR stays 0.00% and agent TPR 97.33%, now measured on a panel that
carries what a real browser carries.
webdriver_configurableandchrome_runtime_missingwere reported as Playwright artifacts and scored at 0.7 and 0.6 in the headless category. Neither indicates automation.Evidence
Measured in Chrome 150, driven by a person,
navigator.webdriver === false:webdriver_configurable— WebIDL definesnavigator.webdriveras configurable, so every browser reports the descriptor exactly as a patched one does.chrome_runtime_missing—chrome.runtimeis only exposed to a page when an extension declaresexternally_connectablematching it. On an ordinary site, an ordinary Chrome has none.Both fired on genuine Chrome, together, giving those visitors a headless category score of 0.771 and roughly +0.116 on their final score.
Change
The client no longer reports either signal, and all three servers ignore them when received, since deployed clients will keep sending them for a while.
webdriver_deletedandplaywright_globalsare unchanged — deleting the property or injecting framework globals is deliberate tampering that no shipped browser does.Impact
Agent detection is unaffected: automated browsers are identified by
navigator.webdriverbeing true, which is dispositive on its own, so every captured agent class stays at 100%.Also: touch form submissions
The submit classifier consulted only
keydownandmousedownwithin 100ms of the submit event.touchstartwas recorded in the event log but never read, so a tap whose compatibility mousedown lagged read as a programmatic submit — as did any click-to-submit slower than 100ms on a loaded device.Now selects by time rather than by the last five entries (scroll events share that log), considers
touchstart, and allows 500ms.Tests
Five end-to-end cases covering both directions of each change: a genuine Chrome's signal set raises no headless detection and is allowed, deleted-webdriver and injected globals are still detected, a tap-submitted form is not flagged, and a genuinely programmatic submit still is. All three servers pass.
Node 98/98, Go 94/98, Python 85/98 — each up by the five new cases, pre-existing divergences unchanged.