Skip to content

Adaptive challenge cost: price by suspicion, on the clock not the hash - #27

Merged
cport1 merged 4 commits into
mainfrom
feat/adaptive-challenge-cost
Jul 28, 2026
Merged

Adaptive challenge cost: price by suspicion, on the clock not the hash#27
cport1 merged 4 commits into
mainfrom
feat/adaptive-challenge-cost

Conversation

@cport1

@cport1 cport1 commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Difficulty was effectively constant — 4, or 5 from a datacenter address, or up to 6 under load. That is the wrong knob to escalate.

Browser JS runs 1–3M hash/s; native code 100–500M/s. Difficulty 6 costs a native solver about a millisecond and a budget Android phone about sixteen seconds. Raising it taxes the slowest legitimate devices and constrains nobody.

Wall-clock has no such asymmetry — nobody can make less time pass. Each challenge now carries a signed minAgeMs, and that is where escalation lands.

source difficulty minAgeMs
clean, or unknown 4 1500
1–2 recent strong bot verdicts 4 4000
3–5 5 8000
6+ 5 15000

A clean visitor pays exactly what they paid before. At the 1.5s baseline one address mints roughly 40 tokens a minute; at 15s, four.

Difficulty now caps at 5 rather than 6, and a datacenter address no longer raises it at all — a real person on a corporate VPN or iCloud Private Relay was paying several seconds of blocked hashing for a shared IP. It raises the time floor instead, which a hosted scraper feels as reduced throughput and someone filling in a form does not feel at all.

Cost lands as latency, not as a score

The client is told minAgeMs and waits it out before submitting, so an ordinary visitor absorbs an elevated cost as a short delay rather than a worse verdict. That matters most for anyone sharing an egress address with whatever earned the delay.

A client that submits early is still scored, but the two cases are separated. Below the universal 1500ms baseline keeps its existing strength — nothing legitimate happens that fast. Between the baseline and the source's own floor is contributory only, because an older cached client does not know to wait and should not be read as automation for it.

Scope of the memory

Per (site key, address), 15-minute window, records only verdicts at or above 0.8, stores nothing but timestamps. Same shape and privacy class as the rate limiter beside it. It is not Workstream F §10.1 cross-session correlation — no behavioral vectors, no per-fingerprint history — and the module comment says so, because those two should not be conflated on the grounds that both are "server-side memory".

Measurement

Human FPR 0.00% (0/126), agent TPR 97.33% — both unchanged.

Two harness defects surfaced, both invisible until a signal was keyed by source.

Sample addresses collided. They were a hash of the sample id, putting 180 samples into a 508-address space. By the birthday bound: 25 shared addresses, 24 of them putting an agent and a human on the same IP. The corpus was manufacturing shared egress its labels never claimed, and the first source-keyed signal read 12.7% FPR on a panel whose real answer was zero. Addresses are now assigned by position, and the pool throws rather than wrapping.

Runs inherited each other's server state. One signal fired on 2/75 agents against a fresh server and 74/75 against one already benchmarked. Each run now namespaces its own site key.

The replayer deliberately still does not honour minAgeMs. A replayer that did would satisfy the timing gate by construction and the panel could never observe it firing on a human — the same trap as pinning a property during normalization and then reporting that nothing disagrees with it.

bench/package.json pointed npm test at lib/lib.test.js, which has never existed; CI runs npm test in server-node, so nothing invoked it. Replaced with real tests for the address pool and wired into the bench workflow.

Also in here

server-node/index.js was missed by the pass that removed HMAC truncation elsewhere: token signatures were cut to 64 bits on both generation and verification, while Go, Python and the standalone server used the full digest. Now full-length. Tokens issued by an older library build will not verify after upgrading.

Verification

  • Go: go vet clean, all tests pass (10 new)
  • Node: all suites pass (12 new); the new suite is wired into npm test, which it was not
  • Python: 43 tests across 5 files pass (10 new)
  • E2E: 103/103 (5 new)
  • Bench harness: 5 new tests
  • All three servers confirmed serving identical tiers end-to-end

cport1 added 4 commits July 28, 2026 16:20
…t the hash

Difficulty was a near-constant: 4, or 5 from a datacenter address, or up to 6
under a high request rate. That is the wrong knob. Browser JS runs 1-3M hash/s
and native code 100-500M/s, so difficulty 6 costs a native solver about a
millisecond and a budget Android phone about sixteen seconds. Escalating there
taxes the slowest legitimate devices and constrains nobody.

Wall-clock does not have that property. Nobody can make less time pass, so a
minimum challenge age caps how fast one source can mint tokens regardless of the
hardware it brings. Each challenge now carries a signed minAgeMs, and that is
where the escalation lands.

  clean or unknown          difficulty 4, 1500ms   (unchanged for everyone)
  1-2 strong bot verdicts   difficulty 4, 4000ms
  3-5                       difficulty 5, 8000ms
  6+                        difficulty 5, 15000ms

Difficulty now caps at 5 rather than 6. A datacenter address no longer raises it
at all — a real person on a corporate VPN or iCloud Private Relay was paying
several seconds of blocked hashing for a shared IP — and raises the time floor
instead, which a hosted scraper feels as reduced throughput and a person filling
in a form does not feel at all. At the 1.5s baseline one address mints roughly 40
tokens a minute; at 15s, four.

The client is told minAgeMs and waits it out, so an ordinary visitor pays the
elevated cost as a short delay rather than as a worse score. That distinction
matters most for anyone sharing an egress address with whatever earned the delay.
A client that submits early is still scored, but the below-baseline case keeps
its existing strength while the between-baseline-and-floor case is contributory:
an older cached client does not know to wait and should not be read as automation
for it.

Suspicion is held per (site key, address) for 15 minutes, records only verdicts
at or above 0.8, and stores nothing but timestamps. It is not Workstream F 10.1
cross-session correlation and must not grow into it — same shape and privacy
class as the rate limiter beside it.

Bench: human FPR 0.00% (0/126), agent TPR 97.33%, both unchanged.

Two harness defects surfaced on the way, both invisible until a signal was keyed
by source:

Sample addresses were a hash of the sample id, putting 180 samples into a
508-address space. They collided constantly — 25 shared addresses, 24 of them
putting an agent and a human on the same IP — so the corpus was manufacturing
shared egress its labels never claimed. The first source-keyed signal read 12.7%
FPR on a panel whose real answer was zero. Addresses are now assigned by position
and the pool throws rather than wrapping.

Runs also inherited server-side state from each other: one signal fired on 2/75
agents against a fresh server and 74/75 against one already benchmarked. Each run
now namespaces its own site key.

bench/package.json pointed npm test at lib/lib.test.js, which has never existed;
CI runs npm test in server-node, so nothing invoked it. Replaced with real tests
for the address pool and wired into the bench workflow.

Also completes the HMAC hardening pass in server-node/index.js, which it had
missed: token signatures were truncated to 64 bits on both generation and
verification while Go, Python and the standalone server used the full digest.
Tokens issued by an older library build will not verify after this upgrade.
Three failures, all the same shape: a file the build needs is not where the
build looks.

`.gitignore` had `bench/*.json` to drop replay output, which also dropped
bench/package.json. The harness could not be installed from a fresh clone, its
playwright dependency was unrecorded, and CI could not run anything through npm
in that directory — which is also why bench's `npm test` pointed at a file that
had never existed without anyone noticing. Narrowed to `bench/*results*.json`.

Both the Node and Python Dockerfiles enumerated the modules their entrypoint
needs, under a comment instructing the reader to keep the list in step. The list
went out of step immediately: neither image had the new module and both crashed
on startup. That is the same failure as upstream #23, and the instruction did not
prevent it because nothing about a Dockerfile review makes a missing require
visible. They now copy the source and drop test files, so the list cannot drift
again.

Verified by building all three images and starting them: each serves /health and
a challenge carrying difficulty and minAgeMs.
node --test only handles glob patterns itself from Node 22; CI runs Node 20 and
received the literal string. Unquoted, so the shell expands it on both.
Neither had ever run in CI. Node's are covered by the benchmark workflow, so
every check on a pull request was green while two of the three implementations
were unverified. Among the tests that had never once executed there:

  - TestOnlyJA4TLSIsImplemented, the only thing standing between this MIT
    project and a FoxIO License 1.1 module that cannot legally ship in it
  - TestWeightsSumToOne, guarding the scoring weights invariant
  - every Python test, on an implementation whose only CI coverage was a
    container smoke test

The sync rule says a change lands in all three servers, which is worth little if
only one is checked. Go now runs vet and `go test -race`; Python installs
requirements.txt and runs discovery.

Discovery had to start working first. The Python tests are plain functions
collected by a decorator rather than TestCase methods, and `unittest discover`
could not see them: test_clientip and test_sitekeys guarded their runner under
__main__ and reported "Ran 0 tests" without complaint, while test_detection and
test_inputforensics ran at import and called sys.exit mid-discovery, surfacing as
errors against suites that passed. Either way the count was wrong — 12 tests and
two errors, where there are 43.

testkit.py now holds the registry, so the same functions run both ways: readable
per-test output from a single file, and visible to unittest for discovery and CI.
It also replaces four copies of the same runner block. Verified that an injected
failure fails through both paths and is attributed to the file that owns it.
@cport1
cport1 merged commit 1f0e96d into main Jul 28, 2026
6 checks passed
@cport1
cport1 deleted the feat/adaptive-challenge-cost branch July 28, 2026 22:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant