Skip to content

Add the adversarial test suite (roadmap 3.5) - #31

Merged
Castor-o7 merged 1 commit into
mainfrom
pdx-15-adversarial
Oct 31, 2026
Merged

Castor-o7 merged 1 commit into
mainfrom
pdx-15-adversarial

Conversation

@Castor-o7

Copy link
Copy Markdown
Owner

Story: PDX-15

Adds tests/test_adversarial.py, which runs each attack through the real OTX parser, reverse DNS check and scorer, offline, with no real lookups spent. "Guarantee" tests must keep holding. "Known weakness" tests pin attacks that work today, so a fix or a regression has to change the number on purpose. No scoring code changes.

What it found

Attack With independent beacons When OTX is the only witness
Smearing One account gets one vote; old pulses fade Works: 5 new accounts take a clean IP to paradox 60, Likely malicious; 10 reach 72, Malicious
Laundering Holds: stays Malicious, veneer fires Works: OTX's own whitelist takes paradox 72, Malicious to 14, Suspicious, no veneer
Dilution Holds: stays Malicious; padding can't weaken another author's pulse Works: every pulse padded to 20,000 indicators reads Likely benign
Label spoofing Holds: a forged PTR earns no label and raises a veneer Clean IP with a forged PTR reads paradox 15, Suspicious

Files PDX-38 (smearing) and PDX-39 (OTX's whitelist). Not fixed here, because each fix risks the early warning or the false-positive fix behind the rule and needs a rescore first. Padding is covered under PDX-38 for now.

Tested

  • 192 tests pass locally. See CI for other versions.
  • With the code intact as a baseline, the suite passes. With one-vote-per-author removed, 2 tests fail. With the whitelist discount removed, 3 subtests or tests fail. With the bulk discount removed, 1 fails.
  • I did not run removal checks on the reverse DNS or veneer rules.

Limits

These are today's rules against attacks we thought of, on fixtures where other beacons are set by hand. They don't show what a real attacker can do, whether OTX would catch sock puppets, or how often this happens. The pinned numbers are exact to a small tolerance, so any legitimate scoring change will need to update them on purpose.

🤖 Generated with Claude Code

Runs smearing, laundering, dilution and label spoofing through the real parser
and scorer, offline, and pins how far each moves the score. With independent
beacons the protections hold. When OTX is the only witness, three attacks work,
so those are pinned as known weaknesses and filed as PDX-38 and PDX-39.

Story: PDX-15

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Castor-o7
Castor-o7 merged commit 6539736 into main Oct 31, 2026
5 checks passed
@Castor-o7
Castor-o7 deleted the pdx-15-adversarial branch October 31, 2026 21:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant