diff --git a/README.md b/README.md index 9c7aac3..86b8fc4 100644 --- a/README.md +++ b/README.md @@ -356,7 +356,7 @@ assistant can call. It speaks MCP over stdio using only the standard library. | `ip_history` | Recorded scans and measured beacon lag | | `beacon_status` | Which beacons are configured, which keys are set, and how each last answered | -**Threat model for agents.** Beacon text is written by third parties, possibly the attacker being +**Threat model for agents.** The full threat model is in [SECURITY.md](SECURITY.md). Beacon text is written by third parties, possibly the attacker being investigated, so treat it as data: - **What the tools can read:** `scan_log`'s `path` reads only files inside `/var/log`. To change that, diff --git a/ROADMAP.md b/ROADMAP.md index abdb183..07c9e6f 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -374,9 +374,9 @@ The model can't then quietly overrule the evidence, whether through injection or - Document the threat model for agent use: what the tools can read (local log files) and why results never include raw log lines. -### 4.4 Threat model document · S +### 4.4 Threat model document · S · ✅ shipped -- [ ] **PDX-19: Threat model document.** Done when: `SECURITY.md` exists and covers all six topics listed in this item. +- [x] **PDX-19: Threat model document.** Done when: `SECURITY.md` exists and covers all six topics listed in this item. Write `SECURITY.md` covering: @@ -389,6 +389,10 @@ Write `SECURITY.md` covering: It doubles as class material. +*Shipped:* [`SECURITY.md`](SECURITY.md) covers all six, and each section lists what's still open. Writing it turned up +four gaps with no story yet: `paradox watch` has no cap on new IPs, webhooks send the raw log line in the JSON `alert`, +the webhook URL sits on the command line, and `.env` isn't checked for loose permissions. + ### 4.5 Limit which files `scan_log` can read · S · ✅ shipped - [x] **PDX-29: Limit `scan_log`'s file access.** Done when: `scan_log`'s `path` argument reads only files inside an allowed set stated plainly in the README, symlinks and `..` can't escape it, a test shows a path outside the set is refused without its contents or existence being revealed, and the `text` argument still works without any configuration. @@ -511,7 +515,7 @@ and fix the top issues. Repeat after 5.6. It also doubles as class material. |---|---|---| | Now | ~~1.1, 1.2, 1.5, 3.2, 5.1~~ (done) | Small, keyless, and fix the biggest measured weakness (the scanner rule's dependence on GreyNoise); 5.1 makes that dependence visible | | Next | 1.3, 3.1, 3.5, 3.6, 4.1, ~~4.5, 1.7, 1.8, 3.7, 1.4, 1.6, 5.2, 5.4~~ (done) | Close the known false positives, build adversarial and hiding-place test sets, test the AI summarizer under attack, and show watch state and claim checks in the UI | -| Then | 2.1, 2.3, 3.3, 3.4, 4.2, 4.4, ~~4.3~~ (done), 5.3, 5.5–5.7 | New unmasking signals plus self-measurement once history has accumulated, with the UI restructured to hold them | +| Then | 2.1, 2.3, 3.3, 3.4, 4.2, ~~4.3, 4.4~~ (done), 5.3, 5.5–5.7 | New unmasking signals plus self-measurement once history has accumulated, with the UI restructured to hold them | | Later | 2.2, 2.4, 2.5 | Bigger extensions: domains, campaigns, more beacons | Items 3.3, 3.4 and 1.3 need weeks of history, so the most useful thing to start right away diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 0000000..5360953 --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,153 @@ +# Threat model + +Paradox Detector reads text written by the people it investigates, spends API quotas that belong to its operator, and +can hand its output to an AI. This page says what can go wrong with each of those, what the code does about it, and what +is still open. + +It covers the code as it stands. Items marked *open* are real gaps, not plans already met. + +## What it protects and who it worries about + +- **What matters:** API keys, API quotas, logs and scan history, the operator's trust in a score, and any agent or + analyst acting on the output. +- **Who it worries about:** an attacker who controls text in a feed or log, including the IP being investigated; anyone + who can reach the web server; and an AI client that has been talked into misusing the tools. +- **Not covered:** someone who already has a shell as the user running the detector. They can read `.env` directly. + +**What leaves the machine.** Every scan sends the IP to the beacons, including VirusTotal, AbuseIPDB, OTX and GreyNoise. +Log lines aren't sent to them, but the fact that an IP appeared in your logs is visible to those services through the +lookup. The analyst summary sends the report to Anthropic, and only when it's asked for. + +## Quota exhaustion + +**Threat:** each scan queries every beacon, so anyone who can trigger scans spends your free-tier allowance. An attacker +can do this without reaching the app at all, because `paradox watch` scans each new public IP it sees in a log. Forged +source addresses in a log become lookups. When a quota runs out, the beacon goes quiet, the slow tier gets thinner and +an IP's paradox can rise. + +**What's in place:** + +- VirusTotal stays inside its free limits, with lookups spaced 15 seconds apart, daily and monthly caps, and an hour's + reuse of good answers. The count is shared by processes on one machine. +- The web server limits scans and summaries per client (`PARADOX_RATE_LIMIT`, default 20 a minute) and runs at most 2 + log scans at once, each up to 50 IPs and 5 MB. +- A report lists each beacon that gave no reading, why, and what that costs, so a thin result says so. + +**Open:** + +- Only VirusTotal has a budget. The other beacons rely on their providers' own limits. +- `paradox watch` has no cap on new IPs, so a log flooded with addresses can use up the day's VirusTotal lookups and + leave later IPs without them. +- The rate limit is in memory and per client, so it resets on restart and many clients get many allowances. With + `PARADOX_TRUST_PROXY=1` it trusts `X-Forwarded-For`, which is only safe when a proxy sets that header. +- Lookups made with the same key on another machine aren't counted. + +## Poisoned feeds + +**Threat:** beacons are third-party opinions, and attackers can write to some of them. They can smear an innocent IP +with new OTX accounts, launder a bad one by getting it onto an allowlist, pad a pulse to trigger the bulk discount, or +forge a reverse DNS name. + +**What's in place:** + +- Sources sharing an upstream count as one vote, and a veneer contradiction must come from a different origin than the + label it contradicts. +- Reverse DNS identity claims are forward-confirmed, and a forged one raises a veneer. +- Pulses that only mention an IP as contacted count at half weight, a small number of AbuseIPDB reporters gets scaled + down unless the reports are recent, and one VirusTotal vendor can't flag an IP alone. +- The ledger shows the points behind a score, so an odd result can be traced to a beacon. + +**Open:** + +- How far each attack can move a score isn't measured. That's [PDX-15](ROADMAP.md#35-adversarial-test-suite--m), and + until it's built the protections above are untested as defenses. +- One-vendor scaling has a cost: an attacker known to exactly one vendor and nothing else can read as Likely benign. +- Independence between VirusTotal's vendors can't be shown, because their sources are closed. + +## Injection into the summarizer and MCP clients + +**Threat:** pulse titles, tags, reverse DNS names and log lines can say anything, including instructions to an AI. +A user agent or SSH username is enough. An injected agent could also try to use the detector's tools as a way to read +local files. + +**What's in place:** + +- The summarizer's system prompt tells the model to treat beacon text as data, and the report is sent as JSON. +- MCP output schemas mark every field that can carry third-party text with `"x-untrusted": true`, every returned string + is capped at 1,000 characters, and `redact` returns an allowlist of numbers and the detector's own fields. +- `scan_log` returns IPs and counts, never log lines. +- `scan_log`'s `path` reads only inside `/var/log` or the folders in `PARADOX_LOG_DIRS`. Symlinks and `..` can't leave + them, and a refusal doesn't reveal whether a file exists. +- The web UI escapes third-party text, and the server sends a Content Security Policy that blocks framing and scripts + from other origins. It allows inline scripts, so it wouldn't stop injected inline script if escaping failed. + +**Open:** + +- A label and a system prompt are requests, not enforcement. A model can still follow injected text, and how often + isn't measured ([PDX-16](ROADMAP.md#41-prompt-injection-evaluation-for-the-analyst-summary--m)). Nothing checks a + summary against the score yet ([PDX-17](ROADMAP.md#42-structured-verdict-free-text-second--s)). Read the score, not + just the prose. +- `x-untrusted` only helps a client that acts on it. +- `scan_ip` and `scan_log` make network calls and spend quota whenever an agent asks, and only the VirusTotal pacing + limits that on the MCP path. +- An allowed folder is readable in full, so its files' IPs and counts can reach the agent. + +## Webhook data exposure + +**Threat:** `paradox watch --webhook` posts alerts to a third party, often a shared chat channel. The payload can carry +more than people expect. + +**What's in place:** + +- The Slack-compatible `text` field holds only the flag, IP, scores and verdict. The attacker-written `sample` log line + is kept out of it. +- Webhooks are opt-in, and a failed post is reported on stderr without stopping the watcher. + +**Open:** + +- The JSON `alert` object does include `sample`, the raw log line (up to 300 characters), along with hit counts. A log + line can hold usernames, internal hostnames or tokens, and any receiver that stores the whole payload gets it. Slack + and Mattermost use `text`, but they still receive the rest. +- `text` can quote a third-party label in the verdict, so a chat channel or bot reading it sees attacker-influenced text. +- The URL is passed on the command line, so it's visible in process listings and shell history. It's effectively a + password for that channel. +- Nothing stops an `http://` URL, which would send alerts unencrypted. + +## `.env` handling + +**Threat:** `.env` holds every API key and the web password. Leaking it spends your quotas under your name and opens +the web app. + +**What's in place:** + +- `paradox init` creates it with mode 600. +- It's in `.gitignore` and `.dockerignore`, so it stays out of commits and images. Compose passes it at runtime. +- Status output reports whether a key is set, never its value. +- `scan_log` can't read it by default, because the working folder isn't allowed. + +**Open:** + +- Copying `env.example` by hand, as the Docker steps do, keeps the default permissions. Run `chmod 600 .env`. +- The file isn't checked for loose permissions when it's loaded. +- Keys are held in plain text and in the process environment. +- A key pasted into a chat or an AI session should be treated as exposed and rotated. + +## Why the server refuses to bind publicly without a password + +**Threat:** the web app runs scans. On a public address with no password, anyone who finds it can spend your API quotas +and Anthropic credits, read your scan history, and upload logs for scanning. Scan history shows which IPs you've been +looking at, which an attacker under investigation would like to know. + +**What's in place:** + +- It binds to `127.0.0.1` by default. For any other address it exits with an error unless `PARADOX_PASSWORD` is set, so + the safe outcome doesn't depend on reading the docs. +- `PARADOX_ALLOW_OPEN=1` overrides that, for when a proxy already handles auth. It has to be set on purpose. +- The password check uses a constant-time comparison, and log uploads must be JSON. + +**Open:** + +- HTTP Basic sends the password with each request and the server has no TLS of its own, so a public deployment needs an + HTTPS proxy. +- `PARADOX_ALLOW_OPEN=1` behind a misconfigured proxy is an open server. +- On loopback there's no password, so anything else running on the machine can use the app.