diff --git a/README.md b/README.md index c39b5d1..e5a6a64 100644 --- a/README.md +++ b/README.md @@ -84,9 +84,8 @@ flood can delay or hide a real attacker that appears only once. The limit resets An alert fires when an IP's flag enters the alert set (`--alert-on`, default `veneer,paradox,threat`) or changes, not on every sighting. Alerts go to stdout (one JSON object per line with `--json`) and, with `--webhook`, are POSTed as JSON. The payload's `text` field works -as-is with Slack and Mattermost incoming webhooks. Log lines are untrusted, so -the attacker-written `sample` line is kept out of `text` and only appears in the JSON `alert` -object. `docker stop` and Ctrl+C shut it down cleanly. It exits 1 if anything alerted. +as-is with Slack and Mattermost incoming webhooks. Log lines are untrusted and can hold private +details, so the `sample` line appears only in `--json` stdout, never in `text` or a webhook post. `docker stop` and Ctrl+C shut it down cleanly. It exits 1 if anything alerted. Developing: `python3 -m unittest discover -s tests` (no keys or network needed). diff --git a/ROADMAP.md b/ROADMAP.md index 2bca401..1a0bffb 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -407,13 +407,15 @@ VirusTotal's 500 for rescans. on restart. Not measured against a real log, so 15 may be too low for a busy one; the watcher warns on stderr when it hits the limit, once per stretch at the limit. -### 4.7 Keep raw log lines out of webhook posts · S +### 4.7 Keep raw log lines out of webhook posts · S · ✅ shipped -- [ ] **PDX-34: Webhooks don't send log lines.** Done when: a test shows a webhook POST contains no `sample` and none of the log line's text, stdout's `--json` output is unchanged, and the README and `SECURITY.md` say so. +- [x] **PDX-34: Webhooks don't send log lines.** Done when: a test shows a webhook POST contains no `sample` and none of the log line's text, stdout's `--json` output is unchanged, and the README and `SECURITY.md` say so. The JSON `alert` includes the raw log line, which can hold usernames, internal hostnames or tokens, and it goes to a third party. +*Downside:* a receiver that read `alert.sample` no longer gets it. `--json` stdout still has it for local use. + ### 4.8 Webhook URL out of the command line · S - [ ] **PDX-35: Read the webhook URL from the environment.** Done when: `PARADOX_WEBHOOK` sets the URL, a test shows it's used without `--webhook`, and the README's examples use it and say why. @@ -549,7 +551,7 @@ and fix the top issues. Repeat after 5.6. It also doubles as class material. | When | Items | Why first | |---|---|---| | Now | ~~1.1, 1.2, 1.5, 3.2, 5.1~~ (done) | Small, keyless, and fix the biggest measured weakness (the scanner rule's dependence on GreyNoise); 5.1 makes that dependence visible | -| Next | 1.3, 3.1, 3.5, 3.6, 4.1, 4.7, 4.8, 4.9, 4.10, ~~4.6, 4.5, 1.7, 1.8, 3.7, 1.4, 1.6, 5.2, 5.4~~ (done) | Close the known false positives, build adversarial and hiding-place test sets, test the AI summarizer under attack, and show watch state and claim checks in the UI | +| Next | 1.3, 3.1, 3.5, 3.6, 4.1, 4.8, 4.9, 4.10, ~~4.6, 4.7, 4.5, 1.7, 1.8, 3.7, 1.4, 1.6, 5.2, 5.4~~ (done) | Close the known false positives, build adversarial and hiding-place test sets, test the AI summarizer under attack, and show watch state and claim checks in the UI | | Then | 2.1, 2.3, 3.3, 3.4, 4.2, ~~4.3, 4.4~~ (done), 5.3, 5.5–5.7 | New unmasking signals plus self-measurement once history has accumulated, with the UI restructured to hold them | | Later | 2.2, 2.4, 2.5 | Bigger extensions: domains, campaigns, more beacons | diff --git a/SECURITY.md b/SECURITY.md index 005da17..0d1d35d 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -100,15 +100,13 @@ more than people expect. **What's in place:** -- The Slack-compatible `text` field holds only the flag, IP, scores and verdict. The attacker-written `sample` log line - is kept out of it. +- The Slack-compatible `text` field holds only the flag, IP, scores and verdict. The raw `sample` log line is removed + from everything posted, and appears only in `--json` stdout on your own machine. - Webhooks are opt-in, and a failed post is reported on stderr without stopping the watcher. **Open:** -- The JSON `alert` object does include `sample`, the raw log line (up to 300 characters), along with hit counts. A log - line can hold usernames, internal hostnames or tokens, and any receiver that stores the whole payload gets it. Slack - and Mattermost use `text`, but they still receive the rest. +- A post still says which IPs were seen, how often, and the log's path (`source`), and the receiver can store all of it. - `text` can quote a third-party label in the verdict, so a chat channel or bot reading it sees attacker-influenced text. - The URL is passed on the command line, so it's visible in process listings and shell history. It's effectively a password for that channel. diff --git a/paradox/watch.py b/paradox/watch.py index 86c7220..31d3175 100644 --- a/paradox/watch.py +++ b/paradox/watch.py @@ -228,7 +228,9 @@ def alert(self, row: dict, previous: str | None) -> None: with self.lock: print(line, file=self.out, flush=True) if self.webhook: - threading.Thread(target=self._post, args=(payload,), daemon=True).start() + # A log line can hold usernames, hostnames or tokens, and this goes to a third party. + outbound = {**payload, "alert": {k: v for k, v in row.items() if k != "sample"}} + threading.Thread(target=self._post, args=(outbound,), daemon=True).start() def _post(self, payload: dict) -> None: req = urllib.request.Request(self.webhook, data=json.dumps(payload).encode(), method="POST", diff --git a/tests/test_app.py b/tests/test_app.py index 8465085..343140e 100644 --- a/tests/test_app.py +++ b/tests/test_app.py @@ -109,6 +109,26 @@ def test_scans_once_and_alerts_on_flagged_ips_only(self): self.assertIn("PARADOX 45.1.1.1", alerts[0]["text"]) self.assertEqual(w.seen["45.1.1.1"]["hits"], 3) + def test_webhook_gets_no_log_line_but_stdout_keeps_it(self): + w, _, out = self.watcher({"45.1.1.1": fake_score(paradox=50, threat=0.5)}, webhook="https://example.test/x") + w.feed("Failed password for token-s3cret from 45.1.1.1", "auth.log") + posted, sent = [], threading.Event() + + def urlopen(req, timeout=None): + posted.append(req.data.decode()) + sent.set() + return mock.MagicMock() + + with mock.patch("urllib.request.urlopen", urlopen): + w.check(w.q.get()) + self.assertTrue(sent.wait(5)) + self.assertNotIn("s3cret", posted[0]) + body = json.loads(posted[0]) + self.assertNotIn("sample", body["alert"]) + self.assertEqual(body["alert"]["ip"], "45.1.1.1") + self.assertIn("PARADOX 45.1.1.1", body["text"]) + self.assertIn("s3cret", json.loads(out.getvalue())["alert"]["sample"]) + def test_new_ip_cap_stops_a_flood_and_recovers(self): w, _, _ = self.watcher({}, max_new_per_hour=3) with mock.patch("paradox.watch.time.time", return_value=1_800_000_000.0) as clock, \