Skip to content

Guard the no-telemetry promise in the code, not just in the prose - #175

Merged
donislawdev merged 4 commits into
masterfrom
feat/no-telemetry-guard
Sep 3, 2026
Merged

Guard the no-telemetry promise in the code, not just in the prose#175
donislawdev merged 4 commits into
masterfrom
feat/no-telemetry-guard

Conversation

@donislawdev

Copy link
Copy Markdown
Owner

Convention 36 - the shipped program reaches no network - was stated in four places and
checked in none. test_license_surface.py and test_windows.py both verify that we SAY
it. Nothing had ever looked at the code. For a tool that intercepts the user's traffic,
"it sends nothing anywhere" is the one promise where being wrong is not a bug.

Two layers, because neither contains the other

The obvious answer is an AST scan. That was the first design, and it picked the wrong
primitive: the interpreter already answers this question a layer down, through PEP 578
audit hooks. Measured on CPython 3.14.7:

what it catches static (AST) runtime (audit hook)
import urllib.request yes yes
__import__("url" + "lib.request") no yes
ctypes.windll.wininet yes yes, with the resolved name
a network library under a computed name no yes
code no test executes yes no
where a connection actually goes no yes, the event carries the address

So the static half answers "was it written" and the runtime half answers "what actually
happened". Dropping either leaves a whole shape of bypass open, which is why both ship.

The allowlists are measured, not guessed

  • a CLI --simulate run with impairment armed: zero network events, zero forbidden
    imports, four local libraries
  • building the whole GUI on the fake toolkit: zero network events
  • host_identity(): exactly five events, all of them the documented route probe

Because the baseline is zero, the guard can assert the strongest shape there is: nothing,
except four entries that each carry a written reason. And because the audit event carries
the destination, it asserts 8.8.8.8:80 and 2001:4860:4860::8888:80 rather than merely
that a socket appeared - "a socket happened" would pass for a socket that shipped the
user's captured traffic somewhere.

What is checked

Static: no network client imported; socket and webbrowser used only where a registry
names a reason; ctypes limited to the ten local Windows libraries; no process spawned
(so no shelling out to curl or PowerShell); URL literals only in appinfo.py and
legal.py, with docstrings exempt because prose about a URL is not a URL.

Runtime: no unexpected network event, no forbidden import, no unexpected library, and
every connect naming a documented probe address.

The exception registry is exact in both directions. An entry naming code that has
moved fails like a call with no entry - permission nobody can point at any more is how an
allowlist becomes a blindfold.

Proof

Eight mutations against the real tree, 8/8 caught. Three are added to the mutation
registry so CI repeats them: a plain import, a network library through ctypes, and a
client imported under a computed name at runtime.

A permanent canary points the scanner at seven shapes it must reject plus one clean case
it must not, modelled on tests/test_public_text_guard.py - the file that earned this
project the lesson that a guard nobody has watched fail is indistinguishable from a guard
that reads nothing.

What it cannot prove, in the docstring rather than implied

  • A C extension calling WinSock directly raises no socket event. psutil is such an
    extension: the hook covers third-party Python, not third-party C.
  • The runtime layer only sees what it runs. It drives the CLI, not the GUI.
  • A computed import inside a function nothing calls is invisible to both layers. That
    one was found by mutating rather than by thinking, and moving the same call onto a path
    the CLI runs gets it caught at once.
  • The runtime library check is vacuous on the Linux leg - the socket half is real on both,
    but nothing loads a Windows library there, and the static half is what covers it.

This is a lock on the surface, not a proof of silence, and it says so.

Two findings the measurement produced

utils.host_identity has a socket.gethostbyname fallback that convention 36 never
mentioned, and a name resolution can put a query on the wire - the convention's reasoning
("a UDP connect sends no packet") does not cover it. Measured: it never runs here, because
the route probe answers first. It stays as an exception, to be measured another time, and
it is named in the registry with that reason rather than passed over silently.

Separately, the convention says "one socket.socket in beantester/" - true of the calls,
but import socket is in two modules (settings.py reads the machine's own services file
to label a port). A guard written from the convention alone would have had the wrong
allowlist on day one.

Checks

test_no_telemetry.py, test_mutation_registry.py, test_repo_conventions.py and
test_code_hygiene.py pass, plus ruff and mypy on the new file. Run on Windows and, for
the platform question above, in WSL.

🤖 Generated with Claude Code

donislawdev and others added 4 commits September 3, 2026 20:27
…in the prose

Convention 36 was stated in four places and checked in none. The two
tests that mention telemetry both check that we SAY it. For a tool that
intercepts the user's traffic, "it sends nothing anywhere" is the one
promise where being wrong is not a bug.

Two layers, because neither contains the other:

- static (AST) sees code that no test executes, and this package has
  such code: host_identity's gethostbyname fallback never runs on an
  ordinary path
- runtime (audit hook, PEP 578, in a subprocess) sees what is not
  written down literally. Measured on 3.14.7: ctypes.dlopen carries the
  library name even when computed, socket.connect carries the
  destination, and the import event catches __import__("url" + "lib")

The allowlists are measured, not guessed. A CLI --simulate run with
impairment armed raises zero network events and zero forbidden imports;
building the whole GUI raises zero; host_identity raises exactly five,
all of them the documented route probe. So the guard asserts where a
connection goes, not merely that a socket appeared.

Eight mutations on the real tree, 8/8 caught, three of them added to the
mutation registry so CI repeats them. A permanent canary points the
scanner at seven shapes it must reject and one clean case it must not.

What it cannot prove is in the docstring rather than implied: a C
extension calling WinSock directly raises no socket event, the runtime
layer only sees what it runs, and a computed import inside a function
nothing calls is invisible to both layers.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Measured in WSL: the socket events and their destinations fire the same
on both runners, but dlopen comes back empty there, so the runtime
library check gives nothing on the Ubuntu leg. The static check is the
one that covers it everywhere, and a green run that measures nothing has
to say so.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The guard shipped saying the overhead was not measured, which is how a
sentence like that survives for years. Two full runs on a bench that
refuses a verdict when the difference sits inside the noise.

An installed hook charges ~235-310 ns per audit event. The ratio
repeated (4.66x and 4.76x on a dispatch-only workload); the absolute
difference did not (0.118 s and 0.154 s), so the range is the honest
form and neither absolute should be quoted as the number.

This test raises 2010 events, roughly half a millisecond, and the bench
refused a conclusion on its workload in both runs: against a 1.2 s run
whose own spread is 50 to 200 ms the hook cannot be seen. What the test
costs is the second that --duration 1 waits out.

The first version of the bench looped on open() and was refused on every
column, correctly - opening a file 20000 times measures file work, not
dispatch, and its own spread was ten times any hook cost.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An outside review pointed at the shape and measurement confirmed it: the
scanner matched socket.x(...) literally, so five ways of doing the same
thing walked straight through. Each was fed to the scanner rather than
reasoned about:

- import socket as s / s.create_connection(...)
- from socket import create_connection / create_connection(...)
- import subprocess as sp / sp.run(...)
- from subprocess import Popen / Popen(...)
- os.popen(...), which was simply missing from the spawn list

The review was wrong about socket.send/sendall/sendto: those need a
socket, and a socket needs socket.socket or socket.create_connection,
which were already gated. The gate sits at creation, which is one place
instead of every method a socket has. That reasoning now lives next to
the registry instead of only in my head.

Measuring it turned up a defect the review did not: a module carrying
the literal "://" CRASHED the guard on an IndexError. A scanner that
dies is worse than one that misses.

The fix has two halves and the second is load-bearing: local names are
resolved against what the module imported, and the import itself is
registered per file. A statement names its module whatever the local
name becomes, so f = socket.create_connection; f(...) - which defeats
any call-site matching - is caught at the import.

Also: the from-ctypes-import loader form, and the canary grown from
seven cases to seventeen, one per spelling.

Verified: 9/9 mutations on the real tree including three new ones, all
four telemetry entries caught through tools/mutate.py, ten tests green
on Windows and in WSL.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@donislawdev
donislawdev merged commit 322f818 into master Sep 3, 2026
14 checks passed
@donislawdev
donislawdev deleted the feat/no-telemetry-guard branch September 3, 2026 19:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant