Skip to content

refactor(py): harden the unsafe soundness argument - #1344

Merged
dekobon merged 1 commit into
mainfrom
fix/1057-safety-doc-pin
Aug 23, 2026
Merged

refactor(py): harden the unsafe soundness argument#1344
dekobon merged 1 commit into
mainfrom
fix/1057-safety-doc-pin

Conversation

@dekobon

@dekobon dekobon commented Aug 23, 2026

Copy link
Copy Markdown
Owner

Closes #1057 (already closed on the tracker; this is the code).

Two hygiene items on big-code-analysis-py/src/node.rs, the canonical
soundness argument for the workspace's only sanctioned unsafe block.
Neither was a live soundness bug — both weakened the argument's
durability.

1. The cited tree-sitter version is now gated, not remembered

The literal was already correct: 67eecdbf (the 0.26.11 → 0.26.12 bump)
updated it by hand, which is exactly the step this issue exists to
remove. So the work is the gate.

utils/check-safety-doc-pin.py fails when a backticked =X.Y.Z in that
module doc is not what [workspace.dependencies] pins, when the citation
is dropped altogether, or when the requirement is not an exact pin.
Dropping the literal in favour of a version-free phrase was considered
and declined in the issue: it removes the stale number by making the
claim unfalsifiable at read time. The gate cannot verify the argument
it forces a diff on the line at bump time, which is the prompt to re-read
it, and the diagnostic names the four premises to re-check rather than
just telling you to edit the line.

Wired in beside check-versions.py: make lint / pre-commit / ci, a
.pre-commit-config.yaml hook pair firing on Cargo.toml and node.rs,
and defensive twins in the lint job of ci.yml.

Watched failing on the real tree, not only on synthetic input — the
historical defect reproduces exactly:

$ sed -i 's/`=0.26.12`/`=0.26.9`/' big-code-analysis-py/src/node.rs
$ python3 utils/check-safety-doc-pin.py; echo "EXIT=$?"
error: big-code-analysis-py/src/node.rs cites a tree-sitter version the workspace does not pin
  big-code-analysis-py/src/node.rs:58: cites `=0.26.9`, pin is `=0.26.12`

EXIT=1

Scope is one file, deliberately. AGENTS.md and
utils/check-excluded-manifests.py also name the pin, but both sit in
prose that cites grammar pins illustratively (=0.23.5, =0.26.10),
so a literal-scanning rule there would fail on versions that are supposed
to differ. That limit is stated in the script's docstring rather than
left for a reader to assume away.

17 self-tests, weighted toward the false-clean direction: both directions
of the module-doc stop condition (an inner attribute must not end the
block — everything after it would silently stop being gated; code must),
the manifest parser over string / table / caret / missing forms, and
main() over the clean, stale, citation-dropped and unreadable-input
branches.

2. The Ast/Node pairing is structural

Field privacy alone changed nothing — all three PyNode { ast, node }
sites were inside the defining module. PyNode and PyNodeWalk now live
in an inner owned module, so the struct literal is unavailable to the
#[pymethods] blocks outside it. Every handle comes from wrap,
rewrap, PyNodeWalk::rooted_at, or PyNodeWalk::next_node.
rooted_at takes the PyNode rather than an (Ast, node) pair, so a
walk's cursor, seed and keep-alive all derive from one already-valid
handle — there is no argument to get wrong.

Everything except wrap is pub(super), not pub(crate): wrap is the
one entry point another module needs (ast.rs builds the root handle),
while node + rewrap together are the mispairing primitive —
a.rewrap(py, b.node()) pairs one tree's node with another tree's
keep-alive. Keeping them file-local is what makes the boundary true
beyond this file.

PyO3 raised no objection: #[pyclass] expands fine inside the inner
module and the #[pymethods] blocks stay outside it. No fallback to a
marker field or newtype was needed.

Verified by perturbation, per .claude/rules/testing.md — not by a
runtime test, which would pass vacuously because the mismatch is
unconstructible from Python. Reintroducing either literal outside the
boundary is now a compile error, where before it was the shipped code:

error[E0451]: fields `cursor`, `stack` and `ast` of struct `PyNodeWalk` are private
error[E0451]: fields `ast` and `node` of struct `PyNode` are private

The issue's context section is preserved

The "what was checked and found sound" list now lives in the node.rs
module doc under ## What has been checked: the absent GC tracking, the
unreachable Python constructor (which is why the pairing is enforced at
the Rust module boundary rather than by a runtime test), the per-yield
refcount bump, and the 3,000-cycle / 8-thread runs. AGENTS.md's
unsafe carve-out now names the owned boundary and the gate too.

Verification

make pre-commitBCA_GATE: pass, with 5,364 Rust tests (10 skipped)
and 363 Python tests (1 xfailed) — the same totals main carries. No
integration snapshot moved: the working tree is free of .snap.new and
the big-code-analysis-output submodule is clean at its recorded SHA.

No STABILITY.md entry: no library behaviour moves and PyNode is not
part of the Rust public API. The gate is recorded under
## [Unreleased], beside the sibling check-ruff-lockstep entry.

Known gap, not fixed here

module_doc_lines skips only single-line #![…] inner attributes; a
multi-line one's continuation ends the scan early, so a stale citation
below the cut would be invisible. node.rs has no inner attributes at
all today, and the fix needs a decision about test_code_ends_the_block,
which currently pins the break as intended behaviour. Filed as #1345.

The module doc of big-code-analysis-py/src/node.rs is this workspace's
canonical justification for its only sanctioned `unsafe` block. Two
things weakened it, neither a live soundness bug.

The argument reasons about a *named* tree-sitter release — `Tree(NonNull
<ffi::TSTree>)`, `Node<'tree>(ffi::TSNode, PhantomData<&'tree ()>)`,
`Tree::edit(&mut self)`, `Send + Sync` — and the literal had drifted two
patch releases behind the pin before anyone noticed. It is correct today
only because the 0.26.12 bump updated it by hand, which is the step this
gate removes. `utils/check-safety-doc-pin.py` fails when a backticked
`=X.Y.Z` in that module doc is not the `[workspace.dependencies]` pin,
when the citation is dropped altogether (the version-free phrasing was
considered and declined: it removes the stale number by making the claim
unfalsifiable at read time), and when the requirement is not an exact
pin at all. The gate cannot verify the argument; it forces a diff on the
line at bump time, which is the prompt to re-read it. Wired into `make
lint` / `pre-commit` / `ci` beside check-versions, with a pre-commit hook
pair on Cargo.toml + node.rs, defensive twins in ci.yml, and 17 self-
tests — including both directions of the module-doc stop condition,
since a scanner that stops reading early reports a clean file.

`detach`'s precondition is that a node is stored alongside the `Py<PyAst>`
that keeps its tree alive, and three sites built `PyNode { ast, node }`
by struct literal. Field privacy does not constrain them: all three were
in the defining module. `PyNode` and `PyNodeWalk` now live in an inner
`owned` module, so the literal is unavailable to the `#[pymethods]`
blocks outside it and every handle comes from `wrap`, `rewrap`,
`rooted_at`, or `next_node`. `rooted_at` takes the `PyNode` rather than
an `(Ast, node)` pair, so a walk's cursor, seed and keep-alive all derive
from one already-valid handle. Verified by perturbation rather than a
test: reintroducing either literal is now E0451, and a runtime test would
pass vacuously because the mismatch is unconstructible from Python.

That last fact, and the rest of the adversarial-harness results from the
issue — no GC tracking, no Python-reachable constructor, the per-yield
refcount bump, the 3,000-cycle and 8-thread runs — are recorded in the
module doc, where an auditor of the `unsafe` block will find them.

The gate is recorded under `## [Unreleased]`, beside the sibling
`check-ruff-lockstep` entry. No `STABILITY.md` entry: no library
behaviour moves and `PyNode` is not part of the Rust public API.

Everything in `owned` except `wrap` is `pub(super)`. `wrap` is the one
entry point another module needs — `ast.rs` builds the root handle —
while `node` + `rewrap` together are the mispairing primitive
(`a.rewrap(py, b.node())` pairs one tree's node with another tree's
keep-alive), so keeping them file-local is what makes the boundary true
beyond this file.

Fixes #1057
@dekobon
dekobon merged commit 277e8da into main Aug 23, 2026
51 checks passed
@dekobon
dekobon deleted the fix/1057-safety-doc-pin branch August 23, 2026 03:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docs(py): unsafe soundness doc cites stale tree-sitter pin; make Node/Ast pairing structural

1 participant