Rebuild the agent-worktree sweep on the site subtree - #525
Conversation
There was a problem hiding this comment.
Sorry @blooop, your pull request is larger than the review limit of 150,000 diff characters
Codecov Report❌ Patch coverage is Additional details and impacted files
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
342dbe3 to
42cd615
Compare
blooop
left a comment
There was a problem hiding this comment.
This was generated by AI during review.
Two independent axes, fresh context, adversarial, at 42cd615b9a4301cf5e04825590dc1cccf200f6d9. Reviewer did not write this code. Every finding below was proved by a probe test run against a worktree of the head, not by reading.
Standards
Axis: pass. The engineering is careful, the tests are real-git and mostly non-vacuous, and cargo test --workspace exits 0 in a clean worktree of the head.
Spec
Axis: fail — 2 blocking findings.
F1 (blocking) — the sweep refuses to collect the directories the ticket exists to reclaim
A finished agent worktree whose only untracked content is gitignored build output reports Reason::Holds { Uncommitted } and stands. The .pixi/envs/default copies that docs/cleanup.md names as "the reason the figure is 104 GB rather than about 10" are exactly that shape.
going: []
standing: [".../repos/o/r/ws-one/.claude/worktrees/agent-one"]
reasons: [Holds { losses: Uncommitted(["!! .pixi/"]) }]
report: leaving .../agent-one: 1 uncommitted change(s) (.pixi/) (in .claude/worktrees/agent-one)
The doc's own worked example (- removing .../agent-a49a (5.8 GiB)) is unreachable without --force-worktrees: 5.8 GiB is an env, and an env is !! .pixi/. The closing sentence "Removing the worktree is the way those bytes come back, which is what this does" is false as written.
The sharp end is what the escape costs. Under Insistence::Insisted, weigh makes own_removable Some(..) for every verdict, so the forest collapses to top-level roots: the one flag that reclaims a pixi env also carries past locks, unpushed commits, and SiteKind::NotOurs. Getting the 104 GB back means typing the flag that disables every protection this PR built.
Fix shape: give ignored-only content its own arm (Loss::Ignored, or a distinct Blank) so the words are honest and so something narrower than "past everything" can carry it. At minimum correct docs/cleanup.md's closing sentence and sample output.
F2 (blocking) — plan/act divergence, and the module header's T2 argument is false
reclaim matches plan units to fresh units by identity only, never by blast radius, so a site created after the plan was printed rides out on an approved parent's subtree removal and is named to git worktree remove.
PLAN: removing .../agent-one (12288 bytes), dropping 1 registration(s)
ACT: removed [(".../agent-one", 86016)] forgotten 2 withheld []
PLAN names: ["/workspaces/.../agent-one"]
ACT invokes: git worktree remove /workspaces/.../agent-one
git worktree remove /workspaces/.../agent-one/.claude/worktrees/agent-two
Three documented claims fall to this:
reclaim: "A site that appeared after the plan has no approval, so it cannot be collectable here" — false; it is classified, found collectable, and absorbed.reclaim: "the approved set can shrink between the report and the act and can never grow" — true of the unit count, false of what goes.- The module header's justification for deleting
git worktree prune— "a registration created after the plan was printed is in its blast radius" — the replacement has the same property for nested registrations.
Also false: RemovedWorktree.usage's doc, "The figure the plan measured, so what somebody is told they got back is what they said yes to". act_on takes confirmed, so the figure is the acting pass's re-measurement (86016, not 12288). The clone arm does this correctly, so the two halves of one report now mean different things.
The branch's T2 test, a_registration_created_after_the_plan_is_never_named, covers only a sibling fresh worktree that is dirty — the two properties that make it withheld. The nested-and-clean case its name generalises to is uncovered and false.
Mitigating: the extra site was independently proved collectable, so this instance is not data loss. It is still "something was removed that the plan did not name".
Fix shape: in reclaim, require confirmed.forgets ⊆ planned.forgets and the fresh subtree to contain no site the plan did not name, before acting; withhold otherwise. Report planned.usage.
F3 (non-blocking) — dl <ws> rm now refuses on the ordinary loop
clone_verdict conjoins every nested site's standing into the dl rm guard. With F1, any workspace where an agent built anything refuses:
dl rm would say: 1 uncommitted change(s) (.pixi/) (in .claude/worktrees/agent-one)
shipped predicate: NothingToLose (same clone, today's build)
--force-worktrees is refused on workspace commands, so the only escape is dl <ws> rm --force, which also carries past the clone's own unpushed commits. Over a loop of many short-lived workspaces this trains --force into the daily command and defeats the #171 guard. It also propagates to dl --ls --json, which is wf's frozen contract.
F4 (non-blocking) — the same rule written twice, disagreeing about the same bytes
Git::status_porcelain (clone level) has no --ignored; Git::worktree_dirt (site level) has it, and its doc says that is load-bearing because the removal deletes ignored bytes silently. The same argument applies verbatim to the clone, which --prune's orphan arm and dl rm both rm -rf:
clone with identical ignored bytes at its ROOT: {"nothingToLose":true}
One conjunction, two definitions of dirty.
F5 (non-blocking) — dl --ls cost, unmeasured
enriched_row now calls account_of per row: a worktree list, then per site a status --porcelain --ignored and a rev-list.
shipped read_clone: 3 git calls, 38ms
new account_of: 20 git calls, 260ms (one clone, 8 worktrees, trivial repo)
The reachability probe is re-issued verbatim per site with no memoisation — a free 2x. On a host with 72 worktrees carrying pixi envs, status --ignored is the expensive one, and this is serial and per row. Nothing in docs/performance.md covers --ls, so nothing catches a regression.
Unverified note
ClonePicture::nothing_at_their_place uses Path::exists() (follows symlinks) while walk_sites uses symlink_metadata. For a dangling symlink at a registered place these disagree, which by inspection yields two sites for one Inside. Not probed, so not claimed.
Claims verified
| Claim | Verdict |
|---|---|
T1: nested worktree cannot be in a Collectable |
True — weigh recurses and gates on every_child_removable; payload types have private fields and no public constructor, pinned by the snapshot |
| T2: a registration created after the plan can never be an argument | False as argued — see F2 |
git worktree prune deleted rather than gated |
True — no prune verb or invocation anywhere in rust/ |
public-api.rest.txt hand-transcribed from CI's diff |
Artifact correct — CI regenerates with the pinned tool and diffs, with a checked -eq 0 guard against a vacuous pass. Provenance was risky; the result is right |
--force's plan line uses a semicolon |
True — render.rs:2089, asserted at lifecycle.rs:2232 |
--force-worktrees distinct from --force |
True — separate arg, refused on every non---prune command |
42cd615 to
fb3eede
Compare
|
Both blockers fixed, plus F4, F5's cheap half and the unverified note. Pushed as F1 — fixed, and the root cause was an unreviewed widening of my ownThe reviewer is right, and the finding is sharper than "the words are wrong": weighing ignored The fix is to take The limit is now stated with its reason in the three places it can be read from — Two tests, and the second is the one that matters:
F2 — fixed, and the header's claim rewritten to what is trueReproduced. A confirmed unit is now acted on only when every registration it names was named by the plan
The module header's justification for deleting Two tests: F3 — resolved by F1, and pinned against the shipped answer
F5 — the cheap half taken, the rest declined with a reasonMemoised: the reachability answer is cached per revision for one clone's weighing, and a memoised Declined, and flagged in The unverified note — real, fixed
On the snapshot's provenanceNoting it here as asked: Local: |
690cd55 to
f921280
Compare
The unit is a site and everything nested inside it, decided bottom-up and conjunctively, so a worktree nested inside one being removed cannot be lost with it: the collectable arm of a verdict is reachable only when every nested site handed one back, and the caller passes no child list. That is T1 from PR #442's second review, made unrepresentable rather than guarded. What decides a site is a verdict rather than a boolean: Collectable(Proof) | Stands(NonEmpty<Reason>), where the proof is a private-field witness only a probe that answered can mint, and reasons accumulate up the subtree, so a site that is both dirty and locked reports both and a parent's line names the child. A lock is an unproved, never a loss. The metadata operation is `git worktree remove <the path git printed>`, per registration by name, and the clone-wide `git worktree prune` is deleted rather than gated: its domain is a readdir at act time, so a registration created after the plan was printed was inside its blast radius and no plan could name it. Ownership is a join against this clone's own listing, never the gitfile tail: a live worktree of another repository, nested in one of ours, used to be offered for removal unopposed under a reason that was false. The clone is the root of the same forest, which closes the clone-level guard's blindness to nested worktrees, and it takes #522's `BareCache` and passes it to the probe underneath rather than deciding the tag question for itself. A site's own reachability probe gives no tag account: it names one revision and asks the clone about it, where the tag question is about which of a clone's refs the mirror does not have, so `by_tags: None` is the honest answer. `Unsaved` survives as the wire flattening.
Two blockers from the fresh-context review, and the first one was mine to undo. `worktree_dirt` passed `--ignored`, which `status_porcelain` does not, so one conjunction had two definitions of dirty. The site-level half then stood every worktree carrying an installed `.pixi/envs/default` — 18 of the 72 on the reference host, the difference between 104 GB and about 10 — behind `--force-worktrees`, which is also the flag that carries past a lock and past another repository's worktree. That trades the whole of this sweep's yield for a habit of typing the flag that switches its protections off, and it made `dl <ws> rm` refuse on the ordinary open/work/rm loop. The flag goes; the limit is stated with its reason in three places instead. The second is a real plan/act divergence. `reclaim` matched the two passes' units by identity, which is a root, where the blast radius is a subtree: a site created inside an approved parent after the plan was printed was weighed, found collectable, absorbed into the parent's unit, and handed to `git worktree remove` — one plan naming one registration acting on two. A confirmed unit is now acted on only when every registration it names was named by the plan, and the report carries the plan's byte figure rather than the acting pass's re-measurement, which is what the clone arm beside it has always done. Also: the reachability answer is memoised per revision for one clone's weighing, and `nothing_at_their_place` asks the walk's question so a dangling symlink cannot produce two sites for one place.
17f6f11 to
2edf1c5
Compare
Taken from CI's own diff, since this host has no nightly toolchain or cargo-public-api, and checked against the absences the design pins: Proof carries no impl block at all, GoingDirectory and Standing expose only readers, so none of them can be minted from outside a probe.
A diff in public-api.api.txt is a change to the tier an external consumer may depend on, so this is the one to read rather than skim. `RemovalRefused` stops being a two-arm enum and becomes a struct carrying the whole standing, which is #446's decision reaching the guard: a clone can hold work *and* have a question that could not be put, and a refusal that named one arm would be telling half the truth. It reaches the promised tier only because main promoted `RemovalRefused` into `api` in #520 while this branch was in flight. The consequence is that `agent_worktrees::Standing` is now named at that tier too, through the `standing` field, and its readers -- would_lose, could_not_tell, describe -- are what a consumer reads the refusal with.
|
One thing worth a reviewer's eye that was not in the review, because it did not exist yet: the promised tier moves.
It is a two-arm enum becoming a struct that carries the whole standing, which is #446's decision arriving at the guard: a clone can hold work and have a question that could not be put, and a refusal naming one arm would be telling half the truth. That part was always in this PR. What is new is the tier. #520's removal fold promoted
If the preference is that the promised tier should not move in this PR, the alternative is to keep |
The promised-tier change takes two types out of the binary-surface residual, so the figure three places carry moves from 39 to 37, and the row total from just over six hundred to 595. The guard that caught it exists for exactly this and says so in its own docstring: the sentence is cheapest to fix in the run that moves it.
ef13762 to
a2d8e7e
Compare
As pushed, `public-api.api.txt` named `agent_worktrees::Standing` in `RemovalRefused` and did not promise it: the type has no struct or impl rows in that file at all. A consumer holding only `api` got a struct with a field whose type it could not name, which is devlaunch#531's gap and would have been its third instance after #427 and #516. Promoting it honestly was the other option and it is not small. `Standing` reaches `StandingSite`, `Reason`, `Place`, `Blank`, `Subject` and `NonEmpty<Loss>`, and `agent_worktrees` has over three hundred rows in the binary-surface snapshot. That is most of a module's internal vocabulary arriving in the one tier whose worth is being small and stable. So the promoted shape was either incomplete or far too wide, and rendering at the seam is the only option that is both complete and narrow. It is the move the `--ls --json` payload already makes for the wire, at the same boundary and for the same reason. `RemovalRefused` now carries a `RemovalGrounds`, which is made of `String`: `WouldLose`, `CouldNotTell`, or `BothAtOnce`. Three arms rather than two options, because a standing is non-empty and every reason in it answers one of the two, so "neither" cannot happen -- and both render sites carried a fourth arm apologising for being unreachable, which this deletes rather than comments. `BothAtOnce` is what keeps #446 true across the seam: a refusal still never picks one of two true things to say. Nothing inside `flows` changed. `Standing` is exactly as it was, the domain type still carries the whole standing, and the conversion is a private free function at the boundary rather than a method -- a public constructor taking a `Standing` would put it straight back into the promised tier's signatures. Also removes `Standing::any_unproved`, which this branch added and nothing ever called. `Standing` is in the residual, so an uncalled reader there is rows a consumer can bind to for nothing.
The promised tier is self-sufficient again: zero `agent_worktrees` rows in `public-api.api.txt`, and `RemovalGrounds` is defined there rather than merely named, every leaf an `alloc::string::String`. The residual goes 37 to 36 and `Standing` is what left it. That is a different event from the last drop and the better one: it left because the promise stopped reaching it, not because the promise swallowed it. Both look like a smaller number, so `docs/development.md` now says how to tell them apart -- promoting `Standing` would have moved the count the same way while dragging `StandingSite`, `Reason`, `Place`, `Blank` and `Subject` into the tier that is supposed to stay small. Also drops the row for `Standing::any_unproved`, deleted in the previous commit.
|
Ruling taken. The standing is rendered at the seam; I checked the deciding measurement myself rather than taking it, because it is the kind of claim that is easy to agree with and wrong. On the previous head, What crosses the promise nowEvery leaf is a Named Three arms, not two optionsThe one place I did not follow the wire literally. Nothing inside The residual, and your point about what the number measures37 → 36, and Both look identical as a number, so I put the distinction in One extra, found while checking:
Verification: nine enumerated runs, |
Closes #426. Supersedes #442.
An agent harness working inside a devcontainer makes its own git worktrees under
<clone>/.claude/worktrees/<name>/, one per task, and nothing ever collected them: 72directories, 104.5 GB, about 82% of everything under
repos/on the host in #426, one cloneholding 55 GB on its own. Every one of them sat inside a clone belonging to a live devpod
workspace, so
--prune's orphan rule not only missed them, it must never fire there.#442 built that and failed review three times on the same shape: a correct guard, a correct
fix, and the same defect class back through a neighbouring door. Map #444's keystone found why
and this is the rebuild on what it decided. The two findings the last review left standing have
no representation here rather than a guard each.
The unit is a site and everything nested in it (#445)
A site is a place inside the clone,
.claude/worktrees/<leaf>, nested as deep as theharness nested it. A site is collectable only when it and every site nested inside it are,
decided bottom-up.
That is not a rule the code checks.
weighrecurses into the children itself and the callerpasses no child list, so the collectable arm is reachable only when every child's recursion
handed one back. Containment edges come from the filesystem walk, never from
prefix-comparing recorded paths: on a host every recorded path is a string about another
machine, and a worktree of a different repository has no edge in this clone's listing at all.
Recorded paths are used for exactly two things, neither of which resolves them: the join key,
and the argument to
git worktree remove.git worktree pruneis deleted, not gated. Its domain is a readdir over$GIT_DIR/worktreesat act time, so no plan-time unit can equal it. The one metadata operationis
git worktree remove <the path git printed>, per registration, by name.The verdict (#446)
A reason is proved unsafe or could not be proved, and reasons accumulate up the subtree, so
a site that is both dirty and locked reports both and a parent's line names the child that
caused it.
Proofhas private fields, noDefaultand no public constructor: it is mintableonly from a probe that answered, so "nothing objected" and "nothing was asked" stop being the
same value.
Four questions with different scopes, two of which the shipped code had transposed. Dirt is
per working tree (a nested worktree has an index of its own, and
.claude/worktrees/isordinarily gitignored, so the clone's own
git statuscannot answer for it) andreachability is per repository. Ignored files count, because
git worktree removedeletesa worktree whose only content is gitignored, exit 0 and silent. A lock is an unproved,
never a loss.
prunableis never read andHeldis gone.Unsavedsurvives as the wire flattening, additively: a standing holding both kinds emits bothwouldLoseandcouldNotTell, so no reader keyed on key presence breaks.T1 and T2, unrepresentable
T1 (a worktree nested inside one being removed is protected by neither path, including when
locked). There is no
Collectablewhose subtree holds aStands, because the recursion thatproduces one visits every child. The three probes on review 5019431339 are the tests, and each
asserts the absence of a unit in the plan rather than a guard firing:
a_nested_worktree_holding_an_uncommitted_note_stands_everything_above_ita_nested_worktree_holding_an_unpushed_commit_stands_everything_above_ita_locked_nested_worktree_cannot_be_inside_anything_that_goesPlus
a_nested_worktree_holding_work_stands_the_whole_subtree_end_to_endthrough the realcommand's two passes.
T2 (the metadata prune reaches registrations neither pass enumerated, so a registration
created after the plan is deleted by the next run). There is no clone-wide prune to reach past
a plan. The only invocation is
worktree remove <a Recorded>, andRecorded's onlyconstructor parses a listing — so a registration created after the plan was printed has never
been one.
a_registration_created_after_the_plan_is_never_namedasserts the spawn log containsno invocation naming it.
Ownership is a registration join (#463)
A gitfile tail says a directory is a worktree of some repository. Reading it as this clone's
is how a live worktree of another repository, nested in one of ours and holding uncommitted
work, was offered for removal unopposed under the printed reason "git has already forgotten
it" — which was false.
SiteKind::OursHerenow carries the joinedRecordedand derivesadminfrom it inside its constructor, so ownership is a positive answer from a listingdevlaunch owns, and the borrowed-index collision is unconstructible rather than fail-safe.
Those sites stand, are reported, and are never probed.
What carried over from #442
The four-way classification's content (what each arm asks and why), the container-path-aware
dirty check through the clone's admin directory, the
.bare-first reachability with the"as of the last fetch" sentence,
--force-worktreesas a flag distinct from--forcewith theparse test that
--forcealone does not reach a worktree, the--ls --sizeattribution andits
disk.worktreesJSON key, the S4 exclude-by-what-a-thing-is rule, and the S6 gitfilenormalisation guard.
MetadataGatedid not: the S1 fix was a correct guard on an operationwhose blast radius cannot be named at plan time, and the decision deletes the operation. T3
evaporates with it — a refused or partial removal forgets nothing, because forgetting is a
consequence of a completed subtree removal. N4 closes structurally: there is no fallthrough arm
to fall through to.
Six false claims in the module and in
docs/cleanup.mdare corrected, of whichagent_worktrees.rs:268-269was a live deletion rather than stale prose.🤖 Generated with Claude Code
After review
Two blocking findings, both fixed; see the reply for the full account.
Ignored bytes are weighed at neither scope.
worktree_dirtpassed--ignoredwherestatus_porcelaindoes not, which was one conjunction with two definitions of dirty and, worse,put every worktree carrying an installed
.pixi/envs/defaultbehind--force-worktrees— theflag that also carries past a lock and past another repository's worktree. Getting the 104 GB
back would have meant typing the flag that switches off every protection here, and
dl <ws> rmwould have started refusing on the ordinary open/work/rm loop. The limit is now stated with its
reason in
worktree_dirt, inWeigher::cleanand indocs/cleanup.md, including what weighingit cost, and whether ignored bytes should be weighed is recorded as one question for both scopes.
The acting pass compares radii, not roots.
reclaimmatched plan units to fresh units byidentity, so a site created inside an approved parent after the plan was printed was absorbed into
that parent's subtree removal and handed to
git worktree remove— one plan naming oneregistration acting on two. A confirmed unit is now acted on only when every registration it names
was named by the plan, and the report carries the plan's byte figure rather than the acting pass's
re-measurement. The module header's justification for deleting
git worktree pruneis rewrittento what is actually true: the metadata radius is one name and the byte radius is the approved
subtree, re-checked under the lock.
Also: the reachability answer is memoised per revision,
nothing_at_their_placeasks the walk'squestion so a dangling symlink cannot produce two sites for one place, and
Reason::Holdsboxesits losses now that #522 has grown them.
On the snapshot's provenance, since it is fair to ask:
public-api.rest.txtis transcribedfrom CI's own
diff -u, because this host has no nightly toolchain and nocargo-public-api.The artifact is verified independently of that route — CI regenerates with the pinned tool and
diffs, with a
checked -eq 0guard against a vacuous pass — and the absences the design dependson are checked by hand each time:
Proofcarries noimplblock at all, andGoingDirectory,GoingRegistrationandStandingexpose only readers.