Skip to content

pgw#1514: skeleton.build asks WHY before saying WHAT — a collected tree is not an absent index - #1085

Open
PaulFidika wants to merge 1 commit into
masterfrom
1514-collected-not-absent
Open

pgw#1514: skeleton.build asks WHY before saying WHAT — a collected tree is not an absent index#1085
PaulFidika wants to merge 1 commit into
masterfrom
1514-collected-not-absent

Conversation

@PaulFidika

Copy link
Copy Markdown
Contributor

Supersedes #1078, which GitHub auto-closed (not merged) when its base 2204-shortwrite was deleted on #1072's merge. Same commit, rebased onto master. Both pins are now on master: ccdb490c (pgw#1511+#1513, the shared helper this calls) and c24636bb (pgw#1518).

skeleton.build reported "carries no model_index.json" about trees that HAVE one. Path.is_file() follows the link, so "this tree never had an index" and "this tree's objects were collected" arrived at the same check as the same False — and the second is a fact about the store that the refusal reported as a fact about the checkpoint. pgw#1308 finding 3, for the fourth time.

Measured, not theorised

A projected tree's manifest pin is the only GC root its objects have. On a real 5.6 GB tensorhub/anima@composed-v3 tree (se#790):

removed pin -> collect_garbage
  -> examined=1211  reachable=0  deleted=1211  bytes_deleted=5,640,566,685
stubs=3   dangling symlinks=14   model_index.json among them

Against that same tree, carries no model_index.json becomes a refusal naming 14 collected entries, saying the bytes must be RE-FETCHED rather than re-pinned, and quoting the false string it replaces so anyone grepping the symptom lands on the cause.

Three decisions worth reviewing

The walk is whole-tree. The index merely dies first, so guarding one file would relocate the wrong message to the next component's config rather than remove it. One walk, before anything is concluded.

No second source of truth — twice. The refusal calls #1072's collected_entries / collected_refusal; an independent implementation of both was written first and thrown away. And the tests fold into tests/test_projected_tree_reading.py rather than adding a module — that file's subject line ("who may, who must refuse, and what they say") already covers this caller; the eager bridge is one reader, the skeleton is the other. Three cases, zero new files. An earlier revision added tests/test_collected_objects_pgw1514.py, which pgw#1362 / DESIGN-RULINGS 4.34b correctly refuses — a filename says when a test was written; the tree needs it to say what it exercises. The guard was run, not routed around.

The original wording survives and is now true whenever reached, with a case asserting it — swallowing the genuine case would trade one wrong message for another.

Red proven — and the first attempt at proving it was vacuous

Recorded because it is how a red proof silently stops proving anything: the first revert used git checkout --, which restores from HEAD — still carrying the fix — and reported a cheerful 10 passed. Reverting to the base branch's skeleton.py instead:

test_skeleton_build_does_not_call_a_collected_tree_index_less   FAILED
test_skeleton_build_walks_the_WHOLE_tree_not_just_the_index     FAILED
2 failed, 8 passed

The third case passes without the fix, correctly — it guards preserved behaviour, not new behaviour.

projected-tree suites          39 passed
ruff · mypy                    clean
incident-naming guard          passes, baseline unchanged (126, and it only shrinks)

Filed by the th#2204 lane on this lane's measurement; owned here per coordinator assignment. Possible visible pause: a re-pin relay outranks this issue — if that arrives mid-review I checkpoint and return.

…ee is not an absent index

`skeleton.build` reported "carries no model_index.json" about trees that HAVE
one. `Path.is_file()` FOLLOWS the link, so "this tree never had an index" and
"this tree's objects were collected" arrived at the same check as the same
False — and the second is a fact about the STORE that the refusal reported as a
fact about the CHECKPOINT. pgw#1308 finding 3, for the fourth time.

MEASURED, not theorised. A projected tree's manifest pin is the ONLY GC root
its objects have. On a real 5.6 GB `tensorhub/anima@composed-v3` tree (se#790):

    removed pin -> collect_garbage
      -> examined=1211 reachable=0 deleted=1211 bytes_deleted=5,640,566,685
    stubs=3   dangling symlinks=14   model_index.json among them

and against that same tree this change turns `carries no model_index.json`
into a refusal that names 14 collected entries, says the bytes must be
RE-FETCHED rather than re-pinned, and quotes the false string it replaces.

THE WALK IS DELIBERATELY WHOLE-TREE. The index merely DIES FIRST, so guarding
one file would relocate the wrong message to the next component's config
rather than remove it. One walk, before anything is concluded.

NO SECOND SOURCE OF TRUTH — twice over.

  * The refusal calls `projection.collected_entries` / `collected_refusal`,
    pgw#1513's shared helper, which is why this is stacked on
    `2204-shortwrite`. An independent implementation of both was written first
    and thrown away: two hand-written strings is how this shape reached four
    callers, and se#790 found a fifth in its OWN loader while documenting the
    lesson in the same file's docstring.
  * The TESTS fold into `tests/test_projected_tree_reading.py` rather than
    adding a module. That file's subject — "Reading a PROJECTED tree: who may,
    who must refuse, and what they say" — already covers this caller; the
    eager bridge is one reader and the skeleton is the other. Three cases, zero
    new files. An earlier revision of this branch added
    `tests/test_collected_objects_pgw1514.py`, which pgw#1362 /
    DESIGN-RULINGS 4.34b correctly refuses: a file name says WHEN a test was
    written, the tree needs it to say WHAT it exercises. The guard was run, not
    routed around.

RED PROVEN, against the BASE BRANCH's skeleton rather than a working-tree
revert (the first attempt reverted from HEAD, which still carried the fix and
reported a vacuous green — recorded because that is how a red proof silently
stops proving anything):

    test_skeleton_build_does_not_call_a_collected_tree_index_less   FAILED
    test_skeleton_build_walks_the_WHOLE_tree_not_just_the_index     FAILED
    2 failed, 8 passed

The third case, `test_a_tree_that_GENUINELY_has_no_index_still_says_so`, passes
WITHOUT the fix and that is correct: it guards preserved behaviour, not new
behaviour. The original wording survives and is now true whenever it is
reached — a fix that swallowed the genuine case would trade one wrong message
for another, which is the mistake this issue is about.

39 passed across the projected-tree suites; ruff and mypy clean; the
incident-naming guard passes with the baseline unchanged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant