pgw#1515: coarse residency cells — the packing half and the hold-while-hot schedule, both in the planner - #1083
Draft
PaulFidika wants to merge 2 commits into
Draft
pgw#1515: coarse residency cells — the packing half and the hold-while-hot schedule, both in the planner#1083PaulFidika wants to merge 2 commits into
PaulFidika wants to merge 2 commits into
Conversation
PaulFidika
force-pushed
the
1515-residency-cells
branch
from
August 20, 2026 00:54
29c1ff7 to
8f75203
Compare
…point)
The packing and scheduling halves of phase 3's residency work, both PLANNER-side
so they sit above the mechanism seam and serve software streaming and the varena
facade alike.
* Cells: pack_cells groups leaves into back/unback/stream units at a configurable
target size, small leaves first. The plan reports the residual granularity tax
against the leaf-granular baseline pgw#1507 measured (13.2%), so the
elimination is a subtraction of two published numbers.
* Schedule: plan_residency decides CALL_BOUNDARY vs PER_STEP from the assigned
{VRAM, RAM} pair and the cell layout. hold_component() is the call-boundary
swap and REFUSES when the plan did not admit it.
* The default geometry is leaf-granular with no page granularity, so every
pre-cell number is unchanged.
…e tests that were not discriminating Three things the red-arm audit and the gate found: * The call-boundary reservation was streams x the largest cell of ANY kind, including the forced core — which packing makes large (500 sub-floor leaves become ONE cell). A cell that is resident in every arrangement never travels, so reserving the ring for it denied CALL_BOUNDARY to budgets that plainly admit it. Caught by a new test, red 5/5 under the old arithmetic. * test_small_leaves_are_packed_first passed under a REVERSED sort (audit M1, missed 0/5) — it asserted an outcome greedy fill produces either way. Replaced with the outcome small-first actually buys: a leaf at or above the cell target lands ALONE, so small leaves are never welded to a big one's residency. Red 5/5 under the reversal. * hold_component has no production caller yet — the serve path has no call-BOUNDARY hook and adding one is the half that must be measured. Baselined in scripts/unreached_surface_baseline.txt with an owner and an expiry (the pgw#1515 GPU sweep either produces the caller or the deletion), not left to go red on master.
PaulFidika
force-pushed
the
1515-residency-cells
branch
from
August 20, 2026 00:57
8f75203 to
8a53986
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 3's residency half (tracker
pgw#1515): the packing half and the schedulinghalf, both PLANNER-side so they sit above the mechanism seam and are available to
the software streaming rung and the pgw#1507 varena facade through the same
plan_residencycontract.Rebased over
11bf60b3(PR #1068, the varena facade) — the shareddiscover_leaveswalk it introduced is where the component is now stated.
The two measured legs this is built on
page alignment — 1.830 GiB of weight in 2.072 GiB of span across 239 regions —
plus ~98 µs of page-table work per streamed leaf per forward. Its close-out's
own directive: a compiled cell must never page the page table per-leaf
per-forward; coarser cells amortise both.
model_offload1.57× againstpartial_stream1.91×at the same 1.9 GB peak, because one is taxed per-CALL and the other per-STEP.
What this adds
CellPolicy— a mechanism's paging geometry as three numbers (cell_bytes,granularity_bytes,intra_align_bytes). The defaultLEAF_CELLSstates nopage granularity, and under it every number the planner produces is
bit-for-bit what pgw#1497 shipped — a standing test, not an argument.
pack_cells— deterministic and order-independent. Small leaves first. A cellnever mixes forced with streamable leaves (one fate) and never straddles two
components (the call-boundary unit).
plan_residency(..., cells=)— the same greedy largest-first fill run to thesame fixed point, with the CELL as the unit, each cell charged its MAPPED SPAN.
device_bytesbecomesresident_span_bytes + window_bytes, which is what keepsfitsexact under a mechanism that pages.granularity_tax_bytes,leaf_granular_tax_bytes(the same resident leaves priced one region each),tax_eliminated_bytes, both ratios — so the elimination is a subtraction of twonumbers the plan computed.
ResidencySchedule+hold_component()— CALL_BOUNDARY whenhot_component_bytes + streams × largest cell ≤ VRAMandcold_component_bytes ≤ RAM(when the RAM half is stated), PER_STEP otherwise.Decided in the plan, deterministic from budget + cell layout, never a runtime
reaction to pressure.
hold_componentREFUSES when the plan did not admit it.Admission-first arithmetic unchanged. compiled⊥offload untouched
(
context.py:582-598is never consulted).🔴 The finding the CPU work already produced, and it cuts against coarse cells
A cell is the stream unit, so the in-flight window is
streams × the largest streamed CELL. Under PER_STEP that comes straight out of the resident set andgrows linearly with the cell size while the tax only falls. Planner arithmetic on
a 239-leaf / 1.830 GiB census shaped like pgw#1507's, at a 1.2 GiB budget:
Whole-census tax, budget-independent: 244 MiB / 13.03 % leaf-granular — which
reproduces pgw#1507's measured 13.2 % to a tenth of a point — falling to 1.71 % at
64 MiB cells and 0.32 % at 256 MiB. The residual is bounded by
granularity / cell_byteswhatever the leaves look like; the leaf-granular tax isunbounded as leaves shrink.
So the tax elimination is real and monotone, and under PER_STEP the weight held
at a fixed lease peaks at a moderate cell size and then collapses. Under
CALL_BOUNDARY nothing streams mid-forward and the elimination is pure gain. That
curve is what the GPU sweep has to price, and it is why the sweep's cell axis must
treat the leaf arm as a real competitor rather than a baseline.
Not done — and the first one is the issue's verdict
model_offloadfloor, 5 %}, sd1.5. Target: beatmodel_offload's measured1.57× at its own floor and hold streaming's advantage below it. Not run here —
the card is on an image window and an overnight soak.
plan_layouthas its ownpacking and
ArenaLayout.costs()feeds LeafCosts already priced at aligned span,so passing
cells=there today would make the tax report read ~0 for a mechanismstill paying it — a dead instrument. The end state is one packing decision
(
plan_layoutconsumesplan.cells); it movesresident_bytessemantics on aGPU-verified path, so it lands with the sweep.
partial_unload/partial_loadstill compute their delta offresident_bytesrather than
resident_span_bytes. Identical under the leaf geometry, wrong byexactly the tax for a paging mechanism. Named, not changed blind.
fast gatesis red on this PR and it is NOT this PR — proven, not assumedThree reds in a row, each revealed by fixing the one in front (the gate stops at
its first failing step):
pgw#849 guard 2— mine,hold_componentas NEW unreached surface. Fixed:baselined with an owner and an expiry.
pgw#849 guard 2again — five stalegen_worker.cli.*rows. Not mine, andfixed on master by PR pgw#1521: delist five CLI rows that acquired production callers — unblock pgw master's
fast gates#1084 (51114dd2) while this sat. Rebased onto it.pgw#931 guard — config reads outside the pipeline— 5 problems incli/compile.py:294andcli/credentials.py:59/129/133. Reproduces on apristine
origin/master(ccdb490c) worktree with zero changes, so pgwmaster's
fast gatesis red right now and every PR inherits it. The fix is a§1.18 classification per read — a judgement about the CLI's config posture,
belonging to whoever landed those reads. Not touched here.
Everything this PR owns is green locally: 95 passed / 3 skipped across the five
residency test modules,
ruffclean,mypy src/gen_worker testsclean over thegate's 618 files, and
lint_incident_test_names/lint_mypy_ratchet/lint_unreached_surface/lint_fence_symbolsall pass.