Skip to content

pgw#1515: coarse residency cells — the packing half and the hold-while-hot schedule, both in the planner - #1083

Draft
PaulFidika wants to merge 2 commits into
masterfrom
1515-residency-cells
Draft

pgw#1515: coarse residency cells — the packing half and the hold-while-hot schedule, both in the planner#1083
PaulFidika wants to merge 2 commits into
masterfrom
1515-residency-cells

Conversation

@PaulFidika

@PaulFidika PaulFidika commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Phase 3's residency half (tracker pgw#1515): the packing half and the scheduling
half, both PLANNER-side so they sit above the mechanism seam and are available to
the software streaming rung and the pgw#1507 varena facade through the same
plan_residency contract.

Rebased over 11bf60b3 (PR #1068, the varena facade) — the shared discover_leaves
walk it introduced is where the component is now stated.

The two measured legs this is built on

  • pgw#1507, on the card: 13.2 % (248 MiB) of sd1.5's weights lost to per-leaf
    page alignment — 1.830 GiB of weight in 2.072 GiB of span across 239 regions —
    plus ~98 µs of page-table work per streamed leaf per forward. Its close-out's
    own directive: a compiled cell must never page the page table per-leaf
    per-forward; coarser cells amortise both.
  • pgw#1497, on the card: model_offload 1.57× against partial_stream 1.91×
    at the same 1.9 GB peak, because one is taxed per-CALL and the other per-STEP.

What this adds

  • CellPolicy — a mechanism's paging geometry as three numbers (cell_bytes,
    granularity_bytes, intra_align_bytes). The default LEAF_CELLS states no
    page granularity, and under it every number the planner produces is
    bit-for-bit what pgw#1497 shipped
    — a standing test, not an argument.
  • pack_cells — deterministic and order-independent. Small leaves first. A cell
    never mixes forced with streamable leaves (one fate) and never straddles two
    components (the call-boundary unit).
  • plan_residency(..., cells=) — the same greedy largest-first fill run to the
    same fixed point, with the CELL as the unit, each cell charged its MAPPED SPAN.
    device_bytes becomes resident_span_bytes + window_bytes, which is what keeps
    fits exact under a mechanism that pages.
  • The plan reports the tax it did and did not pay — granularity_tax_bytes,
    leaf_granular_tax_bytes (the same resident leaves priced one region each),
    tax_eliminated_bytes, both ratios — so the elimination is a subtraction of two
    numbers the plan computed.
  • ResidencySchedule + hold_component() — CALL_BOUNDARY when
    hot_component_bytes + streams × largest cell ≤ VRAM and
    cold_component_bytes ≤ RAM (when the RAM half is stated), PER_STEP otherwise.
    Decided in the plan, deterministic from budget + cell layout, never a runtime
    reaction to pressure. hold_component REFUSES when the plan did not admit it.

Admission-first arithmetic unchanged. compiled⊥offload untouched
(context.py:582-598 is never consulted).

🔴 The finding the CPU work already produced, and it cuts against coarse cells

A cell is the stream unit, so the in-flight window is streams × the largest streamed CELL. Under PER_STEP that comes straight out of the resident set and
grows linearly with the cell size while the tax only falls. Planner arithmetic on
a 239-leaf / 1.830 GiB census shaped like pgw#1507's, at a 1.2 GiB budget:

cell target cells resident WEIGHT window residual tax leaf-granular tax
leaf 239 1118.4 MiB 19.6 MiB 8.01 % 8.01 %
16 MiB 156 1112.0 MiB 23.9 MiB 7.73 % 15.29 %
64 MiB 33 1086.4 MiB 115.3 MiB 1.62 % 17.09 %
256 MiB 8 629.7 MiB 507.5 MiB 0.37 % 8.31 %

Whole-census tax, budget-independent: 244 MiB / 13.03 % leaf-granular — which
reproduces pgw#1507's measured 13.2 % to a tenth of a point — falling to 1.71 % at
64 MiB cells and 0.32 % at 256 MiB. The residual is bounded by
granularity / cell_bytes whatever the leaves look like; the leaf-granular tax is
unbounded as leaves shrink.

So the tax elimination is real and monotone, and under PER_STEP the weight held
at a fixed lease peaks at a moderate cell size and then collapses.
Under
CALL_BOUNDARY nothing streams mid-forward and the elimination is pure gain. That
curve is what the GPU sweep has to price, and it is why the sweep's cell axis must
treat the leaf arm as a real competitor rather than a baseline.

Not done — and the first one is the issue's verdict

  • GPU pricing. {leaf, 64 MiB, 256 MiB, component} × {50 %, 25 %, the ~1.9 GB
    model_offload floor, 5 %}, sd1.5. Target: beat model_offload's measured
    1.57× at its own floor and hold streaming's advantage below it. Not run here —
    the card is on an image window and an overnight soak.
  • The arena does not consume cells yet, deliberately. plan_layout has its own
    packing and ArenaLayout.costs() feeds LeafCosts already priced at aligned span,
    so passing cells= there today would make the tax report read ~0 for a mechanism
    still paying it — a dead instrument. The end state is one packing decision
    (plan_layout consumes plan.cells); it moves resident_bytes semantics on a
    GPU-verified path, so it lands with the sweep.
  • partial_unload/partial_load still compute their delta off resident_bytes
    rather than resident_span_bytes. Identical under the leaf geometry, wrong by
    exactly the tax for a paging mechanism. Named, not changed blind.

⚠️ fast gates is red on this PR and it is NOT this PR — proven, not assumed

Three reds in a row, each revealed by fixing the one in front (the gate stops at
its first failing step):

  1. pgw#849 guard 2mine, hold_component as NEW unreached surface. Fixed:
    baselined with an owner and an expiry.
  2. pgw#849 guard 2 again — five stale gen_worker.cli.* rows. Not mine, and
    fixed on master by PR pgw#1521: delist five CLI rows that acquired production callers — unblock pgw master's fast gates #1084 (51114dd2) while this sat. Rebased onto it.
  3. pgw#931 guard — config reads outside the pipeline — 5 problems in
    cli/compile.py:294 and cli/credentials.py:59/129/133. Reproduces on a
    pristine origin/master (ccdb490c) worktree with zero changes
    , so pgw
    master's fast gates is red right now and every PR inherits it. The fix is a
    §1.18 classification per read — a judgement about the CLI's config posture,
    belonging to whoever landed those reads. Not touched here.

Everything this PR owns is green locally: 95 passed / 3 skipped across the five
residency test modules, ruff clean, mypy src/gen_worker tests clean over the
gate's 618 files, and lint_incident_test_names / lint_mypy_ratchet /
lint_unreached_surface / lint_fence_symbols all pass.

@PaulFidika
PaulFidika force-pushed the 1515-residency-cells branch from 29c1ff7 to 8f75203 Compare August 20, 2026 00:54
…point)

The packing and scheduling halves of phase 3's residency work, both PLANNER-side
so they sit above the mechanism seam and serve software streaming and the varena
facade alike.

* Cells: pack_cells groups leaves into back/unback/stream units at a configurable
  target size, small leaves first. The plan reports the residual granularity tax
  against the leaf-granular baseline pgw#1507 measured (13.2%), so the
  elimination is a subtraction of two published numbers.
* Schedule: plan_residency decides CALL_BOUNDARY vs PER_STEP from the assigned
  {VRAM, RAM} pair and the cell layout. hold_component() is the call-boundary
  swap and REFUSES when the plan did not admit it.
* The default geometry is leaf-granular with no page granularity, so every
  pre-cell number is unchanged.
…e tests that were not discriminating

Three things the red-arm audit and the gate found:

* The call-boundary reservation was streams x the largest cell of ANY kind,
  including the forced core — which packing makes large (500 sub-floor leaves
  become ONE cell). A cell that is resident in every arrangement never travels,
  so reserving the ring for it denied CALL_BOUNDARY to budgets that plainly
  admit it. Caught by a new test, red 5/5 under the old arithmetic.
* test_small_leaves_are_packed_first passed under a REVERSED sort (audit M1,
  missed 0/5) — it asserted an outcome greedy fill produces either way.
  Replaced with the outcome small-first actually buys: a leaf at or above the
  cell target lands ALONE, so small leaves are never welded to a big one's
  residency. Red 5/5 under the reversal.
* hold_component has no production caller yet — the serve path has no
  call-BOUNDARY hook and adding one is the half that must be measured. Baselined
  in scripts/unreached_surface_baseline.txt with an owner and an expiry (the
  pgw#1515 GPU sweep either produces the caller or the deletion), not left to go
  red on master.
@PaulFidika
PaulFidika force-pushed the 1515-residency-cells branch from 8f75203 to 8a53986 Compare August 20, 2026 00:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant