Skip to content

perf(gc): exact-start bitmap + page-class-first pointer validation; header-first buffer-registry rejection - #9919

Draft
proggeramlug wants to merge 4 commits into
PerryTS:mainfrom
proggeramlug:perf/gc-pointer-validation
Draft

perf(gc): exact-start bitmap + page-class-first pointer validation; header-first buffer-registry rejection#9919
proggeramlug wants to merge 4 commits into
PerryTS:mainfrom
proggeramlug:perf/gc-pointer-validation

Conversation

@proggeramlug

@proggeramlug proggeramlug commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Runtime-only, on main 504e180d0. Written by codex from the campaign's I7-view profile; not yet compiled — gates and the cc-rig measurement run on perrymaster and will be appended here.

Why

On the current best cc bundle the collector's per-pointer validation is the largest remaining collector item: gc::trace::ValidPointerSet::find_arena_floor 5.16 % self, ValidPointerSet::contains 0.95 %, gc_pointer_and_type_from_value 1.37 % (≈ 7.5 % of the main thread), and the buffer-registry probes is_registered_buffer_slow 1.61 % + is_registered_buffer 1.15 % + is_uint8array_buffer_slow 0.90 % (3.7 %).

What changes

  • The synchronous exact ValidPointerSet census clears each arena block's object-start bitmap and stamps every walkable header it visits (including headers written by codegen's inline allocator). The allocation path gains no per-allocation bitmap write.
  • ValidPointerSet::contains asks the direct page-class metadata and the exact-start bitmap before the two-level sorted-run floor lookup. Non-arena and page-table-control cases keep the sorted-run fallback; malloc membership stays an exact BTreeSet union after an arena miss; enclosing_object still uses the sorted runs.
  • buffer::is_registered_buffer uses try_read_tracked_gc_header to reject a tracked non-GC_TYPE_BUFFER object before the address filter and the exact registries. A header match is never accepted as a positive: managed buffers, headerless external buffers and SharedArrayBuffer backings keep the authoritative registry path.
  • Diagnostics: PERRY_GC_DIAG prints one [gc-pointer-validation] and one [gc-runtime-handles] line per run; PERRY_BUFFER_DIAG gains header_rejects= and renames its downstream count to filter_probes=.

Handle scopes (audit, no change)

js_segments_view_open opens one scope and pushes one root; _next/_code_point_at open none. js_regexp_new opens one scope and pushes two roots. Neither can be hoisted to a caller-owned scope without an ABI change (raw arguments cannot be refreshed after a relocation). Counts on the cc turn come from the new [gc-runtime-handles] line.

Tests

Two named sabotage tests (the page-class exact-start arm; the Buffer header-class rejection), authored, not yet executed.

Gates

Run: rustfmt, scripts/check_file_size.sh, scripts/addr_class_inventory.py (pass; two pre-existing stale baseline reductions reported, not rewritten), git diff --check. Not run: cargo build --release -p perry default features, the runtime suite, the two sabotage tests, nm. Draft until perrymaster's seam runs them.

Gate history

  • c1ce04915: cargo build --release -p perry rc=101 on perrymaster: error[E0364]: emit_runtime_handle_diag is private, and cannot be re-exported (gc/roots.rs:21) and error[E0603] at the gc/mod.rs:1398 call. Nothing past the first gate ran.
  • 153616859 (head): one-word fix, pub(super)pub(crate) on the emitter, matching its scan-state siblings. Re-gating from the top (default features, archive feature set, lib tests, the two sabotage tests) before the relink and rows.
  • 153616859 on perrymaster: cargo build --release -p perry rc=0; archives with --features perry-runtime/wasm-host rc=0; RUST_TEST_THREADS=1 cargo test --release -p perry-runtime --lib -- --test-threads=1: 3232 passed, 0 failed, 4 ignored, with the named tests green (managed_non_buffer_header_rejects_before_registry, object_start_bitmap_stamps_only_maps_and_clears_on_reset, page_class_exact_membership_rejects_pointer_just_outside_arena_chunk, valid_pointer_membership_spans_every_census_run_including_the_partial_one_7646, the build_valid_pointer_set_* cycle-state tests, the forwarding-target and inline-generation sabotage tests, the four page-class-table tests, buffer_span_guards_require_registered_buffer, try_read_tracked_gc_header_rejects_unrelated_allocation). scripts/addr_class_inventory.py passed. scripts/gc_runtime_root_holders.py failed: four new rule T thread-locals without verdicts and the PASS1_MARKED source pin for gc/mod.rs changed.
  • 790519622 (head): inventory-only commit. Verdicts: VALID_POINTER_DIAG not_a_gc_pointer (seven u64 counters, no pointer field); TEST_PAGE_CLASS_EXACT_START_ANSWERS, TEST_DISABLE_PAGE_CLASS_EXACT_START, TEST_DISABLE_BUFFER_HEADER_CLASS test_only (all #[cfg(test)]). PASS1_MARKED re-audited: the snapshot is stored after mark propagation and taken at sweep entry within one synchronous full cycle; both boundaries require no minor in progress and a non-budgeted cycle; evacuation runs only on the minor path; this branch's gc/mod.rs change is two diag-emitter calls in the exit teardown that touch neither the snapshot nor phase control. Pin recomputed with the script's recipe. Gate rc=0, self-test OK. No runtime source changed, so the perrymaster measurement on 153616859 stands for this head.

Measured (perrymaster, cc 3300-char reply, 5-round paired rotation vs main 504e180d0, runtime relinked on main's cache)

turn CPU s min / mean per-round Δ peak RSS settled 120 s
main 3.58 / 3.62 595–613 490
this PR 3.46 / 3.58 −0.01, −0.19, −0.02, −0.03, +0.07 within ±6 MB 519 (+29, one row each)

400-char turn: +0.03…+0.05 s (slower, 2/2). Absolutes on this host carry a foreign-service shift; the pairs are interleaved.

Buffer diag (whole last line of the reply): main probes=11534337 admits=306091 (2.65 %) rejected=11228246 (97.35 %) true_positives=241228 (78.8 % of admits); this PR header_rejects=10485761 filter_probes=15338 admits=14776 (96.34 %) rejected=562 true_positives=14776 (100 % of admits). 10.5 M of 11.5 M probes per reply now die on the header class; admits are all true.

Perf draw (3,468 main-thread samples): ValidPointerSet::find_arena_floor (5.16 % before) and ::contains (0.95 %) are gone from the top-25; in their place addr_class::try_read_tracked_gc_header 1.59 %, trace::classifier_valid_object_start 0.94 %, young_log::addr_is_minor_relevant 0.90 %, and ArenaObjectCursor::next_budgeted 2.41 % (the synchronous census stamping the object-start bitmap). The typed-array registry probes are not covered by the header reject: lookup_registered_typed_array_kind 2.35 %, is_uint8array_buffer_slow 1.72 %. Collector/mutator on this no-tier bundle 51.4 / 48.6 %.

Reading: the validation cost moved (−6 % of samples in the floor search, +5 % in header read + classifier + bitmap census) and the buffer-probe cost was mostly the address filter, which the header reject replaces at similar price. Net ≈ flat. Stays draft; not a landing candidate in this form. What the data says to do instead: (1) stop issuing 11.5 M buffer/typed-array probes per reply at the callers (97 % are rejections), (2) do not pay a census stamp per arena block for a bitmap the header read already answers. The whole-run [gc-pointer-validation] / [gc-runtime-handles] lines are captured by a graceful run queued separately.

Whole-run counters (one graceful 1-turn 3300 reply per arm; CPU columns void — taken under a concurrent compile)

main: [buffer-diag] probes=27262977 admits=7268934 (26.66 %) rejected=19994043 (73.34 %) true_positives=241349; [gc-time] minor_us=1960601 step_us=518191 share_permille=272.
this PR: [gc-pointer-validation] contains=4708524 page_hits=0 page_rejects=0 run_fallbacks=0 malloc_probes=0 malloc_hits=0 enclosing=1569; [gc-runtime-handles] scopes=18736200 pushes=42238507 pushes_per_scope=2.254 max_slots=3333; [buffer-diag] header_rejects=27262977 filter_probes=242338 admits=241597 (99.69 %) rejected=741 true_positives=241593; [gc-time] minor_us=1924822 step_us=530609 share_permille=270.

Two facts these settle: (1) the 27.26 M registry probes became exactly 27.26 M header reads — the probe count did not change, its price moved from the address filter to the header read, which is the flat CPU; (2) contains ran 4.7 M times with zero page hits, page rejects, run fallbacks or malloc probes, so the page-class + exact-start path never engaged in this run — either the counters sit on a path the census-stamped bitmap does not reach, or the page-class metadata is unavailable at that point. Until that is explained, the pointer-validation half of this PR is unmeasured, not flat. The 27 M probes are attacked at their source by #9937 (no probe for receivers whose kind the dispatch already knows); if that lands, the buffer half here is moot.

@coderabbitai

coderabbitai Bot commented Sep 6, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

`emit_runtime_handle_diag` was declared `pub(super)` in
`gc::roots::runtime_handles` but re-exported from `gc::roots` and called
from `gc` (E0364/E0603 under the default feature set). Same visibility as
its scan-state siblings.
@proggeramlug
proggeramlug force-pushed the perf/gc-pointer-validation branch from 273bad9 to 1536168 Compare September 6, 2026 20:47
proggeramlug pushed a commit to proggeramlug/perry that referenced this pull request Sep 6, 2026
The entry was created under PerryTS#9919, which is the GC pointer-validation PR.
This change is PR PerryTS#9921.
Record the diagnostic and test-only thread locals introduced by the
pointer-validation work, and refresh PASS1_MARKED's audited gc/mod.rs pin.

@jdalton jdalton left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review of 7905196226d8384662e6baede23aaa6d5cffe15e (2026-09-07).

Please add an exact-set lifetime regression around the new shared object-start bitmap. ValidPointerSet::arena_start_censused now returns the live page bitmap's answer; ArenaObjectCursorBuilder clears that same bitmap while building a new census. Exercise an existing finalized set while a second census is partially built, and allocation/reuse after finalization, or document/assert that these interleavings cannot occur at product call sites. The invariant to pin is equality with the finalized set's sorted census runs, in both directions (the debug assertion currently only checks a bitmap rejection). This is a validation request for the new shared-state contract, not a reproduced failure.

Validation scope: source/diff inspection; I have not run this PR's build or test suite locally.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants