Skip to content

LATX, fix: Preserve queued RCU callbacks across fork - #474

Open
LaurenIsACoder wants to merge 2 commits into
lat-opensource:masterfrom
LaurenIsACoder:latx-rcu-fork-queue
Open

LATX, fix: Preserve queued RCU callbacks across fork#474
LaurenIsACoder wants to merge 2 commits into
lat-opensource:masterfrom
LaurenIsACoder:latx-rcu-fork-queue

Conversation

@LaurenIsACoder

@LaurenIsACoder LaurenIsACoder commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Scope and invariant

Based on upstream bfe17ae7c1. Pending RCU callbacks must stay represented by both complete queue links and the pending count when fork snapshots the process.

Previously the consumer subtracted a batch before waiting for its grace period. A fork during that wait inherited queued nodes without the matching count; the child could leave them unprocessed or block a later drain. Concurrent producers could also be captured between queue-link updates.

Serialize enqueue/dequeue/count transitions with fork, decrement only when a node is actually dequeued, and preserve the child's queued work. The queue mutex is never held over a callback, grace-period wait or worker creation. Keep the existing lazy/deferred worker policy so namespace setup does not gain an unexpected helper thread.

Boundary: callbacks already dequeued into a vanished thread's stack are not replayed. Inherited queued callbacks start at the existing first-call/drain/explicit-deferred-start safe points; this is not a promise of immediate cleanup in an otherwise idle child.

Author self-review and combined validation

The actual published diffs were reviewed individually and checked against the local candidates. Follow-up corrections were folded into their existing commits; author/sign-off metadata updates preserved the per-PR source trees. The series remains six commits across #472, #473, #474, #475 and #476.

All five branches merge without conflicts on bfe17ae7c1. The combined source tree 80aaaaf13688387640f528d1c0bccb6a800f8d6e builds and passes 33/33 lat-pr-fast tests. On that combined binary, the 500-cycle KZT unload and retained-handle checks, 520-cycle SMC/fork checks, and 512 successful TSYNC operations alongside thread creation also complete successfully. A tests-disabled configuration registers no tests.

The integration suite still reports 27 skips, including after selecting the available guest sysroot with LATX_X86_64_SYSROOT: 25 require clang and two require the x86 cross compiler. These are not passes. This author self-review does not substitute for the upstream CI matrix, maintainer review or real-application acceptance.

Validation

LoongArch64, GCC 15.3, x86_64-linux-user, LAT O1, KZT compiled in, release:

./configure --target-list=x86_64-linux-user --enable-latx --optimize-O1 \
  --extra-ldflags=-ldl --enable-kzt --enable-tests --disable-docs \
  --disable-werror --meson=meson
ninja -C build -j2 all meson-test-prereq
env -u LAT_LD_PREFIX meson test -C build --no-rebuild \
  --suite lat-pr-fast --print-errorlogs --num-processes 2
env -u LAT_LD_PREFIX meson test -C build --no-rebuild \
  --suite latx-integration --print-errorlogs --num-processes 2
  • Isolated branch: 25/25 lat-pr-fast passed.
  • Maintained native regression uses the real public RCU API, pthreads and fork: a reader-held grace period, lazy/deferred child startup, empty queues, callbacks that queue callbacks, and four concurrent producers across repeated forks. It verifies exact-once callback delivery within its covered queue boundary and uses bounded timeouts.
  • The reader-held fork regression fails against the old queue accounting; both the new regression and the pre-existing lazy-worker test pass with this change.
  • The held-reader scenario is registered for LAT/KZT, whose existing atfork path does not wait for the parent grace-period lock. It is not evidence for every build configuration.
  • Integration suite: 27 skipped (exit 77) because the clang/guest-building toolchain is unavailable; skipped tests are not passes.

This is an independent infrastructure fix. The separate KZT unloaded-header reclamation PR must be merged after this one to preserve its queued retirements across fork. No runtime installation or application acceptance is included.

The RCU consumer subtracts a batch from the pending count before waiting
for its grace period. A fork during that wait inherits queued callbacks
without their count, so the child can leave work pending or hang on drain.
Concurrent enqueue can also expose incomplete links to the fork snapshot.

Serialize enqueue, dequeue and count updates with the atfork queue lock.
Decrement the count only when a callback is dequeued, and never hold the
queue lock across callbacks, grace-period waits or worker creation. Retain
the existing lazy and deferred child-worker startup policy. Callbacks
already dispatched to a vanished thread are outside this queue contract.

Add a native RCU fork regression covering held readers, child startup,
reentrant callbacks and concurrent producers. The held-reader drain times
out with the old accounting and passes with the fix. The original KZT
branch passed 25 lat-pr-fast tests; unavailable guest integration tests
were skipped.

Signed-off-by: Hanlu Li <heuleehanlu@gmail.com>
The RCU queue fix affects all configurations, but its regression is only
registered for LATX/KZT. Comparing completed totals across concurrent
publication can also hide a lost callback when later callbacks compensate
for it.

Register the common fork regression on POSIX hosts and keep the held-reader
scenario as a separate LATX/KZT invocation. Pause the common-case consumer
inside an already-dispatched callback, outside the grace-period lock, so
ordinary atfork locking can proceed while producer callbacks stay queued.
Mark each callback published after call_rcu1() returns and require every
published callback in the child snapshot to run exactly once after drain.
Keep the executable out of the default build and document explicit runs.

On LoongArch, the common KZT, non-KZT and non-LATX cases and the KZT
held-reader case each passed 20 repetitions. Full lat-pr-fast suites passed
26/26 with KZT and 25/25 without it. Reversing the production fix reproduced
the held-reader drain timeout. Tests-disabled registration and final
product builds were also checked.

Signed-off-by: Hanlu Li <heuleehanlu@gmail.com>
LaurenIsACoder added a commit to LaurenIsACoder/lat that referenced this pull request Sep 8, 2026
Repeated public-loader dlopen/dlclose cycles leave stale ELF metadata in
the context even after its guest mappings disappear. Freeing it immediately
would invalidate metadata still borrowed by native frames or RCU readers.

At a guest-execution safe point, scan under the mapping lock using a fresh
loader snapshot and retire unowned metadata through RCU. Retain the main
executable, attached libraries, malloc-backed borrowers, live link-map
entries and ELFs with surviving load segments. Reuse vacant context slots
without shifting other library indexes. Busy or stale snapshots and an
exclusive-barrier timeout conservatively defer reclamation.

Add lifecycle regressions for retained owners, snapshots, slot reuse and
deferred destruction. The branch passed 26 lat-pr-fast tests and sanitizer
checks. A 500-cycle libbz2 loader workload kept elfsize at 7; retaining a
handle kept it at 8. This does not bound translation caches or establish
full application memory stability; unavailable integration tests were
skipped.

Requires the queued-RCU fork fix in PR lat-opensource#474 before merging, so a child
preserves inherited metadata retirements.

Signed-off-by: Hanlu Li <heuleehanlu@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant