Repository navigation
Apply cross-process work inside a syscall callback - #11
Closed
utkarshdalal wants to merge 1 commit into
Closed
utkarshdalal wants to merge 1 commit into
utkarshdalal wants to merge 1 commit into
Conversation
Author
|
Closing: the Wine-side patch (GameNative/proton-wine#133) is sufficient for this. It guards the cross-process work drain in ntdll, so the re-entry cannot happen with any FEX build. |
utkarshdalal
added a commit
to GameNative/proton-wine
that referenced
this pull request
Nov 15, 2026
…133) Builds on #131 and includes its commits. Merge #131 first; this PR then adds one commit. ## What changes Two patch files in `android/patches/arm64ec/`, both registered in `build-scripts/build-step-arm64ec.sh` after #131's patch: - `dlls_ntdll_unix_virtual_c_report_emulated_code_write.patch`: when #131 makes a write-protected RWX page writable for a system call, ntdll also queues a `CrossProcessMemoryWrite` entry for that page on the emulator's cross-process work list. The emulator applies it the next time it drains the list and drops its translations of the page, so code that a system call overwrote is translated again. If no entry is free, the work list is flagged for a full flush, so the write is never dropped. - `dlls_ntdll_signal_arm64ec_c_cross_process_work_callback.patch`: `ProcessPendingCrossProcessEmulatorWork` now applies the work inside a syscall callback (`enter_syscall_callback` / `leave_syscall_callback`, cleared only when this call set it), as every other emulator notification already does. ## Why the second patch is needed Without it, the first entry hangs the thread. Stack captured on the device (FEX 2512, one thread, newest frame first): ``` SleepConditionVariableSRW std::__shared_mutex_base::lock CodeInvalidationMutex, second time InvalidationTracker::InvalidateIntervalInternal InvalidationTracker::InvalidateAlignedInterval recursive_mutex::lock NotifyMemoryFree VirtualFree allocator, inside the invalidation ContextImpl::InvalidateThreadCachedCodeRange InvalidationTracker::InvalidateIntervalInternal CodeInvalidationMutex held BTCpu64NotifyMemoryDirty ProcessPendingCrossProcessEmulatorWork ECSyscallHandler::HandleSyscall ``` FEX holds its code invalidation lock while its allocator frees memory. ntdll only skips memory notifications inside a syscall callback, so the nested `NtFreeVirtualMemory` was reported back to FEX, which took the same non-recursive lock again on the same thread. The lock is not taken from the system call or the unix fault handler: the entry is queued there without locks and applied later on the PE side. Upstream Wine has the same drain without the guard. Nothing in Wine queues `CrossProcessMemoryWrite` today, so there it only shows up with work queued by another process. ## Results AYN Odin 3, stock Proton 11.0-2 tree with this PR's `ntdll.so` / `ntdll.dll` from CI run 43691683416. The test is an x86-64 program that runs code in an RWX page, then lets system calls write into the page, both through the ntdll export and through a raw `syscall` instruction. | Case | Stock 11.0-2 | #131 only | #131 + this PR | |---|---|---|---| | `NtQuerySystemInformation` into a page whose code already ran | `c0000005` | 0 | 0 | | `NtReadVirtualMemory` overwrites code that already ran, then call it (1,000×, ntdll export) | fails, 1,000 stale | succeeds, **1,000 stale** | 0 stale | | Same through a raw `syscall` | fails, 1,000 stale | succeeds, **1,000 stale** | 0 stale | | 10,000 syscall writes into the page while 2 threads execute from it | 8,550 errors | 0 errors | 0 errors, no hang | | Self-modifying code, 10,000 same-thread + 10,000 cross-thread | 0 wrong | 0 wrong | 0 wrong | | 20,000 × (data write + call) on one RWX page | 1654 ms | 1653 ms | 1587 ms (FEX with the GameNative/FEX#11 fix) / 1493 ms (gn-6) | The work-list guard was checked against the build without it, the first version of #131, which queues the entry but has no guard. With FEX 2611-gn-6 it hung in the concurrent case; with GameNative/FEX#11, or with this PR's guard, it does not. Either side's change stops the hang on its own. Hogwarts Legacy, real Steam client headless, overlay off, gamescope, with this build and the FEX#11 build: the game reaches its first screen, the WB EULA with Accept, at 31 fps on the HUD. Known, not changed here: - `NtQueryVirtualMemory` reports what FEX's guest-visible protection says. With the GameNative FEX series that is 0x40 (patch 0002); a stock FEX reports 0x20 for a page it trapped. - A `WriteProcessMemory` from another process into code the target already ran still runs the old code in the target. The bytes change, but the target never drops its translation, even after a later system call. The same happens on stock 11.0-2; it is a separate path (`CrossProcessFlushCache` from kernelbase). --------- Co-authored-by: Utkarsh Dalal <utkarsh.dalal@toptal.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds patch 0013 to the series. Pairs with GameNative/proton-wine#133. Either change on its own stops the hang, and each fixes a real bug on its side.
Problem
FEX drains the cross-process work list from
HandleSyscall,PreCompileandSyncThreadContext. A memory-write entry ends inInvalidateIntervalInternal, which holdsCodeInvalidationMutexwhile the allocator may free memory. ntdll only skips its memory notifications for calls made inside a syscall callback. The drain does not run inside one, so the nestedNtFreeVirtualMemorycalled back intoNotifyMemoryFreeon the same thread. That lockedCodeInvalidationMutexagain and hung.Stack captured on the device (one thread, newest frame first):
The lock is never taken from a system call or from the unix fault handler. ntdll only queues the entry there, without locks; FEX applies it later on the PE side.
Change
The three drain sites go through one helper. It holds the existing
ScopedCallbackDisableacrossProcessPendingCrossProcessEmulatorWork, so the thread counts as inside a syscall callback, the same as inResetToConsistentState. The class moves above the exception namespace so the helper can use it. Upstreammainhas the same unguarded drain; the patch applies there unchanged.Not changed: on a page FEX has trapped,
NtQueryVirtualMemoryreports what patch 0002 tracks for the guest. Using Wine's write-exception interface (ProcessManageWritesToExecutableMemory/VmPageDirtyStateInformation) instead of the protection trap was tried and dropped:STATUS_IN_PAGE_ERRORand the process hung.KernelWriteToExecutableSignalis declared and never read, so a system call writing into an armed page fails.Results
AYN Odin 3, CI run 43691753463 (
2611.1-gn6-2117faf). The test is an x86-64 program: code runs in an RWX page, then system calls write into the page through the ntdll export and through a rawsyscall, including 10,000 writes while two other threads execute from the page.c0000005With either fix in place, code a system call overwrote runs as the new code on the next call, 1,000 of 1,000 on both paths. Self-modifying code: 10,000 same-thread and 10,000 cross-thread iterations, 0 wrong in every run.
Games:
Note: another series branch (gn-7, stale NoExec retry) is in progress. The two touch different patches; whichever merges second renumbers.