gh-146181: Spawn asyncio Unix subprocesses in a worker thread

[sreehariannam]

Fixes gh-146181.

Problem

asyncio.create_subprocess_exec()/create_subprocess_shell() call subprocess.Popen() synchronously on the event loop thread. Popen() doesn't return until the child has called execve(), so a slow-to-start child (large statically-linked binary, contended disk during page-in, etc.) can block the entire event loop for multiple seconds, as reported in production.

Fix (Unix only — see note below)

Following the direction @gvanrossum and @kumaraditya303 converged on in the issue: only the subprocess.Popen() call itself moves to a worker thread, via loop.run_in_executor(). Everything else (task creation, watcher registration) stays on the loop thread, since loop.create_task() and friends aren't thread-safe.

Concretely:

• Lib/asyncio/unix_events.py: extracted the Popen() + AIX/cygwin stdin-socketpair logic out of _UnixSubprocessTransport._start() into a module-level _spawn_subprocess() function, so it can run inside run_in_executor(). _make_subprocess_transport() now awaits that in the executor, then constructs the transport with the already-created process.
• Lib/asyncio/base_subprocess.py: BaseSubprocessTransport.__init__ gained an optional _proc=None parameter — if a process is already provided, it skips calling self._start(). Purely additive; unset by default, so this is a no-op for any existing caller (including Windows, whose _make_subprocess_transport doesn't pass it).

Cancellation (the part flagged as tricky)

Worker threads can't be interrupted, so if the task awaiting subprocess creation is cancelled while Popen() is still running in the executor, the call keeps running in the background regardless. To avoid leaking an orphaned, untracked child process (or its pipe fds) in that case:

try:
    proc = await tasks.shield(spawn_fut)
except exceptions.CancelledError:
    proc = await spawn_fut  # wait for the in-flight Popen() to actually finish
    proc.kill()
    proc.wait()
    for pipe in (proc.stdin, proc.stdout, proc.stderr):
        if pipe is not None:
            pipe.close()
    raise

I found this the hard way: my first version only did proc.kill()/proc.wait() and leaked the three pipe fds Popen() opens for stdin=PIPE, stdout=PIPE, stderr=PIPE, since no transport is ever constructed in this path to own that cleanup — caught it via a ResourceWarning that showed up on the existing test_cancel_post_init test once I ran it a few times.

@kumaraditya303 — does this cleanup behavior match what you had in mind, or would you rather cancellation left the process running for some caller-visible cleanup semantics? Wanted to raise this concretely against real code rather than in the abstract, since the general direction (thread dispatch for Popen()) already had your and Guido's go-ahead in the issue.

Windows

windows_events.py has the identical structural issue (_WindowsSubprocessTransport._start() calling Popen() synchronously from an async _make_subprocess_transport()), but I can only build/test on Linux (WSL) and have no way to exercise the IOCP/proactor code path. Deliberately leaving Windows out of this PR rather than changing code I can't verify — happy to do it as a follow-up if someone can test on Windows, or if this Unix-side approach gets a thumbs up first.

Testing

• Existing test_asyncio.test_subprocess suite (85 tests after this change) passes, including both SubprocessThreadedWatcherTests and SubprocessPidfdWatcherTests variants.
• Full test_asyncio suite (2784 tests) passes.
• Added two new tests:
  • test_subprocess_exec_does_not_block_loop: patches _spawn_subprocess to block on a threading.Event, and asserts a concurrent coroutine keeps ticking while the "spawn" is stuck.
  • test_cancel_make_subprocess_transport_kills_process: cancels subprocess creation mid-spawn and asserts the resulting process was killed/reaped and its pipe fds closed (not just that CancelledError propagates, which the existing test_cancel_post_init already covered).
• Verified both new tests actually exercise the fix by reverting unix_events.py/base_subprocess.py and confirming they fail without it.