Repository navigation
Sandbox: bubblewrap starts where a protected path is missing - #319
Merged
Merged
Conversation
execute_python never ran inside bubblewrap: every call failed with "bwrap:
Can't create file at ~/.bash_profile: Read-only file system" (or another
protected path the host lacks; /System and /Library, macOS paths in the same
list, are on no Linux host). The backend bound /dev/null over each missing
deny-write path whose parent existed, and bwrap makes a missing mount point
itself, which below --ro-bind / / fails with EROFS before the code runs. The
logic is the same in bwrap 0.9.0 (Ubuntu 24.04) and 0.11.1 (26.04); CI saw it
only now because GitHub's ubuntu-26.04 image is the first with bwrap on PATH,
so TestLiveSandbox had always skipped.
A missing protected path the read-only root covers now gets no mount: the
code cannot create it there. One the code could create (in a folder it may
write: a project's .git before git init, its .{app_name} before the app makes
it) would have been made by bwrap on the host as an empty file left behind,
and an empty .git file breaks git. The wrap now holds a placeholder there
instead: an empty folder, made with a hold file inside and renamed into
place, bound read-only like any existing protected path. Each run holds it
with its own file, in this process or another, and OSSandboxResult.release
drops the hold and removes the folder when no hold is left (rmdir: whatever
was put in it meanwhile stays). The executor releases after every call. A
placeholder that cannot be made fails the wrap, since the code could still
create the path; a missing folder above a protected path takes the
placeholder, so neither can be made.
The live test of the project's .env now checks that the key never reaches
the code: bubblewrap shows /dev/null in its place, which reads empty, where
seatbelt refuses the open.
The code now runs as the sandbox's pid 1 (--as-pid-1). bwrap's reaper used to report the command's exit as soon as it came, while a process the code forked could still run: the executor then released the placeholders, an rmdir that detaches the sandbox's mount of .git, and the survivor could put a repository there. As pid 1 the code's exit completes only once the kernel has killed and reaped every process in the namespace, and bwrap waits for it. A run killed at its timeout may still leave them alive a moment, so the executor releases only after a run that ended on its own (or code the checks refused); a timed-out run keeps its holds, and the placeholder stays, empty and protected, for later runs to hold. The hold protocol: - An empty folder at a creation point below a writable bind is joined like a placeholder: between a release's unlink of the last hold and its rmdir the folder is empty, and a run wrapping then bound it as a real folder that the rmdir removed from under it. O_EXCL and rmdir's emptiness check now order the two. - The hold step is the plan: it returns the points it holds, and the argv builder binds the protected paths that exist as themselves plus those, no missing path and no placeholder another run holds. - A placeholder that cannot be made fails the wrap only where the code could create the path; a read-only file system, or a folder the user may not write and does not own, needs none. - A wrap that fails after holding releases; the release is safe from two threads; holds and staging folders are made, and dropped, in folders opened without following a symlink below the writable bind (a folder swapped for one fails the wrap). - OSSandboxResult.release is left out of equality.
The changelog bullet and CLAUDE.md's os_sandbox line say the code is the sandbox's pid 1 and that a timed-out run keeps its placeholder. The README and OSSandboxPolicy.writable_paths say that a writable path holding the home directory gets placeholder folders at the start-up files it lacks, and that a folder at ~/.bash_profile makes a login bash skip ~/.profile. The package docstring's example releases only after the command ended on its own.
Round one joined any empty folder at a creation point as a placeholder and
removed it at release. That deleted a project's empty .git (CI's
test_git_dir_cannot_be_moved_aside), an empty .{app_name} or memory folder
and, with a writable home, an empty ~/.ssh or ~/.aws; and an empty folder
above a protected path (an empty working directory under a writable home,
an empty ~/.config) became the placeholder, bound read-only and removed.
- Only the protected path itself is joined while empty: that closes the
mid-release race. A folder above it is a creation point only when it
holds a hold, our own placeholder.
- A hold records whether a run made the folder. Joining a folder that holds
no hold, or a kept one, writes a kept hold, and its release only unlinks
it: a run removes no folder it did not make, whoever joins after it. The
cost is an empty folder after the mid-release race.
- A protected folder that is there but takes no hold (one the user may not
write) is bound as it is, rather than judged by its parent's owner and
left unbound.
- ENOENT on the way to a creation point, or for the staging folder, means
the place changed: looked at again, not refused.
- The executor keeps the holds when a status says a signal ended the run
(negative, or 128 and above from a shell): a signal from outside that
kills the shell or bwrap's monitor returns before the kernel has emptied
the sandbox.
The timeout's live test let its child write at 1 s, before the 2 s timeout;
the child now waits 5 s, the file is checked at the timeout and after the
5 s, and the test runs both as the executor runs the command and behind a
shell that stays, so the kill reaches the shell alone.
The changelog bullet and CLAUDE.md's os_sandbox line say an empty protected folder that was there is left as it was, and that a run a signal ended is not released. The package docstring's example checks result.success before it runs the command (a failed wrap's command is the bare one, unsandboxed) and releases only after a status no signal set. The README sandbox paragraph is wrapped again.
CI showed a timed-out run's code still running 4 s later, on Linux as on macOS. subprocess.run kills only the process it started: the shell, or bwrap's monitor. bwrap sets PR_SET_PDEATHSIG in the sandbox's pid 1 just before execve, but its init outlived the monitor on the runner, and it is in a session of its own (--new-session); on macOS a process the code forks outlives sandbox-exec. The executor now runs its command through _run_command: Popen in a session of its own, communicate with the input and the timeout, and, on a timeout or any interruption while it waits, SIGKILL first for each child of the command's process group that left the group (read from /proc on Linux: bwrap's sandbox init, whose death ends every process in its namespace), then for the whole group. The result, the timeout's dict and the environment are as before; the tests that recorded subprocess.run record _run_command. On macOS, code that calls setsid() itself escapes the group. The timeout's live test runs through _run_command on both backends and, when a forked writer survives, shows the shells, sandboxes and Pythons still running. A hold that cannot be written into a folder another run holds now fails the wrap unless the cause is one a run that made the folder would meet too (EACCES, EPERM, EROFS): bound without a hold on a full disk, the folder could be removed by that run while this one runs. A hold whose write fails is removed. The staging folder's ENOENT branch has a test.
The changelog gets a Fixed bullet for the timeout, and its bubblewrap bullet says that after a timeout or a signal the run's hold file stays in a folder that was there. The README's sandbox paragraph says a placeholder is kept after a signal too, holding the run's hidden hold file, and that a timeout stops every process the code started. CLAUDE.md's executor line names _run_command.
…walk A run in a session of its own no longer got the hangup a closing terminal sends the app's process group, nor a SIGTERM sent to the app: the app died and the code ran on. The executor keeps the live runs, kills them all as a timeout does (_kill_live_runs), at exit through atexit, and BaseCLIApp.run() installs SIGHUP and SIGTERM handlers (_stop_runs_on_signals, once, from the main thread) that kill them, then call the handler they replaced, or restore the default and send the signal again, so the app still dies of it. A library user who never runs the CLI gets the atexit cleanup alone. - Windows has no os.killpg nor signal.SIGKILL: a timeout raised AttributeError and then waited forever. There a run gets no session and a timeout kills the process alone, as subprocess.run did; everywhere the process the run started is killed in a finally, whatever the walk did. - The group is stopped (SIGSTOP) before the walk for children that left it: nothing in it can fork, leave or reap meanwhile, so a child found keeps its pid until it is killed. A walk that fails is logged; the group is killed all the same. - A run that ends on its own kills its own group: what the code left in it ends with it, as bubblewrap's namespace ends with the code. - The comment on why bubblewrap's init is killed by its host pid names the cause: an exec into another AppArmor label clears the parent-death signal (apparmor_bprm_committing_creds; Ubuntu's bwrap-userns-restrict execs the code with pix). - A kept placeholder holds the run's hold file: the docstrings no longer call it empty. The os_sandbox example says how to run the command, in words, without subprocess.run's timeout, which leaves the sandbox running. - The live tests run their command through _run_command, so the fork test runs on seatbelt too.
… namespace The changelog's timeout bullet and the README's sandbox paragraph say that a timeout, a KeyboardInterrupt in a main-thread caller, or the app's end stops the code and the processes it started, and that code that calls setsid() and forks again can escape outside bubblewrap's pid namespace (macOS, and Linux without the sandbox). The bubblewrap bullet no longer calls a kept placeholder empty. CLAUDE.md's executor and run() lines name the live runs and the handlers.
…killed - No handler is installed for a SIGHUP or SIGTERM the app ignores (under nohup): the hangup killed the running call while the app went on. The app and its runs now go on, each run to its timeout. - The handler runs the chain, or the default's re-raise, whatever killing the runs does (try/finally). - _kill_session kills the group in a finally: interrupted between its SIGSTOP and its SIGKILL (a KeyboardInterrupt, a second signal), a group left stopped, orphaned in a session of its own, would never be continued. - A run is listed inside the try that unlists it, and _kill_live_runs skips a run already reaped. - The setsid() wording says what escapes: outside a pid namespace, code that calls setsid() can outlive the run (on macOS a fork that calls it once; on Linux the walk only catches a direct child of a live group member). The executor's docstring says a normal exit waits for a running call, as before, and atexit reaches only runs on daemon threads.
The changelog's timeout bullet says a hangup or SIGTERM kills a live run, an ignored one changes nothing, and a normal exit waits for the run, as before, after which nothing it started is left. The README, the changelog and CLAUDE.md say that outside a pid namespace code that calls setsid() can outlive the run (on Linux the walk only catches a direct child of a live group member).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On Linux with bubblewrap installed, the OS sandbox never started: every
execute_pythoncall was refused ("it did not start (bubblewrap, exit status 1): bwrap: Can't create file at ~/.bashrc: Read-only file system"), and withos_sandbox_strictoff the code ran unsandboxed. CI's Linux runner now ships bwrap (ubuntu-26.04), soTestLiveSandboxran there for the first time; on ubuntu-24.04 it was skipped. Running it also showed that a timed-outexecute_pythonkept running, on Linux and on macOS.Cause: for a protected path that does not exist (
~/.bashrc,~/.zshrc, and on any Linux root/Systemand/Library), the wrap passed--ro-bind /dev/null <path>. bwrap must create that mount point below the root it has already mounted read-only, and dies with EROFS. Where the parent was writable (a project's.gitwhen it is not a repository,.{app_name}), bwrap would have left an empty 0444 file behind, which breaks git and the app's own folder.The sandbox (
tools/os_sandbox/bubblewrap.py,base.py):--as-pid-1), so a run that ends leaves no process behind.OSSandboxResult.release()(new, a no-op by default) drops the run's holds only after a run that ended on its own (status 0-127). After a timeout or a signal, the hold stays and the placeholder stays protected.The executor (
tools/executor.py), on every platform:/proc), then SIGKILLs the group, always in afinally. Ubuntu's AppArmor profile for bwrap clears pid 1's parent-death signal at exec, so killing bwrap alone left the code running.BaseCLIApp.run()installs SIGHUP and SIGTERM handlers, unless the signal is ignored. Each kills the live runs, then chains to the previous handler, so the app still ends as before. A normal exit waits for a run, as before. Outside a pid namespace, code that callssetsid()can outlive a run.Tests: the new offline tests on the argv, the placeholders, the holds, the executor's kill paths and the signal handlers were each seen failing first. The live tests (12 on ubuntu-26.04, all passing) cover a missing shell start-up file, a project that is no repository, the code as pid 1, forks ending with the run, and a timeout stopping the code and what it forked.