diff --git a/CHANGELOG.md b/CHANGELOG.md index 102f3fe1..e386a8ab 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2789,6 +2789,50 @@ step; each entry says what to use instead. ### Fixed +- **A timed-out `execute_python` stops the code and what it started.** The + timeout killed only the process the executor started (the shell, or + bubblewrap's monitor), so the code ran on: on Linux the sandbox's init, + in a session of its own, outlived bubblewrap, and on macOS a process the + code forked outlived `sandbox-exec`. The command now runs in a session of + its own, and a timeout, a KeyboardInterrupt while it waits (in a caller + on the main thread), or, in the CLI, a hangup or SIGTERM kills its whole + process group and, on Linux, each child of it that left the group + (bubblewrap's sandbox init, whose death ends every process in the + sandbox); a signal the app ignores (under `nohup`) still changes nothing. + A normal exit waits for a running call, as before (to its timeout), and + then nothing it started is left; a run that ends on its own takes with it + what it left in its group. Outside a pid namespace (on macOS, and on Linux + without the sandbox), code that calls `setsid()` can outlive them (on + Linux the walk only catches a direct child of a live group member). + +- **`execute_python` runs inside bubblewrap on Linux.** It never did: every + call was refused with "it did not start (bubblewrap, exit status 1): bwrap: + Can't create file at ~/.bash_profile: Read-only file system" (or another + protected file the home lacks: most homes lack some of `~/.bashrc`, + `~/.bash_profile`, `~/.zshrc`, `~/.profile`, …, and `/System`, a macOS + path in the same list, is on no Linux host), or, with `os_sandbox_strict` + off, run without the sandbox. The bubblewrap backend bound `/dev/null` + over each protected path that did not exist, and bwrap must make that + mount point itself, which it cannot do below the read-only root; seen + first on GitHub's ubuntu-26.04 runner, the first image with `bwrap` on + it, and the same with every bwrap version. A missing protected path the + read-only root covers is now left to it (the code still cannot create + it). One the code could create (a project's `.git` before `git init`, its + `.{app_name}` before the app makes it) is held by a placeholder, an empty + folder bound read-only, which the wrap makes and removes once the last run + holding it is over (`OSSandboxResult.release`, which the executor calls): + bwrap would have left an empty file there, and an empty `.git` file breaks + git ("invalid gitfile format"). A run never removes a folder it did not + make: a protected folder that is there and empty (an empty `.git`) is bound + read-only, and left as it was after a run that ended on its own (after a + timeout, or a run a signal ended, the run's hidden hold file stays in it). + The code runs as the sandbox's pid 1 + (`--as-pid-1`), so a run that returned has left no process behind; one that + timed out, or that a signal ended, is not released, and its placeholder + stays, holding the run's hidden hold file and protected, for later runs to + hold. A placeholder that + cannot be made where the code could still create the path fails the wrap. + - **A model call a tool makes reports its usage.** `generate_simple` (web_fetch's and the knowledge base's summaries, the memory proposals) reported no `LLM_USAGE` on ADK or Deep Agents, so the status bar's totals diff --git a/CLAUDE.md b/CLAUDE.md index feee29d8..de40e228 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -30,7 +30,7 @@ agentic-cli/ │ ├── protected_paths.py # The locations no tool reaches (credential stores, never-writable files and dirs, the project's .git/.env files/.{app_name}, ~/.{app_name} with its .env and settings.json): the engine's builtin denies and the OS sandbox's policy are both built from it (stdlib only) │ ├── logging.py # configure_logging (third-party loggers quieted there, google_genai.types too: no import sets a level); LOG_STREAM, which every logger writes to (structlog through _LineLogger, no lock); log_lines_to(sink): each line to a sink instead of stderr │ ├── cli/ -│ │ ├── app.py # BaseCLIApp (main(): the command line, add_arguments/from_args, exit_status; session_start: how run() begins, the run's own conversation loaded before the prompt, also when the workflow did not start (_adopt_its_own: an id the app was given held from the start, one the run made from its first write); its resume once the prompt is up (_open_on_startup, holding the turn lock from before the prompt takes a line: a turn typed ahead waits and runs there; Ctrl+C there ends the app; --continue passes over conversations another process holds, _newest_free, the one chosen held beside the run's own until the switch); --continue and --resume find the conversation's backend from every installed backend's reader before the workflow is built (_start_on_its_backend: the run builds that backend once); the --resume hint at exit (the run's store named after a session_store change; the backend only when no reader reads it, LangGraph: --resume opens a conversation on its own); the session it builds; get_theme(name); /settings: _build_ui_items, apply_settings and save_settings delegate to cli/settings_ui.py (SettingsUI: an app's override applies); run() installs the ADK log filters first (workflow/adk/log_filters.py: an app built but never run installs none); the LangGraph deprecation notice; start_new_session(backend=None): /new [backend] (the settings' orchestrator by default; another backend: the memories proposed, WorkflowController.switch_backend, then the switch to the new id at its place; a Ctrl+C once it runs opens its conversation first); a line typed runs in ConversationCore.run_turn (on _turn_parts: the app's parts as they are), then the jobs at the turn's end through the app's own resume_finished_jobs(); compact() (ConversationCore.compact), start_new_session() and resume_session(): /compact, /new and /resume, under the conversation's turn lock (_turn_lock: ConversationCore.turn_lock; pick_session(), the picker, is not: it waits on a dialog); session_store=memory keeps one conversation at a time (Place.kept; session_store: the run's store): /new drops the one left once the switch is made, after its memories are proposed (_drop: its delete_session, shielded; on another backend the manager released takes it along; one it cannot confirm gone, the workflow down or its lookup or delete raising, is remembered and dropped once a workflow is ready: before the next turn, or right after the turn whose own start revives the workflow, and always before that turn's delivery of finished jobs, /compact and the next /new, never while it is the open one: ConversationCore.drop_left), and resume_session, pick_session, --resume (status 1) and --continue (a new conversation) are refused with NOT_KEPT, before any look-up; every installed backend's conversations (list_conversations, delete_conversation, session_backend; cli/stored_conversations.py): the picker names each row's backend and returns a PickedConversation, and /resume opens a conversation on the backend that keeps it (_open_elsewhere: held there, memories proposed, switch_backend with the run's services, then the switch at its place; an id two backends keep opens on the running one, the other named); every change of conversation through ConversationCore.switch (_conversation; _session_id and _turned forward to it); the session_id setter deprecated (its synchronous part, switch_now, and a DeprecationWarning); a conversation's memories proposed once per conversation used (_turned); the memory proposals waiting, said at startup and after /new and /resume (_name_waiting_proposals); resume_finished_jobs(): ConversationCore.deliver_finished_jobs, the jobs waiting named at startup by ConversationCore.waiting_jobs (_name_its_waiting_jobs)) +│ │ ├── app.py # BaseCLIApp (main(): the command line, add_arguments/from_args, exit_status; session_start: how run() begins, the run's own conversation loaded before the prompt, also when the workflow did not start (_adopt_its_own: an id the app was given held from the start, one the run made from its first write); its resume once the prompt is up (_open_on_startup, holding the turn lock from before the prompt takes a line: a turn typed ahead waits and runs there; Ctrl+C there ends the app; --continue passes over conversations another process holds, _newest_free, the one chosen held beside the run's own until the switch); --continue and --resume find the conversation's backend from every installed backend's reader before the workflow is built (_start_on_its_backend: the run builds that backend once); the --resume hint at exit (the run's store named after a session_store change; the backend only when no reader reads it, LangGraph: --resume opens a conversation on its own); the session it builds; get_theme(name); /settings: _build_ui_items, apply_settings and save_settings delegate to cli/settings_ui.py (SettingsUI: an app's override applies); run() installs the ADK log filters first (workflow/adk/log_filters.py: an app built but never run installs none), then the SIGHUP/SIGTERM handlers that kill live execute_python runs before chaining to the previous handler (_stop_runs_on_signals); the LangGraph deprecation notice; start_new_session(backend=None): /new [backend] (the settings' orchestrator by default; another backend: the memories proposed, WorkflowController.switch_backend, then the switch to the new id at its place; a Ctrl+C once it runs opens its conversation first); a line typed runs in ConversationCore.run_turn (on _turn_parts: the app's parts as they are), then the jobs at the turn's end through the app's own resume_finished_jobs(); compact() (ConversationCore.compact), start_new_session() and resume_session(): /compact, /new and /resume, under the conversation's turn lock (_turn_lock: ConversationCore.turn_lock; pick_session(), the picker, is not: it waits on a dialog); session_store=memory keeps one conversation at a time (Place.kept; session_store: the run's store): /new drops the one left once the switch is made, after its memories are proposed (_drop: its delete_session, shielded; on another backend the manager released takes it along; one it cannot confirm gone, the workflow down or its lookup or delete raising, is remembered and dropped once a workflow is ready: before the next turn, or right after the turn whose own start revives the workflow, and always before that turn's delivery of finished jobs, /compact and the next /new, never while it is the open one: ConversationCore.drop_left), and resume_session, pick_session, --resume (status 1) and --continue (a new conversation) are refused with NOT_KEPT, before any look-up; every installed backend's conversations (list_conversations, delete_conversation, session_backend; cli/stored_conversations.py): the picker names each row's backend and returns a PickedConversation, and /resume opens a conversation on the backend that keeps it (_open_elsewhere: held there, memories proposed, switch_backend with the run's services, then the switch at its place; an id two backends keep opens on the running one, the other named); every change of conversation through ConversationCore.switch (_conversation; _session_id and _turned forward to it); the session_id setter deprecated (its synchronous part, switch_now, and a DeprecationWarning); a conversation's memories proposed once per conversation used (_turned); the memory proposals waiting, said at startup and after /new and /resume (_name_waiting_proposals); resume_finished_jobs(): ConversationCore.deliver_finished_jobs, the jobs waiting named at startup by ConversationCore.waiting_jobs (_name_its_waiting_jobs)) │ │ ├── commands.py # Command, CommandRegistry │ │ ├── job_monitor.py # JobMonitor: the status bar's jobs segment (running, queued, a finished job's note, ↻N: waiting_jobs' rule, can_resume answers kept 30 s (the other backends' look too: kept_elsewhere), forgotten after /sessions --delete or when the workflow is replaced), one tick every 2 s, its JobManager.list() on a worker thread │ │ ├── conversation.py # ConversationCore: the conversation the turns run in (its id, turned, its lock at its Place: the backend and store, the store the run's for its whole life (Place.kept: not the memory store, where the CLI keeps one conversation at a time and NOT_KEPT says why nothing else opens), the backend recorded as the app starts, for a new conversation on another backend, and on a /settings change of the orchestrator while the workflow is down, its next start builds afresh and nothing of the conversation is stored (SettingsUI._move_here, WorkflowController.next_start_builds), never read from the settings in between; deferred: an id this run made (the start's, /new's, an orchestrator change's) is in no store, so it is held from its first write, hold_to_write under the turn lock before a turn, a job's notice or /compact (_held_to_write; a run in which no turn runs makes no lock file: a line typed while the workflow is down takes the hold before its retried start; a lock that cannot be taken, an OSError, leaves it unheld, said once, and the write goes on); fresh: nothing of it stored, opened_empty (an id this run made, or one a load found empty) and not written (no turn of it ran on a ready workflow), the one condition for another backend); switch: the one protocol (/new on any backend, /resume, startup, the same id in another place while the workflow is down): the target's lock taken (another process's refused), memories proposed, the workflow loads it (a stored one gone: not made, or, made already in another place, opened empty: gone), then made at once (lock, id, task list, usage counters) or let go, a workflow that took it given the old one back; made before the load when there is no other conversation to stay in (the same id, or another place: a new or stored conversation on another backend); a Ctrl+C at any await leaves one conversation held; switch_now: the deprecated session_id setter's part; turn_lock: what writes to the conversation and every switch take it; run_turn, compact and run_job_resume: a turn (echoed, held to write, turned, written once it reached a ready workflow; run_turn says whether it ran, whatever the processor returns), /compact and a finished job's notice under it, on TurnParts (the app's parts as they are); deliver_finished_jobs: each job that waits here (cli/waiting_jobs.py's rule, which waiting_jobs names them by, the other backends' look through TurnParts.kept_elsewhere), every JobManager call on a worker thread, a delivery claimed only once it can reach the agent (process_resume(claim=), after the readiness check and can_resume; _claim_delivery: a claim a Ctrl+C stopped is handed back, JobManager.release_resume), a failure before the claim said once per job and conversation when the CLI tells jobs on its own (telling_on_its_own); to_drop/drop_left: the conversations /new left on the memory store that the store may still hold (the workflow down, a lookup or delete that raised), dropped through TurnParts.drop under the turn lock once a workflow is ready, before a turn, /compact or a delivery uses it (after a turn or /compact that started it), one try per point, done once the store has it no more (a workflow without supports_sessions has none), the open one skipped; a delivery with none to drop takes no turn lock @@ -127,7 +127,7 @@ agentic-cli/ │ │ ├── registry.py # ToolRegistry, @register_tool (outside_content=: a call that brings outside content), ToolCategory │ │ ├── _saved_outputs.py # reads_saved_output: read_file/diff_compare/kb_ingest_file on a saved page or tool output, and grep on a path in or above (no hidden entry between) a saved folder, bring outside content (compared as the engine compares a deny: NFC, casefolded) │ │ ├── factories.py # Service-bound tool builders (per-manager flavors) -│ │ ├── executor.py # SafePythonExecutor: where each call runs is chosen when it runs (_choose_run): inside the OS sandbox (CORE_MODULES + SANDBOXED_MODULES), refused when the policy is strict (the default) and that fails, else as the user (CORE_MODULES only; the checks are no boundary); then validated for that module set +│ │ ├── executor.py # SafePythonExecutor: where each call runs is chosen when it runs (_choose_run): inside the OS sandbox (CORE_MODULES + SANDBOXED_MODULES), refused when the policy is strict (the default) and that fails, else as the user (CORE_MODULES only; the checks are no boundary); then validated for that module set; _run_command: in a session of its own (POSIX; Windows kills the process alone), a timeout, a KeyboardInterrupt or the app's end (_kill_live_runs: atexit, and the CLI's SIGHUP/SIGTERM handlers) stops the group, kills each child that left it (bubblewrap's sandbox init, whose parent-death signal AppArmor clears) then the group, and a normal end kills what stayed in the group (no handler over an ignored signal); outside a pid namespace setsid() can outlive them (on Linux the walk only catches a direct child of a live group member) │ │ ├── exec_policy.py # Settings → where code runs: python_sandbox_policy (execute_python's), uncontained_exec_capabilities (python.exec unless the sandbox is on and strict, python.exec.stateful on the local backend: the engine's uncontained=); the memory folders made and denied to the code (deny_read_paths, deny_write_paths, deny_write_missing_paths, deny_rename_paths) │ │ ├── execution_tools.py # execute_python │ │ ├── _knowledge_base.py # current_knowledge_base(): the knowledge_base key @@ -156,7 +156,7 @@ agentic-cli/ │ │ ├── jobs/ # Long-running jobs: JobManager, backends (a spec pinned at submit: a subprocess job runs where it was submitted; one on a backend that does not survive a restart, in-process, is claimed at submit and runs only in its submitter, FAILED if still queued at close; JobRecord.started_by: the sub-agent instance whose call submitted it), job_* tools │ │ │ └── resume.py # job_result_payload, JOB_NOTICE_HEADER, job_notice_ask (a sub-agent's job named: JOB_NOTICE_ASK_FOR): what a finished job tells the agent │ │ ├── sandbox/ # Stateful code-execution sandbox (sandbox_execute); stage_inputs and the docker backend's _collect_outputs copy through descriptors of inputs/ and outputs/ (the kernel can swap either for a symlink); an input whose path runs through the session's directory is refused, on docker any session's (sandbox_root_for(settings): /sandbox); calls on one session run one at a time (SandboxManager: a lock per session; a reset or cleanup during a call stops its cell, SandboxBackend.stop, and leaves the kernel's shutdown to that call) -│ │ ├── os_sandbox/ # The OS sandbox execute_python runs in: seatbelt (macOS), bubblewrap (Linux), else NoOpSandbox; policy.py builds its lists from protected_paths +│ │ ├── os_sandbox/ # The OS sandbox execute_python runs in: seatbelt (macOS), bubblewrap (Linux), else NoOpSandbox; policy.py builds its lists from protected_paths; bubblewrap runs the code as the sandbox's pid 1 (--as-pid-1: a run that returned left no process), binds nothing over a missing protected path below its read-only root (bwrap could make no mount point there) and a placeholder folder over one the code could create (an empty folder at the path itself joined and never removed: a run removes only a folder it made; made by the wrap, removed by OSSandboxResult.release, which the executor calls only after a run that ended on its own: a timed-out or signalled run keeps it) │ │ └── webfetch/ # Fetcher (fetch(max_pdf_bytes=): a call's own PDF limit, uncached; FetchResult.transient), converter (page_text, text_extension), validator (DNS in a worker thread: the loop's getaddrinfo), robots, summarizer, saved pages │ ├── memory/ # Knowledge base + memory store; imports nothing else from agentic_cli │ │ ├── __init__.py # MemoryStore, Memory, Proposal, Candidate, the store's errors, extraction_prompt/parse_candidates, looks_like_secret, EmbeddingService, EmbeddingConfig, MockEmbeddingService diff --git a/README.md b/README.md index 5401edbe..2609a3c2 100644 --- a/README.md +++ b/README.md @@ -1678,7 +1678,19 @@ The `/sandbox` CLI command lists and resets sandbox sessions. `execute_python` runs its code only inside the OS sandbox: it writes only to the working directory (and `os_sandbox_writable_paths`), reads no credential store, no `~/.{app_name}` and not the project's `.env`, and has no network -unless `os_sandbox_allow_network` is on. On a host with no sandbox backend +unless `os_sandbox_allow_network` is on. On Linux, bubblewrap keeps the code +from creating a protected path that does not exist in a folder it may write +(the project's `.git` before `git init`) with a placeholder: a folder there +while the code runs and removed once it is over, but kept after a run that +timed out or that a signal ended (holding only that run's hidden hold file, +and still protected). An `os_sandbox_writable_paths` entry that holds your +home directory puts such folders at the shell start-up files it lacks +(`~/.bash_profile`, `~/.zshrc`, …), and a folder at `~/.bash_profile` makes a +login bash skip `~/.profile`. A timeout stops the code and the processes it +started; outside a pid namespace (on macOS, and on Linux without the +sandbox), code that calls `setsid()` can outlive it (on Linux only a direct +child of a live group member is caught). +On a host with no sandbox backend (Windows has none), when the sandbox cannot be set up, or when the sandbox tool fails as it starts (on Ubuntu 24.04 and later, `bwrap` needs unprivileged user namespaces, which AppArmor restricts by default), the call is refused with an diff --git a/src/agentic_cli/cli/app.py b/src/agentic_cli/cli/app.py index d1bae4e4..705de671 100644 --- a/src/agentic_cli/cli/app.py +++ b/src/agentic_cli/cli/app.py @@ -16,6 +16,7 @@ import re import shlex import shutil +import signal import sys import threading import warnings @@ -161,6 +162,57 @@ def sink(line: str) -> None: _NOT_KEPT_LEFT = "not kept (session_store is memory)" +#: Whether ``_stop_runs_on_signals`` installed its handlers in this process. +_runs_stop_on_signals = False + + +def _stop_runs_on_signals() -> None: + """Have a hangup (the terminal closed) or a SIGTERM kill the code + ``execute_python`` runs before the app goes. + + Each run is in a session of its own (``tools/executor.py::_run_command``), + where the signal the terminal sends the app's process group does not + reach it. Each handler kills the live runs, then does what the signal did + before: it calls the handler it replaced, or, where that was the default, + restores the default and sends the signal again, so the app still dies + of it. A signal the app ignores (under ``nohup``) gets no handler: the + app goes on, and so do its runs, each to its timeout. Installed once per + process, from the main thread (where a handler can be set); SIGINT is + left alone. + """ + global _runs_stop_on_signals + if _runs_stop_on_signals or threading.current_thread() is not threading.main_thread(): + return + from agentic_cli.tools.executor import _kill_live_runs + + for name in ("SIGHUP", "SIGTERM"): + signum = getattr(signal, name, None) + if signum is None: + continue + previous = signal.getsignal(signum) + if previous != signal.SIG_IGN: + signal.signal(signum, _killing_runs_first(_kill_live_runs, previous)) + _runs_stop_on_signals = True + + +def _killing_runs_first(kill_runs, previous): + """A handler that calls ``kill_runs``, then, whatever that did, does what + ``previous`` (a handler, ``SIG_DFL``, or None for one set outside Python) + did.""" + + def handler(signum, frame): + try: + kill_runs() + finally: + if callable(previous): + previous(signum, frame) + else: + signal.signal(signum, signal.SIG_DFL) + os.kill(os.getpid(), signum) + + return handler + + def _passed_over_line(passed_over: list[tuple[str, int]], *, continuing: str | None) -> str: """What ``--continue`` says of the newer conversations other processes have open: each by its id and holder, past three a count.""" @@ -2174,10 +2226,14 @@ async def run(self) -> None: It installs the filters that log what ADK would print over the prompt (``workflow/adk/log_filters.py``) first: an app that is built - but never run, as a library user's, keeps Python's own handling. + but never run, as a library user's, keeps Python's own handling. It + installs there, likewise, the SIGHUP and SIGTERM handlers that kill + the code ``execute_python`` runs before the app goes + (``_stop_runs_on_signals``). """ logger.info("repl_starting") install_log_filters() + _stop_runs_on_signals() job_monitor = self._new_job_monitor() self._job_monitor = job_monitor # told when a conversation is deleted diff --git a/src/agentic_cli/tools/exec_policy.py b/src/agentic_cli/tools/exec_policy.py index e1d7d46b..6dca7d24 100644 --- a/src/agentic_cli/tools/exec_policy.py +++ b/src/agentic_cli/tools/exec_policy.py @@ -87,8 +87,8 @@ def _protect_memory_folders(policy: "OSSandboxPolicy", settings: Any) -> None: the read-only mount. * A folder that exists is denied for both backends; one that could not be made is denied by seatbelt alone (``deny_write_missing_paths``): - bubblewrap binds only what exists, and a bind over a missing path - leaves a stray file where the store later makes its folder. + bubblewrap covers a missing path with a placeholder folder, which + could not be made there either (the run would be refused). """ workspace = getattr(settings, "workspace_dir", None) if workspace is None: diff --git a/src/agentic_cli/tools/executor.py b/src/agentic_cli/tools/executor.py index 16beb7a4..470a250b 100644 --- a/src/agentic_cli/tools/executor.py +++ b/src/agentic_cli/tools/executor.py @@ -10,11 +10,17 @@ from __future__ import annotations import ast +import atexit +import contextlib import json +import os +import signal import subprocess import sys import textwrap +import threading import time +from collections.abc import Callable from dataclasses import dataclass from pathlib import Path from typing import Any, TYPE_CHECKING @@ -58,13 +64,177 @@ def how_to_run(opt_out: str) -> str: ) +#: The runs ``_run_command`` started that have not ended, each the leader of +#: a session of its own: ``_kill_live_runs`` kills them all, as a timeout +#: does, when the app ends (``atexit``, and the CLI's SIGHUP and SIGTERM +#: handlers, ``cli/app.py::_stop_runs_on_signals``): a signal sent to the +#: app's process group no longer reaches them. +_live_runs: set[subprocess.Popen[str]] = set() +_live_runs_lock = threading.RLock() + + +def _own_sessions() -> bool: + """Whether a run gets a session of its own: where a process group can be + killed (POSIX). On Windows the run is killed alone, as ``subprocess.run`` + kills it.""" + return hasattr(os, "killpg") and hasattr(signal, "SIGKILL") + + +def _run_command( + command: str | list[str], + *, + shell: bool, + input: str, + timeout: float, + env: dict[str, str] | None = None, + cwd: str | os.PathLike[str] | None = None, +) -> subprocess.CompletedProcess[str]: + """``subprocess.run(command, input=input, capture_output=True, text=True, + timeout=timeout)``, in a session of its own, so that a timeout, a + KeyboardInterrupt while it waits (in a main-thread caller: the CLI runs + it on a worker thread, where none reaches it), or ``_kill_live_runs`` + kills every process the command started, and a run that ends on its own + takes with it what it left in its process group. + + ``subprocess.run`` kills only the process it started: the shell, or a + sandbox tool's monitor, while the code and what it forked ran on. Here + the whole process group is stopped, then each child of the group that + left it is killed (Linux, through ``/proc``), then the group. Such a + child is bubblewrap's sandbox init, in a session of its own + (``--new-session``), whose death ends every process in the sandbox. It + must be killed by its host pid: bwrap sets its parent-death signal before + it execs the code, but an exec into another AppArmor label clears it + (``security/apparmor/lsm.c::apparmor_bprm_committing_creds``; Ubuntu's + ``bwrap-userns-restrict`` profile execs the code so, with ``pix``), so it + does not die with bwrap. Outside a pid namespace, code that calls + ``setsid()`` can outlive the run: on macOS, which has no ``/proc`` walk, + a fork that calls it once; on Linux the walk only catches a direct child + of a live group member. + + Raises: + subprocess.TimeoutExpired: The command was killed at ``timeout``. + """ + own_session = _own_sessions() + with subprocess.Popen( + command, + shell=shell, + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + text=True, + env=env, + cwd=cwd, + start_new_session=own_session, + ) as proc: + try: + with _live_runs_lock: + _live_runs.add(proc) + try: + stdout, stderr = proc.communicate(input, timeout=timeout) + except BaseException: + _kill_run(proc, own_session) + proc.wait() + raise + if own_session: + # Ended on its own: what it left in its group ends with it, + # as bubblewrap's namespace ends with the code. + with contextlib.suppress(ProcessLookupError, PermissionError): + os.killpg(proc.pid, signal.SIGKILL) + return subprocess.CompletedProcess(proc.args, proc.returncode, stdout, stderr) + finally: + with _live_runs_lock: + _live_runs.discard(proc) + + +def _kill_live_runs() -> None: + """Kill every run still going, as a timeout does. Registered with + ``atexit``; the CLI also calls it on SIGHUP and SIGTERM.""" + with _live_runs_lock: + runs = [proc for proc in _live_runs if proc.returncode is None] # not reaped + for proc in runs: + _kill_run(proc, _own_sessions()) + + +atexit.register(_kill_live_runs) + + +def _kill_run(proc: subprocess.Popen[str], own_session: bool) -> None: + """Kill everything ``proc``'s run started: its session (``_kill_session``) + and, whatever happens there, the process it started (all there is to + kill without a session).""" + try: + if own_session: + _kill_session(proc.pid) + finally: + with contextlib.suppress(OSError): + if hasattr(signal, "SIGKILL"): + proc.kill() + else: + proc.terminate() + + +def _kill_session(leader: int) -> None: + """SIGKILL the process group ``leader`` leads and, first, each child of + the group that left it. + + The group is stopped first: nothing in it can then fork, leave the group + or reap a child, so a child the walk finds keeps its pid until it is + killed. A SIGKILL from the parent namespace is delivered to a pid + namespace's init, and its death ends every process in it; it also ends + stopped processes. The walk is best effort: if it fails, or anything + interrupts it, the group is killed all the same.""" + with contextlib.suppress(ProcessLookupError, PermissionError): + os.killpg(leader, signal.SIGSTOP) + try: + try: + children = _children_out_of_group(leader) + except Exception: # noqa: BLE001 - the group is killed whatever the walk does + logger.warning("python_executor.kill_walk_failed", exc_info=True) + children = [] + for pid in children: + with contextlib.suppress(ProcessLookupError, PermissionError): + os.kill(pid, signal.SIGKILL) + finally: + # Whatever interrupts the walk: a group left stopped, orphaned in a + # session of its own, would never be continued. + with contextlib.suppress(ProcessLookupError, PermissionError): + os.killpg(leader, signal.SIGKILL) + + +def _children_out_of_group(group: int) -> list[int]: + """The processes whose parent is in the process group ``group`` but which + are not, read from ``/proc`` (Linux; elsewhere none).""" + processes: dict[int, tuple[int, int]] = {} + try: + names = os.listdir("/proc") + except OSError: + return [] + for name in names: + if not name.isdigit(): + continue + try: + with open(f"/proc/{name}/stat", encoding="utf-8", errors="replace") as stat: + fields = stat.read().rsplit(")", 1)[1].split() + except (OSError, IndexError): + continue # gone meanwhile + # After the command's name: state, ppid, pgrp. + processes[int(name)] = (int(fields[1]), int(fields[2])) + members = {pid for pid, (_, pgrp) in processes.items() if pgrp == group} + return [ + pid for pid, (ppid, pgrp) in processes.items() if ppid in members and pgrp != group + ] + + @dataclass(frozen=True) class _Run: """Where one call's code runs, decided before it is validated. - ``command`` is what ``subprocess.run`` gets (a shell string when the - sandbox wrapped it), ``modules`` what the code may import there, and - ``refusal`` the error when it runs nowhere. + ``command`` is what ``_run_command`` gets (a shell string when the + sandbox wrapped it), ``modules`` what the code may import there, + ``refusal`` the error when it runs nowhere, and ``release`` the wrap's + (``OSSandboxResult.release``), called once the code was refused by the + checks or its process ended on its own: never after a timeout, nor after + a status that says a signal ended it. """ command: str | list[str] = "" @@ -73,6 +243,7 @@ class _Run: sandboxed: bool = False sandbox_type: str = "" refusal: str | None = None + release: Callable[[], None] = lambda: None class SafePythonExecutor: @@ -100,7 +271,14 @@ class SafePythonExecutor: Every run also has a timeout, an address-space limit (``RLIMIT_AS``, which macOS does not enforce), the sentinel output protocol, and an - environment without the provider keys. + environment without the provider keys. The timeout kills every process + the run started (``_run_command``), and so do, where the CLI runs + (``BaseCLIApp.run``), a hangup or SIGTERM the app does not ignore. A + normal exit waits for a call still running on the tool thread pool, as + before (to its timeout); ``atexit`` kills what is left, a run on a daemon + thread, which is all a library user who never runs the CLI gets. Outside a pid namespace, code that calls ``setsid()`` + can outlive them (on Linux the walk only catches a direct child of a + live group member). """ # Modules that expose file, network, or pickle I/O through public, @@ -406,6 +584,7 @@ def _choose_run(self, context: dict[str, Any] | None) -> _Run: modules=modules, sandboxed=True, sandbox_type=wrap.sandbox_type, + release=wrap.release, ) if strict: return _Run(refusal=( @@ -462,6 +641,7 @@ def _validate_and_run( logger.debug("python_executor.validation", code_length=len(code)) is_valid, error = self.validate_code(code, allowed_modules=run.modules) if not is_valid: + run.release() # nothing ran return { "success": False, "output": "", @@ -477,16 +657,17 @@ def _validate_and_run( sandboxed=run.sandboxed, ) try: - proc = subprocess.run( + proc = _run_command( run.command, shell=run.shell, input=code, - capture_output=True, - text=True, timeout=timeout, env=self._subprocess_env(), ) except subprocess.TimeoutExpired: + # Killed, every process it started with it, though the kernel may + # take a moment to empty the sandbox: what the wrap holds for the + # run is kept, not released. elapsed = (time.time() - start_time) * 1000 logger.warning("python_executor.timeout", timeout=timeout) return { @@ -496,6 +677,13 @@ def _validate_and_run( "error": f"Execution timed out after {timeout} seconds", "execution_time_ms": round(elapsed, 2), } + # Ended on its own: in bubblewrap the code is the sandbox's pid 1, so + # every process it started is gone by now. A status that says a signal + # ended it (negative, or 128 and above from a shell) may be a signal + # from outside that killed the shell or bwrap's monitor, which returns + # before the kernel has emptied the sandbox: the holds are kept then. + if 0 <= proc.returncode < 128: + run.release() elapsed = (time.time() - start_time) * 1000 diff --git a/src/agentic_cli/tools/os_sandbox/__init__.py b/src/agentic_cli/tools/os_sandbox/__init__.py index 0489b3b7..f51fcad6 100644 --- a/src/agentic_cli/tools/os_sandbox/__init__.py +++ b/src/agentic_cli/tools/os_sandbox/__init__.py @@ -11,7 +11,13 @@ sandbox = get_os_sandbox() policy = OSSandboxPolicy(writable_paths=["/tmp"]) result = sandbox.wrap_shell_command("echo hello", Path.cwd(), policy) + if not result.success: # result.command is then the bare command + raise RuntimeError(result.error) # result.command is now "sandbox-exec -p '...' /bin/bash -c 'echo hello'" + # Run it in a session of its own, and on a timeout kill the session, not + # the shell alone (subprocess.run's way: the sandbox runs on), as + # tools/executor.py::_run_command does. Once it ended on its own (a + # status from 0 to 127), result.release(); never after a timeout. """ from __future__ import annotations diff --git a/src/agentic_cli/tools/os_sandbox/base.py b/src/agentic_cli/tools/os_sandbox/base.py index 69fa7b9c..56cde06d 100644 --- a/src/agentic_cli/tools/os_sandbox/base.py +++ b/src/agentic_cli/tools/os_sandbox/base.py @@ -9,6 +9,7 @@ import shlex from abc import ABC, abstractmethod +from collections.abc import Callable from dataclasses import dataclass, field from pathlib import Path from typing import TYPE_CHECKING @@ -17,6 +18,10 @@ from agentic_cli.tools.os_sandbox.policy import OSSandboxPolicy +def _nothing_to_release() -> None: + """The release of a wrap that holds nothing on the host.""" + + @dataclass class OSSandboxResult: """Result of wrapping a command with OS sandbox. @@ -27,6 +32,17 @@ class OSSandboxResult: success: Whether wrapping succeeded. error: Error message if wrapping failed. sandbox_type: Identifier for which sandbox was used. + release: Called once the command has ended on its own, or will not + run: it removes what the wrap made on the host for the run + (bubblewrap's placeholders over missing protected paths). Not + called after a timeout or a kill (a status that says a signal + ended the command), which may leave the sandbox's processes alive + a moment longer: what the run holds then stays + (a placeholder holds the run's hold file and stays protected, + and later runs hold it in turn), as it does for a caller that + never calls it. A failed wrap + holds nothing. No part of the result's value: two results that + run the same command compare equal. """ command: str @@ -34,6 +50,7 @@ class OSSandboxResult: success: bool = True error: str | None = None sandbox_type: str = "none" + release: Callable[[], None] = field(default=_nothing_to_release, repr=False, compare=False) class OSSandbox(ABC): diff --git a/src/agentic_cli/tools/os_sandbox/bubblewrap.py b/src/agentic_cli/tools/os_sandbox/bubblewrap.py index 04745bf9..97a5e652 100644 --- a/src/agentic_cli/tools/os_sandbox/bubblewrap.py +++ b/src/agentic_cli/tools/os_sandbox/bubblewrap.py @@ -6,21 +6,74 @@ Bubblewrap creates a lightweight user namespace with: - Read-only root filesystem - Writable bind mounts for allowed paths -- PID namespace isolation +- PID namespace isolation, the command its pid 1 (``--as-pid-1``): when it + exits, the kernel kills and reaps everything it started before bwrap + returns, so a run that returned has left nothing behind in the sandbox - Network namespace isolation (no network) + +A protected path that does not exist (``~/.bash_profile``, a project's +``.git`` before ``git init``) is bound over only where the code could create +it. Below the read-only root it needs nothing, and bwrap, which makes a +missing mount point itself, could make none there: it would exit with "Can't +create file at …: Read-only file system" before the code runs. In a folder +the code may write, bwrap would make the mount point on the host, an empty +file left behind after the run (an empty ``.git`` file breaks git: "invalid +gitfile format"), so the wrap makes a placeholder there instead: an empty +folder bound read-only. Each run holds it with a file of its own, in this +process or another, and ``OSSandboxResult.release`` drops the run's hold and +removes the folder once no hold is left. A run never removes a folder it did +not make: a protected folder that is there and empty is held and bound like a +placeholder (it may be one whose last run is removing it), but a hold in it +says no run made it, and the folder stays. A run that did not end on its own +(a timeout, a kill) is not released: its placeholder stays, holding the run's +hold file and protected, and later runs hold it in turn. """ from __future__ import annotations +import contextlib +import errno +import os import shlex +import uuid +from collections.abc import Callable, Collection +from dataclasses import dataclass from pathlib import Path from typing import TYPE_CHECKING +from agentic_cli.file_utils import open_dir_no_follow from agentic_cli.tools.os_sandbox.base import OSSandbox, OSSandboxResult if TYPE_CHECKING: from agentic_cli.tools.os_sandbox.policy import OSSandboxPolicy +#: A run's hold in a placeholder is a file named so. A placeholder is a folder +#: below a writable bind holding nothing else; each run that binds it holds +#: it, and the last to release removes it if a run made it (see ``_KEPT``). +_HOLD = ".agentic-cli-sandbox-hold-" +#: A hold in a folder no run made: one that was there and empty when the run +#: joined it, or one holding such a hold. No run removes it. +_KEPT = f"{_HOLD}kept-" +#: The folder a placeholder is made in, with its first hold, before it is +#: renamed into place: it never appears without one. +_STAGING = ".agentic-cli-sandbox-staging-" +_HOLD_NOTE = "A sandboxed agentic-cli run keeps the folder this file is in until it ends.\n" +#: How often the wrap looks again at a protected path that changed under it. +_TRIES = 5 + + +@dataclass(frozen=True) +class _Hold: + """A run's hold: the file ``name`` in the placeholder ``point``, which + lies below the writable bind ``root``. ``removes``: the release may remove + the folder (a run made it). ``name`` None: a protected folder no hold + could go in, bound as it is, with nothing to drop.""" + + root: Path + point: Path + name: str | None + removes: bool = False + class BubblewrapSandbox(OSSandbox): """Linux sandbox using bubblewrap (bwrap) with namespace isolation.""" @@ -39,14 +92,18 @@ def wrap_shell_command( policy: Sandbox policy. Returns: - OSSandboxResult with the wrapped command. + OSSandboxResult with the wrapped command; call its ``release`` + once the command has ended on its own. """ + release: Callable[[], None] | None = None try: - bwrap_args = self._build_bwrap_args(working_dir, policy) + bwrap_args, release = self._prepare(working_dir, policy) escaped_command = command.replace("'", "'\\''") wrapped = f"{shlex.join(bwrap_args)} -- /bin/bash -c '{escaped_command}'" - return OSSandboxResult(command=wrapped, sandbox_type="bubblewrap") + return OSSandboxResult(command=wrapped, sandbox_type="bubblewrap", release=release) except Exception as e: + if release is not None: + release() return OSSandboxResult( command=command, success=False, @@ -68,13 +125,17 @@ def wrap_python_command( policy: Sandbox policy. Returns: - OSSandboxResult with the wrapped command. + OSSandboxResult with the wrapped command; call its ``release`` + once the command has ended on its own. """ + release: Callable[[], None] | None = None try: - bwrap_args = self._build_bwrap_args(working_dir, policy) + bwrap_args, release = self._prepare(working_dir, policy) wrapped = shlex.join(bwrap_args + ["--"] + python_args) - return OSSandboxResult(command=wrapped, sandbox_type="bubblewrap") + return OSSandboxResult(command=wrapped, sandbox_type="bubblewrap", release=release) except Exception as e: + if release is not None: + release() return OSSandboxResult( command=shlex.join(python_args), success=False, @@ -82,8 +143,79 @@ def wrap_python_command( sandbox_type="bubblewrap", ) - def _build_bwrap_args( + def _prepare( + self, working_dir: Path, policy: OSSandboxPolicy + ) -> tuple[list[str], Callable[[], None]]: + """The bwrap arguments, with the placeholders they bind held, and the + release of those holds.""" + holds = self._hold_placeholders(working_dir, policy) + release = _releaser(holds) + try: + held = [hold.point for hold in holds] + return self._build_bwrap_args(working_dir, policy, held=held), release + except BaseException: + release() + raise + + def _hold_placeholders( self, working_dir: Path, policy: OSSandboxPolicy + ) -> list[_Hold]: + """Hold a placeholder at each missing protected path the code could + create (joining one another run holds, or one being removed), and + return this run's holds. + + A path the code cannot create needs none: one below the read-only + root, or where no placeholder can be made because the file system is + read-only, or the folder is one the user may not write and does not + own (so the code cannot give itself the right). A protected folder + that is there but takes no hold is bound as it is. + + Raises: + RuntimeError: Any other placeholder could not be made (a folder of + the user's that is read-only by its mode, which the code may + chmod; a folder swapped for a symlink), or a path kept + changing: the code could create the path, so no run. + """ + writable = _bound(policy, working_dir) + deny_writes = policy.resolved_deny_write_paths(working_dir) + read_only = _real(deny_writes, writable) + holds: dict[Path, _Hold] = {} + try: + for path in deny_writes: + self._hold_one(path, writable, read_only, holds) + except BaseException: + _releaser(list(holds.values()))() + raise + return list(holds.values()) + + @staticmethod + def _hold_one( + path: Path, writable: list[Path], read_only: list[Path], holds: dict[Path, _Hold] + ) -> None: + """Hold the placeholder ``path`` needs, if any, adding it to ``holds``.""" + for _ in range(_TRIES): + point = _creation_point(path, writable) + root = None if point is None else _writable_root(point, writable, read_only) + if point is None or root is None or point in holds: + return + try: + hold = _hold(root, point, own=point == path) + except OSError as e: + if _code_cannot_create(e, point): + return + raise RuntimeError( + f"no placeholder could be made at {point}, a protected path " + f"the code could otherwise create: {e.strerror or e}" + ) from e + if hold is not None: + holds[point] = hold + return + # A real file or folder took its place, or another run's + # placeholder came and went: look again. + raise RuntimeError(f"{path} kept changing while its placeholder was made") + + def _build_bwrap_args( + self, working_dir: Path, policy: OSSandboxPolicy, held: Collection[Path] = () ) -> list[str]: """Build bwrap command-line arguments from policy. @@ -91,18 +223,22 @@ def _build_bwrap_args( so more specific mounts must come after less specific ones. Structure: - 1. Isolation flags (session, die-with-parent, PID namespace) + 1. Isolation flags (session, die-with-parent, PID namespace, the + command as its pid 1) 2. Network namespace (if blocking network) 3. Read-only root (--ro-bind / /) 4. Writable bind mounts for allowed paths - 5. Read-only overrides for denied paths within writable regions - 6. Block nonexistent deny targets - 7. Hide denied read paths - 8. Device and proc filesystems + 5. Read-only overrides for denied paths that exist within writable + regions, and for the placeholders this run holds + 6. Hide denied read paths + 7. Device and proc filesystems Args: working_dir: Working directory (always writable). policy: Sandbox policy. + held: The placeholders this run holds (``_hold_placeholders``, + which the wraps call first). A missing protected path the + code could create is bound only through one of them. Returns: List of bwrap arguments (without the trailing command). @@ -112,6 +248,7 @@ def _build_bwrap_args( "--new-session", "--die-with-parent", "--unshare-pid", + "--as-pid-1", ] # Network isolation @@ -131,15 +268,12 @@ def _build_bwrap_args( # can be written to inside the sandbox args.extend(["--tmpfs", str(wp)]) - # Deny writes within writable regions: re-mount as read-only + # Deny writes within writable regions: re-mount as read-only what + # exists, and the placeholders of what does not; elsewhere the + # read-only root covers a missing path (see the module docstring). deny_writes = policy.resolved_deny_write_paths(working_dir) - for dp in deny_writes: - if dp.exists(): - args.extend(["--ro-bind", str(dp), str(dp)]) - elif dp.parent.exists(): - # Block creation of nonexistent deny targets by binding - # /dev/null over them - args.extend(["--ro-bind", "/dev/null", str(dp)]) + for target in dict.fromkeys([*_real(deny_writes, _bound(policy, working_dir)), *held]): + args.extend(["--ro-bind", str(target), str(target)]) # Hide denied read paths with empty tmpfs overlays deny_reads = policy.resolved_deny_read_paths(working_dir) @@ -165,3 +299,235 @@ def is_available(self) -> bool: @property def sandbox_type(self) -> str: return "bubblewrap" + + +def _bound(policy: OSSandboxPolicy, working_dir: Path) -> list[Path]: + """The writable paths bound from the host (a missing one is a tmpfs, + which nothing of the host's lies below).""" + return [w for w in policy.resolved_writable_paths(working_dir) if w.exists()] + + +def _below(path: Path, place: Path) -> bool: + return path != place and path.is_relative_to(place) + + +def _real(deny_writes: list[Path], writable: list[Path]) -> list[Path]: + """The protected paths that exist as themselves, not as placeholders.""" + return [d for d in deny_writes if os.path.lexists(d) and _creation_point(d, writable) is None] + + +def _holds_in(fd_or_path: int | Path) -> list[str] | None: + """The holds in a folder holding nothing but holds (none, when it is + empty); None for anything else (a real folder, a file, one this user + cannot list).""" + try: + with os.scandir(fd_or_path) as entries: + names = [entry.name for entry in entries] + except OSError: + return None + return names if all(name.startswith(_HOLD) for name in names) else None + + +def _joinable(holds: list[str] | None, own: bool) -> bool: + """A placeholder a run may hold: one holding holds, or, at the protected + path itself, an empty folder (a placeholder whose last run is removing + it, or one that was there: either way joined, never taken for a real + folder the release could remove from under the run). An empty folder + above the path is the user's (a working directory, ``~/.config``).""" + return holds is not None and (bool(holds) or own) + + +def _creation_point(path: Path, writable: list[Path]) -> Path | None: + """Where code would start creating ``path``: its first component that is + missing or, below a writable bind, a placeholder (``path`` itself, or a + folder above it). None when ``path`` exists as itself.""" + for part in (*reversed(path.parents), path): + if not os.path.lexists(part): + return part + if ( + any(_below(part, w) for w in writable) + and not part.is_symlink() + and _joinable(_holds_in(part), own=part == path) + ): + return part + return None + + +def _writable_root(point: Path, writable: list[Path], read_only: list[Path]) -> Path | None: + """The writable bind ``point`` lies below, when the code could create it + there (below no protected path bound read-only over it); else None.""" + if any(_below(point, d) for d in read_only): + return None + roots = [w for w in writable if _below(point, w)] + return max(roots, key=lambda w: len(w.parts), default=None) + + +def _code_cannot_create(error: OSError, point: Path) -> bool: + """Whether making the placeholder failed where the code would fail too: a + read-only file system, or a folder the user may not write and does not + own (its owner could give the code the right with chmod).""" + if error.errno == errno.EROFS: + return True + if error.errno in (errno.EACCES, errno.EPERM): + try: + return os.stat(point.parent).st_uid != os.geteuid() + except OSError: + return False + return False + + +def _open_dir_below(root: Path, folder: Path) -> int: + """Open ``folder``, ``root`` or below it, following no symlink below + ``root``: code a run may write there could swap a folder for one. ``root`` + is a writable bind, a mount point in every sandbox, which the code cannot + move.""" + fd = os.open(root, os.O_RDONLY | os.O_DIRECTORY) + try: + for name in folder.relative_to(root).parts: + inner = open_dir_no_follow(name, dir_fd=fd) + os.close(fd) + fd = inner + return fd + except BaseException: + os.close(fd) + raise + + +def _hold(root: Path, point: Path, own: bool) -> _Hold | None: + """This run's hold on the placeholder at ``point`` (``own``: the protected + path itself, not a folder above it), made if missing; None when the place + changed meanwhile (a folder on the way went away, a real file or folder + now stands there, the placeholder was removed before the hold was + written, or another run made it first). Every step is taken in folders + opened without following a symlink below ``root``.""" + run = f"{os.getpid()}-{uuid.uuid4().hex}" + try: + parent = _open_dir_below(root, point.parent) + except FileNotFoundError: + return None + try: + try: + folder = open_dir_no_follow(point.name, dir_fd=parent) + except FileNotFoundError: + folder = None + except OSError as e: + if e.errno in (errno.ELOOP, errno.ENOTDIR): + return None # a symlink or a file: no placeholder of ours + raise + if folder is not None: + try: + return _join(root, point, folder, run, own) + finally: + os.close(folder) + staging = f"{_STAGING}{uuid.uuid4().hex}" + name = f"{_HOLD}{run}" + try: + os.mkdir(staging, 0o700, dir_fd=parent) + except FileNotFoundError: + return None # the folder it goes in went away + try: + folder = open_dir_no_follow(staging, dir_fd=parent) + try: + _write_hold(name, folder) + finally: + os.close(folder) + os.rename(staging, point.name, src_dir_fd=parent, dst_dir_fd=parent) + except BaseException as error: + _unhold(parent, staging, name) + if not (isinstance(error, OSError) and _exists(point.name, parent)): + raise + return None # made meanwhile, by another run or for real + return _Hold(root, point, name, removes=True) + finally: + os.close(parent) + + +def _join(root: Path, point: Path, folder: int, run: str, own: bool) -> _Hold | None: + """Hold the existing folder at ``point`` (open as ``folder``) as a + placeholder. A folder that holds no hold, or a kept one, was not made by + a run: the hold says so, and its release leaves the folder. One the user + may not write takes no hold, as a run that made it could not have, so no + run removes it: it is bound as it is (the code could still move it + aside). Any other failure (a full disk) raises: bound without a hold, the + folder could be removed by the run that made it while this one runs.""" + holds = _holds_in(folder) + if not _joinable(holds, own): + return None # a real folder now: looked at again, bound as it is + kept = not holds or any(name.startswith(_KEPT) for name in holds) + name = f"{_KEPT if kept else _HOLD}{run}" + try: + _write_hold(name, folder) + except FileNotFoundError: + return None # removed by its last run before the hold was written + except OSError as e: + if e.errno in (errno.EACCES, errno.EPERM, errno.EROFS): + return _Hold(root, point, None) + raise + return _Hold(root, point, name, removes=not kept) + + +def _exists(name: str, parent: int) -> bool: + try: + os.stat(name, dir_fd=parent, follow_symlinks=False) + except FileNotFoundError: + return False + return True + + +def _write_hold(name: str, folder: int) -> None: + """Write the hold ``name`` in ``folder``; one whose write fails is removed.""" + fd = os.open(name, os.O_WRONLY | os.O_CREAT | os.O_EXCL | os.O_NOFOLLOW, 0o600, dir_fd=folder) + try: + os.write(fd, _HOLD_NOTE.encode()) + except BaseException: + with contextlib.suppress(OSError): + os.unlink(name, dir_fd=folder) + raise + finally: + os.close(fd) + + +def _unhold(parent: int, folder_name: str, hold_name: str, *, remove: bool = True) -> None: + """Remove the hold ``hold_name`` from the folder ``folder_name`` of + ``parent``, then, if ``remove``, the folder if nothing is left in it + (``rmdir``: another run's hold, or whatever the user put there meanwhile, + keeps it).""" + with contextlib.suppress(OSError): + folder = open_dir_no_follow(folder_name, dir_fd=parent) + try: + os.unlink(hold_name, dir_fd=folder) + finally: + os.close(folder) + if remove: + with contextlib.suppress(OSError): + os.rmdir(folder_name, dir_fd=parent) + + +def _drop(hold: _Hold) -> None: + """Drop a hold, and its placeholder if a run made it and no other run + holds it.""" + if hold.name is None: + return + try: + parent = _open_dir_below(hold.root, hold.point.parent) + except OSError: + return # moved or swapped meanwhile: left as it is + try: + _unhold(parent, hold.point.name, hold.name, remove=hold.removes) + finally: + os.close(parent) + + +def _releaser(holds: list[_Hold]) -> Callable[[], None]: + """The release of ``holds``: each is dropped once, however often and from + however many threads it is called.""" + + def release() -> None: + while True: + try: + hold = holds.pop() + except IndexError: + return + _drop(hold) + + return release diff --git a/src/agentic_cli/tools/os_sandbox/policy.py b/src/agentic_cli/tools/os_sandbox/policy.py index e308e0ac..9c7f074b 100644 --- a/src/agentic_cli/tools/os_sandbox/policy.py +++ b/src/agentic_cli/tools/os_sandbox/policy.py @@ -67,15 +67,22 @@ class OSSandboxPolicy: enabled: Whether OS sandboxing is active. writable_paths: Additional paths the sandboxed process can write to. The working directory is always writable regardless of this list. + A protected path that does not exist below one of them (a missing + ``~/.bash_profile`` when the list holds the home directory) is + held by bubblewrap with a placeholder folder while a run lasts, + and for good after a run that timed out or was killed: a folder at + ``~/.bash_profile`` makes a login bash skip ``~/.profile``. deny_write_paths: Paths denied for writing even within writable dirs. Defaults to MANDATORY_DENY_WRITE; a caller that passes its own list replaces it (the framework never passes one). deny_write_missing_paths: Paths denied for writing that may not exist yet, so the code cannot create them. Seatbelt names a path in a - rule, so it denies them all; bubblewrap binds only what exists - (a bind over a missing path leaves a stray file behind), so it - leaves out those that do not. A path that exists belongs in - ``deny_write_paths``, which both backends honor. + rule, so it denies them all; bubblewrap leaves out those that do + not exist: it covers a missing path with a placeholder folder, + and these are folders that could not be made, where no + placeholder could be either (the run would be refused). A path + that exists belongs in ``deny_write_paths``, which both backends + honor. deny_rename_paths: Paths whose own entry cannot be renamed, moved or removed, though what is below them may still be written (a ``literal`` rule). Code that may write a folder's parent could diff --git a/tests/cli/test_runs_stop_on_signals.py b/tests/cli/test_runs_stop_on_signals.py new file mode 100644 index 00000000..a3148a66 --- /dev/null +++ b/tests/cli/test_runs_stop_on_signals.py @@ -0,0 +1,165 @@ +"""A call ``execute_python`` runs is in a session of its own +(``tools/executor.py::_run_command``), where the hangup a closing terminal +sends the app's process group, or a SIGTERM sent to the app, does not reach +it. ``BaseCLIApp.run()`` installs handlers that kill the live runs first, +then do what the signal did before: the app still dies of it.""" + +from __future__ import annotations + +import os +import signal +import subprocess +import sys +import textwrap +import time +from pathlib import Path + +import pytest + +from agentic_cli.cli import AppInfo +from agentic_cli.cli import app as app_module +from agentic_cli.cli.app import BaseCLIApp +from agentic_cli.config import BaseSettings +from agentic_cli.workflow.config import AgentConfig + +pytestmark = pytest.mark.skipif(not hasattr(signal, "SIGHUP"), reason="POSIX signals") + + +def _alive(pid: int) -> bool: + try: + os.kill(pid, 0) + except ProcessLookupError: + return False + return True + + +def _app(pidfile: Path, before: str = "") -> str: + """An app as a job: the handlers installed, a call running on a worker + thread (its process writes its pid and sleeps), the main thread idle.""" + sleeper = [sys.executable, "-c", + f"import os, time; open({str(pidfile)!r}, 'w').write(str(os.getpid())); " + "time.sleep(30)"] + return textwrap.dedent(f""" + import threading, time + {before} + from agentic_cli.cli.app import _stop_runs_on_signals + from agentic_cli.tools import executor + _stop_runs_on_signals() + threading.Thread(target=lambda: executor._run_command( + {sleeper!r}, shell=False, input="", timeout=60), daemon=True).start() + time.sleep(60) + """) + + +def _signal_the_job(app: str, pidfile: Path, signum: int) -> tuple[int, bool]: + """Start ``app`` as a job (a process group of its own), send the group + ``signum`` once its run started, as a shell does to a job when the + terminal closes; the app's exit status, and whether the run outlived it.""" + job = subprocess.Popen([sys.executable, "-c", app], process_group=0) + try: + deadline = time.monotonic() + 20 + while not (pidfile.exists() and pidfile.read_text()): + assert time.monotonic() < deadline, "the run never started" + assert job.poll() is None, f"the app ended first: {job.returncode}" + time.sleep(0.05) + run = int(pidfile.read_text()) + os.killpg(job.pid, signum) + job.wait(20) + finally: + if job.poll() is None: + job.kill() + time.sleep(1) + survived = _alive(run) + if survived: + os.kill(run, signal.SIGKILL) + return job.returncode, survived + + +@pytest.mark.parametrize("signum", [signal.SIGHUP, signal.SIGTERM], ids=["SIGHUP", "SIGTERM"]) +def test_the_signal_kills_the_run_and_still_ends_the_app(tmp_path, signum): + pidfile = tmp_path / "run.pid" + status, survived = _signal_the_job(_app(pidfile), pidfile, signum) + assert not survived, "the run outlived the app" + assert status == -signum # it died of the signal, as it did before + + +def test_a_handler_the_app_had_still_runs(tmp_path): + """A SIGTERM handler of the app's own: the runs are killed, then it runs + (here it writes a file and exits 3).""" + pidfile, marker = tmp_path / "run.pid", tmp_path / "handled" + before = textwrap.dedent(f""" + import os, signal + def own(signum, frame): + open({str(marker)!r}, "w").write("handled") + os._exit(3) + signal.signal(signal.SIGTERM, own) + """).replace("\n", "\n ") + status, survived = _signal_the_job(_app(pidfile, before), pidfile, signal.SIGTERM) + assert not survived + assert marker.read_text() == "handled" + assert status == 3 + + +def test_an_ignored_signal_stays_ignored(tmp_path): + """Under ``nohup``, or in an app that ignores SIGHUP, a hangup changes + nothing: no handler is installed for it, the app goes on, and so does + its run (whose timeout still applies).""" + pidfile = tmp_path / "run.pid" + before = "import signal; signal.signal(signal.SIGHUP, signal.SIG_IGN)" + job = subprocess.Popen([sys.executable, "-c", _app(pidfile, before)], process_group=0) + run = None + try: + deadline = time.monotonic() + 20 + while not (pidfile.exists() and pidfile.read_text()): + assert time.monotonic() < deadline, "the run never started" + assert job.poll() is None, f"the app ended first: {job.returncode}" + time.sleep(0.05) + run = int(pidfile.read_text()) + os.killpg(job.pid, signal.SIGHUP) + time.sleep(1) + assert job.poll() is None, "the app died of a signal it ignores" + assert _alive(run), "the hangup killed the run of an app that ignores it" + finally: + job.kill() + job.wait(10) + if run is not None and _alive(run): + os.kill(run, signal.SIGKILL) + + +def test_the_signal_still_does_what_it_did_when_killing_the_runs_fails(): + """The chain (or the default's re-raise) runs whatever killing the runs + does.""" + chained: list[int] = [] + + def broken(): + raise RuntimeError("no runs today") + + handler = app_module._killing_runs_first(broken, lambda signum, frame: chained.append(signum)) + with pytest.raises(RuntimeError): + handler(signal.SIGTERM, None) + assert chained == [signal.SIGTERM] + + +def test_run_installs_the_handlers_first(tmp_path, monkeypatch): + (tmp_path / "home").mkdir() + monkeypatch.chdir(tmp_path) + monkeypatch.setenv("HOME", str(tmp_path / "home")) + monkeypatch.setattr(app_module, "configure_logging", lambda *a, **k: None) + monkeypatch.setattr(app_module, "install_log_filters", lambda: None) + + class Installed(Exception): + pass + + def installed(): + raise Installed + + monkeypatch.setattr(app_module, "_stop_runs_on_signals", installed) + app = BaseCLIApp( + app_info=AppInfo(name="Signals test", version="0"), + agent_configs=[AgentConfig(name="a", prompt="p")], + settings=BaseSettings(session_store="memory"), + ) + import asyncio + + with pytest.raises(Installed): + asyncio.run(app.run()) diff --git a/tests/test_executor_security.py b/tests/test_executor_security.py index 1c03b2d2..33216863 100644 --- a/tests/test_executor_security.py +++ b/tests/test_executor_security.py @@ -8,7 +8,6 @@ """ import shlex -import subprocess from types import SimpleNamespace import pytest @@ -48,15 +47,17 @@ def _patch_sandbox(monkeypatch, sandbox_type: str) -> _FakeSandbox: def _scripts_run(monkeypatch) -> list[str]: """The runner script of each unwrapped subprocess the executor starts.""" + from agentic_cli.tools import executor as executor_module + scripts: list[str] = [] - real_run = subprocess.run + real_run = executor_module._run_command def _run(*args, **kwargs): if not kwargs.get("shell"): scripts.append(args[0][2]) return real_run(*args, **kwargs) - monkeypatch.setattr(subprocess, "run", _run) + monkeypatch.setattr(executor_module, "_run_command", _run) return scripts diff --git a/tests/tools/test_bubblewrap.py b/tests/tools/test_bubblewrap.py index 0fc6214c..9bc3474d 100644 --- a/tests/tools/test_bubblewrap.py +++ b/tests/tools/test_bubblewrap.py @@ -15,12 +15,16 @@ from __future__ import annotations +import errno +import os +import shlex from pathlib import Path from unittest.mock import patch import pytest -from agentic_cli.tools.os_sandbox.bubblewrap import BubblewrapSandbox +from agentic_cli.tools.os_sandbox import bubblewrap +from agentic_cli.tools.os_sandbox.bubblewrap import _STAGING, BubblewrapSandbox from agentic_cli.tools.os_sandbox.policy import OSSandboxPolicy @@ -41,6 +45,45 @@ def _minimal_policy(**overrides) -> OSSandboxPolicy: return OSSandboxPolicy(**defaults) +def _bwrap_args(result) -> list[str]: + """The bwrap arguments of a wrapped command (what precedes ``--``).""" + argv = shlex.split(result.command) + return argv[: argv.index("--")] + + +def _ro_binds(args: list[str]) -> list[tuple[str, str]]: + return [(args[i + 1], args[i + 2]) for i in range(len(args)) if args[i] == "--ro-bind"] + + +def _what_bwrap_cannot_set_up(args: list[str]) -> list[str]: + """The mounts bwrap would exit on before the code runs, as it sets them up + in order: a bind whose source is missing, and a mount point that is missing + where no earlier ``--bind`` lets bwrap make it (bwrap makes a missing mount + point itself, on the host, and below ``--ro-bind / /`` that fails with + "Can't create file at …: Read-only file system").""" + writable: list[Path] = [] + failures: list[str] = [] + i = 0 + while i < len(args): + op = args[i] + if op in ("--bind", "--ro-bind"): + source, dest = args[i + 1], Path(args[i + 2]) + i += 3 + elif op == "--tmpfs": + source, dest = None, Path(args[i + 1]) + i += 2 + else: + i += 1 + continue + if source is not None and not os.path.lexists(source): + failures.append(f"no source {source}") + if not os.path.lexists(dest) and not any(dest.parent.is_relative_to(w) for w in writable): + failures.append(f"cannot make {dest}") + if op == "--bind": + writable.append(dest) + return failures + + # --------------------------------------------------------------------------- # Basic bwrap args # --------------------------------------------------------------------------- @@ -56,6 +99,14 @@ def test_starts_with_bwrap_and_isolation_flags(self, tmp_path: Path): assert "--die-with-parent" in args assert "--unshare-pid" in args + def test_the_code_is_pid_1(self, tmp_path: Path): + """The command is the sandbox's pid 1, not a child of bwrap's reaper: + when it exits the kernel kills and reaps everything it started, and + bwrap returns only after that. With the reaper, bwrap returned as + soon as the command exited, while what it forked could still run.""" + args = BubblewrapSandbox()._build_bwrap_args(tmp_path, _minimal_policy()) + assert "--as-pid-1" in args + def test_ro_bind_root(self, tmp_path: Path): sandbox = BubblewrapSandbox() policy = _minimal_policy() @@ -134,6 +185,7 @@ class TestDenyWritePaths: def test_existing_deny_write_path_gets_ro_bind(self, tmp_path: Path): protected_dir = tmp_path / "protected" protected_dir.mkdir() + (protected_dir / "config").write_text("x") sandbox = BubblewrapSandbox() policy = _minimal_policy(deny_write_paths=[str(protected_dir)]) @@ -148,41 +200,475 @@ def test_existing_deny_write_path_gets_ro_bind(self, tmp_path: Path): ] assert (resolved, resolved) in ro_bind_pairs - def test_nonexistent_deny_write_path_with_existing_parent_gets_devnull( + def test_a_missing_deny_write_path_in_a_writable_dir_is_held_by_a_placeholder( self, tmp_path: Path ): - # Parent exists but target does not - missing_file = tmp_path / "secret.txt" - - sandbox = BubblewrapSandbox() + """The code could create it, so a read-only mount must cover it, and + bwrap would make that mount point on the host itself, as an empty file + it leaves behind. The wrap makes a placeholder folder instead and + removes it once the run is over.""" + missing_file = (tmp_path / "secret.txt").resolve() policy = _minimal_policy(deny_write_paths=[str(missing_file)]) - args = sandbox._build_bwrap_args(tmp_path, policy) - resolved = str(missing_file.resolve()) - # Find --ro-bind /dev/null - devnull_targets = [ - args[i + 2] - for i in range(len(args)) - if args[i] == "--ro-bind" - and i + 2 < len(args) - and args[i + 1] == "/dev/null" - ] - assert resolved in devnull_targets + result = BubblewrapSandbox().wrap_python_command(["python3", "-c", "pass"], tmp_path, policy) - def test_nonexistent_deny_write_path_with_missing_parent_ignored( + assert result.success, result.error + assert missing_file.is_dir() + assert (str(missing_file), str(missing_file)) in _ro_binds(_bwrap_args(result)) + result.release() + assert not os.path.lexists(missing_file) + + def test_an_empty_deny_write_folder_in_a_writable_dir_is_held_and_kept( self, tmp_path: Path ): - # Both parent and target do not exist - deep_missing = tmp_path / "no_parent" / "no_file.txt" + """An empty folder there may be a placeholder whose last run is + removing it, so it is held like one and bound read-only; but no run + made it, so none removes it.""" + empty = (tmp_path / "protected").resolve() + empty.mkdir() + policy = _minimal_policy(deny_write_paths=[str(empty)]) - sandbox = BubblewrapSandbox() - policy = _minimal_policy(deny_write_paths=[str(deep_missing)]) - args = sandbox._build_bwrap_args(tmp_path, policy) + first = BubblewrapSandbox().wrap_python_command(["python3", "-c", "pass"], tmp_path, policy) + second = BubblewrapSandbox().wrap_python_command(["python3", "-c", "pass"], tmp_path, policy) - resolved = str(deep_missing.resolve()) - # Should not appear anywhere in args (neither ro-bind nor devnull) - all_args_str = " ".join(args) - assert resolved not in all_args_str + assert (str(empty), str(empty)) in _ro_binds(_bwrap_args(first)) + assert (str(empty), str(empty)) in _ro_binds(_bwrap_args(second)) + first.release() + second.release() + assert empty.is_dir() + assert os.listdir(empty) == [] + + def test_a_missing_folder_above_a_deny_write_path_is_held_too(self, tmp_path: Path): + """Below a missing folder, the code would create the folder first: the + placeholder goes there, so neither can be made.""" + missing_folder = (tmp_path / "no_parent").resolve() + policy = _minimal_policy(deny_write_paths=[str(missing_folder / "no_file.txt")]) + + result = BubblewrapSandbox().wrap_python_command(["python3", "-c", "pass"], tmp_path, policy) + + assert missing_folder.is_dir() + assert (str(missing_folder), str(missing_folder)) in _ro_binds(_bwrap_args(result)) + result.release() + assert not os.path.lexists(missing_folder) + + +# --------------------------------------------------------------------------- +# The default policy's protected paths, where they do not exist +# --------------------------------------------------------------------------- + +APP = "demoapp" + + +@pytest.fixture +def home(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path: + """A home holding none of the protected files (no ~/.bashrc, ~/.ssh, …).""" + home = (tmp_path / "home").resolve() + home.mkdir() + monkeypatch.setenv("HOME", str(home)) + return home + + +@pytest.fixture +def project(home: Path) -> Path: + """A project that is no git repository and has no ``.demoapp`` yet.""" + project = home / "project" + project.mkdir() + return project + + +def _wrap(project: Path): + return BubblewrapSandbox().wrap_python_command( + ["python3", "-c", "pass"], project, OSSandboxPolicy(app_name=APP) + ) + + +class TestMissingProtectedPaths: + def test_bwrap_can_set_up_every_mount_of_the_default_policy(self, home, project): + """A missing protected file in home (``~/.bash_profile``), or at the + root (``/System`` on Linux), lies below ``--ro-bind / /``: bwrap could + make no mount point there, and the code can create nothing there.""" + result = _wrap(project) + assert result.success, result.error + args = _bwrap_args(result) + try: + assert _what_bwrap_cannot_set_up(args) == [] + writable = [Path(args[i + 2]) for i in range(len(args)) if args[i] == "--bind"] + assert writable == [project] + finally: + getattr(result, "release", lambda: None)() + + def test_the_wrap_writes_nothing_where_the_code_cannot(self, home, project): + result = _wrap(project) + assert sorted(p.name for p in home.iterdir()) == ["project"] + assert str(home / ".bash_profile") not in _bwrap_args(result) + result.release() + + def test_a_project_that_is_no_repository_keeps_no_stray_git(self, home, project): + """While the code runs, ``.git`` and ``.demoapp`` are read-only folders + it cannot write into; once it is over, the project is as it was (an + empty ``.git`` file left behind would break git there: "invalid + gitfile format").""" + result = _wrap(project) + binds = _ro_binds(_bwrap_args(result)) + for name in (".git", f".{APP}"): + assert (project / name).is_dir(), name + assert (str(project / name), str(project / name)) in binds + result.release() + assert os.listdir(project) == [] + + def test_a_placeholder_stays_while_another_run_holds_it(self, home, project): + first = _wrap(project) + second = _wrap(project) + git = project / ".git" + assert git.is_dir() + first.release() + assert git.is_dir() + second.release() + assert not os.path.lexists(git) + + def test_a_placeholder_another_process_holds_is_kept(self, home, project): + """The holds are files in the placeholder, so a run of another process + (another app in the same project) keeps it too.""" + import subprocess + import sys + + child = subprocess.Popen( + [sys.executable, "-c", ( + "import sys\n" + "from pathlib import Path\n" + "from agentic_cli.tools.os_sandbox.bubblewrap import BubblewrapSandbox\n" + "from agentic_cli.tools.os_sandbox.policy import OSSandboxPolicy\n" + "r = BubblewrapSandbox().wrap_python_command(['python3'], Path(sys.argv[1])," + f" OSSandboxPolicy(app_name={APP!r}))\n" + "print('held', flush=True)\n" + "sys.stdin.readline()\n" + "r.release()\n" + ), str(project)], + stdin=subprocess.PIPE, stdout=subprocess.PIPE, text=True, + ) + git = project / ".git" + try: + assert child.stdout.readline().strip() == "held" + ours = _wrap(project) + ours.release() + assert git.is_dir() # the child still holds it + finally: + child.communicate("\n", timeout=30) + assert not os.path.lexists(git) + + def test_what_the_user_puts_in_a_placeholder_is_kept(self, home, project): + """``git init`` run by the user while the code runs makes the + placeholder a repository: the release leaves it.""" + result = _wrap(project) + git = project / ".git" + assert git.is_dir() + (git / "HEAD").write_text("ref: refs/heads/main\n") + result.release() + assert os.listdir(git) == ["HEAD"] + + @pytest.mark.skipif(os.geteuid() == 0, reason="root writes a folder whatever its mode") + def test_a_placeholder_that_cannot_be_made_is_a_failed_wrap(self, home, project): + """The code could still create the path (a folder of the user's, read-only + by its mode, is one the code may chmod), so the wrap fails rather than + leave it unprotected.""" + project.chmod(0o555) + try: + result = _wrap(project) + finally: + project.chmod(0o755) + assert result.success is False + assert str(project / ".git") in result.error + assert os.listdir(project) == [] + + + def test_a_read_only_file_system_needs_no_placeholder(self, home, project, monkeypatch): + """Where no placeholder can be made because the file system is + read-only, the code cannot create the path either.""" + real_mkdir = os.mkdir + + def read_only(path, *args, **kwargs): + if os.path.basename(path).startswith(_STAGING): + raise OSError(errno.EROFS, "Read-only file system", str(path)) + return real_mkdir(path, *args, **kwargs) + + monkeypatch.setattr(os, "mkdir", read_only) + result = _wrap(project) + monkeypatch.undo() + assert result.success, result.error + assert str(project / ".git") not in _bwrap_args(result) + assert os.listdir(project) == [] + + @pytest.mark.skipif(os.geteuid() == 0, reason="root writes a folder whatever its mode") + def test_a_folder_of_another_user_needs_no_placeholder(self, home, project, monkeypatch): + """A folder the user may not write and does not own: the code can + neither write it nor give itself the right to.""" + uid = os.geteuid() + project.chmod(0o555) + monkeypatch.setattr(os, "geteuid", lambda: uid + 1) + try: + result = _wrap(project) + finally: + monkeypatch.undo() + project.chmod(0o755) + assert result.success, result.error + assert str(project / ".git") not in _bwrap_args(result) + + def test_a_placeholder_mid_release_is_joined(self, home, project, monkeypatch): + """Between a release's unlink of the last hold and its rmdir the + placeholder is an empty folder. A run that wraps then holds it, so the + rmdir finds it not empty, rather than binding it as a real folder that + the rmdir then removes from under it.""" + first = _wrap(project) + real_rmdir = os.rmdir + second = [] + + def rmdir_after_another_wrap(path, *args, **kwargs): + if not second: + second.append(None) + second[0] = _wrap(project) + return real_rmdir(path, *args, **kwargs) + + monkeypatch.setattr(os, "rmdir", rmdir_after_another_wrap) + first.release() + monkeypatch.undo() + [later] = second + binds = _ro_binds(_bwrap_args(later)) + for name in (".git", f".{APP}"): + assert (project / name).is_dir(), name + assert (str(project / name), str(project / name)) in binds + later.release() + # The run that joined the emptied folder cannot tell it from one that + # was there before any run: it leaves it, empty. + assert sorted(os.listdir(project)) == [f".{APP}"] + assert os.listdir(project / f".{APP}") == [] + + def test_a_run_removes_no_folder_it_did_not_make(self, home, project): + """The project's ``.git``, there and empty before the run, is still + there after it.""" + (project / ".git").mkdir() + result = _wrap(project) + assert (str(project / ".git"), str(project / ".git")) in _ro_binds(_bwrap_args(result)) + result.release() + assert sorted(os.listdir(project)) == [".git"] + assert os.listdir(project / ".git") == [] + + def test_an_empty_folder_above_a_protected_path_is_no_placeholder(self, home): + """With the home writable, an empty working directory (above ``.git``) + and an empty ``~/.config`` (above ``~/.config/git``) are the user's + folders: neither is bound read-only, nor removed.""" + work = home / "work" / "proj" + work.mkdir(parents=True) + (home / ".config").mkdir() + policy = OSSandboxPolicy(app_name=APP, writable_paths=[str(home)]) + result = BubblewrapSandbox().wrap_python_command(["python3"], work, policy) + assert result.success, result.error + binds = _ro_binds(_bwrap_args(result)) + assert (str(work), str(work)) not in binds + assert (str(home / ".config"), str(home / ".config")) not in binds + assert (str(work / ".git"), str(work / ".git")) in binds + assert (str(home / ".config" / "git"), str(home / ".config" / "git")) in binds + result.release() + assert work.is_dir() and (home / ".config").is_dir() + assert os.listdir(work) == [] and os.listdir(home / ".config") == [] + + def test_a_folder_that_vanishes_before_the_hold_is_looked_at_again( + self, home, project, monkeypatch + ): + real_open = bubblewrap._open_dir_below + calls = [] + + def vanished_once(root, folder): + if not calls: + calls.append(folder) + raise FileNotFoundError(errno.ENOENT, "No such file or directory", str(folder)) + return real_open(root, folder) + + monkeypatch.setattr(bubblewrap, "_open_dir_below", vanished_once) + result = _wrap(project) + assert result.success, result.error + assert (project / ".git").is_dir() + result.release() + + @pytest.mark.skipif(os.geteuid() == 0, reason="root writes a folder whatever its mode") + def test_an_empty_protected_folder_that_cannot_be_held_is_bound_as_it_is( + self, home, monkeypatch + ): + """A ``.git`` the user may not write, in a folder another user owns but + lets everyone write: no hold can go in, but the code could still move + it aside, so it is bound as a real folder.""" + shared = home / "shared" + shared.mkdir() + (shared / "README").write_text("x") + git = shared / ".git" + git.mkdir() + git.chmod(0o555) + shared.chmod(0o777) + uid = os.geteuid() + monkeypatch.setattr(os, "geteuid", lambda: uid + 1) + try: + result = _wrap(shared) + finally: + monkeypatch.undo() + git.chmod(0o755) + shared.chmod(0o755) + assert result.success, result.error + assert (str(git), str(git)) in _ro_binds(_bwrap_args(result)) + result.release() + assert git.is_dir() + + def test_the_argv_alone_binds_only_what_exists_or_the_run_holds(self, home, project): + """Built without the wrap's holds, the argv binds no missing path (bwrap + would exit: no source) and no placeholder another run holds (that run + could remove it while this one runs).""" + policy = OSSandboxPolicy(app_name=APP) + assert _what_bwrap_cannot_set_up(BubblewrapSandbox()._build_bwrap_args(project, policy)) == [] + other = _wrap(project) + try: + args = BubblewrapSandbox()._build_bwrap_args(project, policy) + assert str(project / ".git") not in args + finally: + other.release() + + def test_a_wrap_that_fails_after_holding_releases(self, home, project): + """A wrap that fails once the placeholders are held (here, on a command + that is no string) holds nothing.""" + result = BubblewrapSandbox().wrap_shell_command(None, project, OSSandboxPolicy(app_name=APP)) + assert result.success is False + assert os.listdir(project) == [] + + def test_no_hold_is_made_through_a_folder_swapped_for_a_symlink( + self, home, project, tmp_path, monkeypatch + ): + """A folder below the writable bind replaced by a symlink after the + policy was resolved: nothing is made where it points.""" + outside = tmp_path / "outside" + outside.mkdir() + (project / "link").symlink_to(outside) + policy = OSSandboxPolicy(app_name=APP) + swapped = project / "link" / ".git" + monkeypatch.setattr(policy, "resolved_deny_write_paths", lambda working_dir=None: [swapped]) + result = BubblewrapSandbox().wrap_python_command(["python3"], project, policy) + assert result.success is False + assert os.listdir(outside) == [] + + def test_wrap_shell_command_holds_and_releases_too(self, home, project): + result = BubblewrapSandbox().wrap_shell_command("true", project, OSSandboxPolicy(app_name=APP)) + assert (project / ".git").is_dir() + result.release() + assert os.listdir(project) == [] + + +class TestHoldRetries: + """A placeholder can change between the look and the hold: another run + removes it, or makes it first.""" + + def test_a_placeholder_removed_while_joining_is_made_anew(self, home, project, monkeypatch): + first = _wrap(project) + real_write = bubblewrap._write_hold + calls = [] + + def release_first_then_write(*args, **kwargs): + if not calls: + calls.append(1) + first.release() # the folder is gone before the hold is written + return real_write(*args, **kwargs) + + monkeypatch.setattr(bubblewrap, "_write_hold", release_first_then_write) + second = _wrap(project) + monkeypatch.undo() + assert second.success, second.error + assert (project / ".git").is_dir() + second.release() + assert os.listdir(project) == [] + + def test_a_lost_rename_joins_the_winner(self, home, project, monkeypatch): + real_rename = os.rename + winner = [] + + def another_run_first(src, dst, *args, **kwargs): + if not winner: + monkeypatch.setattr(os, "rename", real_rename) + winner.append(_wrap(project)) + return real_rename(src, dst, *args, **kwargs) + + monkeypatch.setattr(os, "rename", another_run_first) + loser = _wrap(project) + monkeypatch.undo() + assert loser.success, loser.error + git = project / ".git" + assert len(os.listdir(git)) == 2 # both runs hold it + assert not [n for n in os.listdir(project) if n.startswith(_STAGING)] + winner[0].release() + assert git.is_dir() + loser.release() + assert os.listdir(project) == [] + + def test_a_folder_gone_before_the_staging_folder_is_made_is_looked_at_again( + self, home, project, monkeypatch + ): + real_mkdir = os.mkdir + calls = [] + + def gone_once(path, *args, **kwargs): + if os.path.basename(path).startswith(_STAGING) and not calls: + calls.append(path) + raise FileNotFoundError(errno.ENOENT, "No such file or directory", str(path)) + return real_mkdir(path, *args, **kwargs) + + monkeypatch.setattr(os, "mkdir", gone_once) + result = _wrap(project) + monkeypatch.undo() + assert calls + assert result.success, result.error + assert (project / ".git").is_dir() + result.release() + assert os.listdir(project) == [] + + def test_a_hold_that_cannot_be_written_for_want_of_space_fails_the_wrap( + self, home, project, monkeypatch + ): + """A placeholder another run holds, joined on a full disk: bound with + no hold, it could be removed by that run's release while this one runs, + so the wrap fails, and leaves no half-written hold behind.""" + first = _wrap(project) + held = sorted(os.listdir(project / ".git")) + real_write = os.write + + def full(fd, data): + if data == bubblewrap._HOLD_NOTE.encode(): + raise OSError(errno.ENOSPC, "No space left on device") + return real_write(fd, data) + + monkeypatch.setattr(os, "write", full) + second = _wrap(project) + monkeypatch.undo() + try: + assert second.success is False + assert "No space left on device" in second.error + assert sorted(os.listdir(project / ".git")) == held + finally: + first.release() + + def test_a_place_that_keeps_changing_fails_the_wrap(self, home, project, monkeypatch): + monkeypatch.setattr(bubblewrap, "_hold", lambda *args, **kwargs: None) + result = _wrap(project) + assert result.success is False + assert str(project / ".git") in result.error + + +class TestRelease: + def test_release_survives_a_hold_another_thread_took(self): + """Two threads releasing at once: the holds list empties between one's + check and its pop.""" + + class TakenMeanwhile(list): + def __bool__(self) -> bool: + return True + + def __len__(self) -> int: + return 1 + + bubblewrap._releaser(TakenMeanwhile())() # --------------------------------------------------------------------------- diff --git a/tests/tools/test_executor_runs.py b/tests/tools/test_executor_runs.py new file mode 100644 index 00000000..1036686f --- /dev/null +++ b/tests/tools/test_executor_runs.py @@ -0,0 +1,215 @@ +"""How the executor starts, ends and stops the command a call runs +(``tools/executor.py::_run_command``): in a session of its own, so that what +the code starts can be found and killed, at a timeout, at the end of the run, +and when the app ends.""" + +from __future__ import annotations + +import os +import shlex +import signal +import subprocess +import sys +import textwrap +import threading +import time +from pathlib import Path + +import pytest + +from agentic_cli.tools import executor + +POSIX = hasattr(os, "killpg") + + +def _alive(pid: int) -> bool: + try: + os.kill(pid, 0) + except ProcessLookupError: + return False + return True + + +def _wait_for_pid(path: Path, seconds: float = 10) -> int: + deadline = time.monotonic() + seconds + while time.monotonic() < deadline: + if path.exists() and path.read_text().strip(): + return int(path.read_text()) + time.sleep(0.05) + raise AssertionError(f"{path} was never written") + + +def _dead_within(pid: int, seconds: float) -> bool: + deadline = time.monotonic() + seconds + while time.monotonic() < deadline: + if not _alive(pid): + return True + time.sleep(0.05) + return False + + +def _sleeper(pidfile: Path) -> list[str]: + """A command that writes its pid and sleeps.""" + code = f"import os, time; open({str(pidfile)!r}, 'w').write(str(os.getpid())); time.sleep(30)" + return [sys.executable, "-c", code] + + +class TestWithoutProcessGroups: + """Windows has no process groups to kill (no ``os.killpg``, no + ``signal.SIGKILL``): a timeout kills the process the run started, as + ``subprocess.run`` does, and is still a timeout.""" + + def test_a_timeout_kills_the_process_and_says_so(self, monkeypatch): + monkeypatch.delattr(os, "killpg") + monkeypatch.delattr(signal, "SIGKILL") + start = time.monotonic() + with pytest.raises(subprocess.TimeoutExpired): + executor._run_command([sys.executable, "-c", "import time; time.sleep(6)"], + shell=False, input="", timeout=1) + assert time.monotonic() - start < 4 + + +@pytest.mark.skipif(not POSIX, reason="process groups") +class TestKillingTheRun: + def test_a_walk_that_fails_still_ends_in_a_timeout(self, monkeypatch): + """The walk for children that left the group is best effort: if it + fails, the group is still killed and the call is still a timeout.""" + def broken(group): + raise RuntimeError("no /proc today") + + monkeypatch.setattr(executor, "_children_out_of_group", broken) + start = time.monotonic() + with pytest.raises(subprocess.TimeoutExpired): + executor._run_command([sys.executable, "-c", "import time; time.sleep(6)"], + shell=False, input="", timeout=1) + assert time.monotonic() - start < 4 + + def test_a_group_stopped_for_the_walk_is_killed_whatever_interrupts_it( + self, monkeypatch, tmp_path + ): + """Interrupted between its stop and its kill (a KeyboardInterrupt, a + second signal), the group is still killed: stopped, in a session of + its own, nothing would ever continue it.""" + pidfile = tmp_path / "forked.pid" + command = ( + f"(exec {shlex.join(_sleeper(pidfile))}) /dev/null 2>&1 & " + "exec sleep 30" + ) + + def interrupted(group): + raise KeyboardInterrupt + + monkeypatch.setattr(executor, "_children_out_of_group", interrupted) + thread_result: list = [] + + def run(): + try: + executor._run_command(command, shell=True, input="", timeout=1) + except BaseException as e: # noqa: BLE001 - recorded for the test + thread_result.append(e) + + worker = threading.Thread(target=run) + worker.start() + forked = _wait_for_pid(pidfile) + worker.join(10) + assert isinstance(thread_result[0], KeyboardInterrupt) + dead = _dead_within(forked, 2) + if not dead: + os.kill(forked, signal.SIGKILL) + assert dead, f"{forked} was left stopped" + + def test_the_group_is_stopped_before_its_children_are_looked_for(self, monkeypatch): + """Stopped, nothing in the group can fork, leave it, or reap a child + whose pid the walk has read: the children found are still those.""" + sent: list[tuple[str, int]] = [] + real_killpg = os.killpg + + def killpg(group, signum): + sent.append(("group", signum)) + real_killpg(group, signum) + + def walk(group): + sent.append(("walk", 0)) + return [] + + monkeypatch.setattr(os, "killpg", killpg) + monkeypatch.setattr(executor, "_children_out_of_group", walk) + with pytest.raises(subprocess.TimeoutExpired): + executor._run_command([sys.executable, "-c", "import time; time.sleep(6)"], + shell=False, input="", timeout=1) + assert sent[:3] == [("group", signal.SIGSTOP), ("walk", 0), ("group", signal.SIGKILL)] + + def test_what_stays_in_the_group_ends_with_a_run_that_ends(self, tmp_path): + """A process the code forks into the background, its output closed, + stays in the run's group when the run ends on its own: it is killed + then, as bubblewrap's namespace would end it.""" + pidfile = tmp_path / "forked.pid" + command = ( + f"(exec {shlex.join(_sleeper(pidfile))}) /dev/null 2>&1 & " + f"while [ ! -s {shlex.quote(str(pidfile))} ]; do sleep 0.05; done; echo done" + ) + result = executor._run_command(command, shell=True, input="", timeout=20) + assert result.stdout.strip() == "done" + forked = int(pidfile.read_text()) + dead = _dead_within(forked, 2) + if not dead: + os.kill(forked, signal.SIGKILL) + assert dead, f"{forked} outlived the run" + + +@pytest.mark.skipif(not POSIX, reason="process groups") +class TestLiveRuns: + def test_a_run_is_listed_while_it_runs(self, tmp_path): + pidfile = tmp_path / "run.pid" + code = (f"import os, time; open({str(pidfile)!r}, 'w').write(str(os.getpid())); " + "time.sleep(1)") + thread = threading.Thread(target=lambda: executor._run_command( + [sys.executable, "-c", code], shell=False, input="", timeout=20)) + thread.start() + pid = _wait_for_pid(pidfile) + while_running = {proc.pid for proc in executor._live_runs} + thread.join(10) + assert pid in while_running + assert pid not in {proc.pid for proc in executor._live_runs} + + def test_killing_the_live_runs_ends_them(self, tmp_path): + pidfile = tmp_path / "run.pid" + results: list = [] + thread = threading.Thread( + target=lambda: results.append(executor._run_command( + _sleeper(pidfile), shell=False, input="", timeout=60)), + daemon=True, + ) + thread.start() + pid = _wait_for_pid(pidfile) + executor._kill_live_runs() + thread.join(5) + assert not thread.is_alive() + assert _dead_within(pid, 2) + assert results[0].returncode == -signal.SIGKILL + + def test_the_live_runs_are_killed_when_the_app_exits(self, tmp_path): + """An app that exits while a call runs on a worker thread: the run, + in a session of its own, is killed at exit (``atexit``).""" + pidfile = tmp_path / "run.pid" + app = textwrap.dedent(f""" + import sys, threading, time + from agentic_cli.tools import executor + command = {_sleeper(pidfile)!r} + threading.Thread(target=lambda: executor._run_command( + command, shell=False, input="", timeout=60), daemon=True).start() + while True: + try: + if open({str(pidfile)!r}).read(): + break + except FileNotFoundError: + pass + time.sleep(0.05) + sys.exit(0) + """) + subprocess.run([sys.executable, "-c", app], timeout=30, check=True) + pid = int(pidfile.read_text()) + dead = _dead_within(pid, 2) + if not dead: + os.kill(pid, signal.SIGKILL) + assert dead, "the run outlived the app" diff --git a/tests/tools/test_executor_security.py b/tests/tools/test_executor_security.py index 6e2086df..e4750714 100644 --- a/tests/tools/test_executor_security.py +++ b/tests/tools/test_executor_security.py @@ -10,7 +10,9 @@ from __future__ import annotations +import shutil import subprocess +import sys import pytest @@ -127,14 +129,16 @@ def test_api_keys_scrubbed_from_subprocess_env( monkeypatch.setenv("SOME_TOKEN", "tok") monkeypatch.setenv("PATH", "/usr/bin") + from agentic_cli.tools import executor as executor_module + captured: dict = {} - real_run = subprocess.run + real_run = executor_module._run_command def _capture(*args, **kwargs): captured["env"] = kwargs.get("env") return real_run(*args, **kwargs) - monkeypatch.setattr(subprocess, "run", _capture) + monkeypatch.setattr(executor_module, "_run_command", _capture) executor.execute("print(1)") env = captured["env"] @@ -188,17 +192,19 @@ class _Backend: """A stand-in OS sandbox: ``"none"`` is no backend; ``wrap_error`` makes the wrap fail; ``command(argv)``, when given, is the wrapped command (one that fails before Python starts, say); otherwise the "wrap" is the plain - command.""" + command. ``release``, when given, is the wrap's release.""" def __init__( self, sandbox_type: str = "seatbelt", wrap_error: str | None = None, command=None, + release=None, ) -> None: self.sandbox_type = sandbox_type self.wrap_error = wrap_error self.command = command + self.release = release self.wrapped: list[list[str]] = [] def wrap_python_command(self, argv, working_dir, policy): @@ -212,7 +218,10 @@ def wrap_python_command(self, argv, working_dir, policy): command="", success=False, error=self.wrap_error, sandbox_type=self.sandbox_type ) command = self.command(argv) if self.command else shlex.join(argv) - return OSSandboxResult(command=command, sandbox_type=self.sandbox_type) + result = OSSandboxResult(command=command, sandbox_type=self.sandbox_type) + if self.release is not None: + result.release = self.release + return result def _fails_at_start(message: str, status: int): @@ -231,15 +240,17 @@ def _use_backend(monkeypatch: pytest.MonkeyPatch, backend: _Backend) -> _Backend def _record_runs(monkeypatch: pytest.MonkeyPatch) -> list: - """Record each subprocess the executor starts (and still start it).""" + """Record each process the executor starts (and still start it).""" + from agentic_cli.tools import executor as executor_module + runs: list = [] - real_run = subprocess.run + real_run = executor_module._run_command def _run(*args, **kwargs): runs.append((args, kwargs)) return real_run(*args, **kwargs) - monkeypatch.setattr(subprocess, "run", _run) + monkeypatch.setattr(executor_module, "_run_command", _run) return runs @@ -454,6 +465,156 @@ def test_a_failed_wrap_without_strict_runs_core_modules_only( assert _runner_modules(args[0][2]) == SafePythonExecutor.CORE_MODULES +def test_the_wrap_is_released_once_the_code_has_run(monkeypatch: pytest.MonkeyPatch) -> None: + """A wrap may hold something on the host for its run (bubblewrap's + placeholders over missing protected paths): it is released once the run + is over, and when the code is refused before it runs.""" + from agentic_cli.tools.execution_tools import execute_python + from tests.conftest import MockContext + + events: list[str] = [] + _use_backend(monkeypatch, _Backend("seatbelt", release=lambda: events.append("release"))) + from agentic_cli.tools import executor as executor_module + + real_run = executor_module._run_command + + def _run(*args, **kwargs): + events.append("run") + return real_run(*args, **kwargs) + + monkeypatch.setattr(executor_module, "_run_command", _run) + with MockContext(): + ran = execute_python("1 + 1") + invalid = execute_python("def (") + + assert ran["success"] is True, ran + assert invalid["error"].startswith("Validation error"), invalid + assert events == ["run", "release", "release"] + + +def _late_writer(late, *, leave_group: bool = False) -> str: + """A command that starts a background process writing ``late`` 3 s later + (in a session of its own when ``leave_group``, as bubblewrap's sandbox + init is), then waits far longer than any timeout here.""" + import shlex + + writer = f"sleep 3; echo late > {shlex.quote(str(late))}" + start = f"setsid sh -c {shlex.quote(writer)}" if leave_group else f"({writer})" + return f"{start} & exec sleep 30" + + +def test_a_timeout_kills_every_process_the_command_started( + monkeypatch: pytest.MonkeyPatch, tmp_path +) -> None: + """The timeout kills the command's whole process group, not its shell + alone: what it started in the background (a process the code forked) + dies with it.""" + import time + + from agentic_cli.tools.execution_tools import execute_python + from tests.conftest import MockContext + + late = tmp_path / "late.txt" + _use_backend(monkeypatch, _Backend("seatbelt", command=lambda argv: _late_writer(late))) + with MockContext(): + result = execute_python("1 + 1", timeout_seconds=1) + + assert "timed out" in result["error"], result + time.sleep(3.5) + assert not late.exists() + + +@pytest.mark.skipif( + not sys.platform.startswith("linux") or shutil.which("setsid") is None, + reason="finds the processes that left the group through /proc", +) +def test_a_timeout_kills_a_child_that_left_the_process_group( + monkeypatch: pytest.MonkeyPatch, tmp_path +) -> None: + """bubblewrap's sandbox init runs in a session of its own + (``--new-session``), out of the command's process group: the timeout + finds it as the child of a process in the group, and kills it too.""" + import time + + from agentic_cli.tools.execution_tools import execute_python + from tests.conftest import MockContext + + late = tmp_path / "late.txt" + _use_backend(monkeypatch, _Backend( + "seatbelt", command=lambda argv: _late_writer(late, leave_group=True), + )) + with MockContext(): + result = execute_python("1 + 1", timeout_seconds=1) + + assert "timed out" in result["error"], result + time.sleep(3.5) + assert not late.exists() + + +def test_an_interrupted_run_kills_its_process_group( + monkeypatch: pytest.MonkeyPatch, tmp_path +) -> None: + """A KeyboardInterrupt while the run waits (in a caller on the main + thread: the CLI runs calls on a worker thread, which none reaches) kills + the whole group too, then goes through.""" + import time + + from agentic_cli.tools.execution_tools import execute_python + from tests.conftest import MockContext + + late = tmp_path / "late.txt" + _use_backend(monkeypatch, _Backend("seatbelt", command=lambda argv: _late_writer(late))) + + def interrupted(self, *args, **kwargs): + time.sleep(0.5) # the background process has started + raise KeyboardInterrupt + + monkeypatch.setattr(subprocess.Popen, "communicate", interrupted) + with MockContext(), pytest.raises(KeyboardInterrupt): + execute_python("1 + 1", timeout_seconds=10) + monkeypatch.undo() + + time.sleep(3.5) + assert not late.exists() + + +def test_a_run_that_times_out_keeps_its_hold(monkeypatch: pytest.MonkeyPatch) -> None: + """A run killed at its timeout may leave the sandbox's processes alive a + moment longer (they are killed asynchronously): what the wrap holds for + the run, bubblewrap's placeholders, is kept rather than released.""" + from agentic_cli.tools.execution_tools import execute_python + from tests.conftest import MockContext + + released: list[str] = [] + _use_backend(monkeypatch, _Backend("seatbelt", release=lambda: released.append("release"))) + with MockContext(): + result = execute_python("while True:\n pass", timeout_seconds=1) + + assert "timed out" in result["error"], result + assert released == [] + + +@pytest.mark.parametrize("command", ["kill -9 $$", "exit 143"], ids=["killed", "shell-reported-signal"]) +def test_a_run_killed_by_a_signal_keeps_its_hold( + monkeypatch: pytest.MonkeyPatch, command: str +) -> None: + """A signal from outside that kills the shell or bwrap's monitor returns + before the kernel has emptied the sandbox: a status that says a signal + ended the run (negative, or 128 and above from a shell) keeps the hold.""" + from agentic_cli.tools.execution_tools import execute_python + from tests.conftest import MockContext + + released: list[str] = [] + _use_backend(monkeypatch, _Backend( + "seatbelt", command=lambda argv: command, release=lambda: released.append("release"), + )) + with MockContext(): + result = execute_python("1 + 1") + + assert result["success"] is False, result + assert released == [] + + def test_a_sandboxed_run_gets_the_sandboxed_module_set(monkeypatch: pytest.MonkeyPatch) -> None: from agentic_cli.tools.execution_tools import execute_python from tests.conftest import MockContext diff --git a/tests/tools/test_os_sandbox.py b/tests/tools/test_os_sandbox.py index 7fac3398..b1934885 100644 --- a/tests/tools/test_os_sandbox.py +++ b/tests/tools/test_os_sandbox.py @@ -287,6 +287,13 @@ def test_sandbox_type(self): assert sandbox.sandbox_type == "none" +class TestOSSandboxResult: + def test_results_are_equal_by_what_they_run(self): + """``release`` is what to undo after the run, not part of the result's + value: two wraps of the same command compare equal.""" + assert OSSandboxResult("cmd", release=lambda: None) == OSSandboxResult("cmd") + + # --------------------------------------------------------------------------- # TestSafePythonExecutorIntegration # --------------------------------------------------------------------------- @@ -306,7 +313,7 @@ def test_no_policy_executes_as_before(self): mock_proc.stderr = "" mock_proc.returncode = 0 - with patch("agentic_cli.tools.executor.subprocess.run", return_value=mock_proc): + with patch("agentic_cli.tools.executor._run_command", return_value=mock_proc): result = executor.execute("print('hello')") assert result["success"] is True @@ -332,7 +339,7 @@ def test_enabled_policy_wraps_python_command(self): mock_proc.returncode = 0 with patch( - "agentic_cli.tools.executor.subprocess.run", return_value=mock_proc + "agentic_cli.tools.executor._run_command", return_value=mock_proc ) as mock_run, patch( "agentic_cli.tools.os_sandbox.get_os_sandbox", return_value=mock_os_sandbox, @@ -342,7 +349,7 @@ def test_enabled_policy_wraps_python_command(self): assert result["success"] is True # wrap_python_command was called mock_os_sandbox.wrap_python_command.assert_called_once() - # subprocess.run was called with shell=True (wrapped command) + # The command ran with shell=True (wrapped command) assert mock_run.call_args.kwargs.get("shell") is True def test_enabled_policy_wrap_failure_strict_fails_closed(self): @@ -360,7 +367,7 @@ def test_enabled_policy_wrap_failure_strict_fails_closed(self): ) with patch( - "agentic_cli.tools.executor.subprocess.run" + "agentic_cli.tools.executor._run_command" ) as mock_run, patch( "agentic_cli.tools.os_sandbox.get_os_sandbox", return_value=mock_os_sandbox, @@ -380,7 +387,7 @@ def test_enabled_policy_no_real_sandbox_strict_fails_closed(self): mock_os_sandbox.sandbox_type = "none" with patch( - "agentic_cli.tools.executor.subprocess.run" + "agentic_cli.tools.executor._run_command" ) as mock_run, patch( "agentic_cli.tools.os_sandbox.get_os_sandbox", return_value=mock_os_sandbox, @@ -411,7 +418,7 @@ def test_enabled_policy_no_real_sandbox_non_strict_falls_back(self): mock_proc.returncode = 0 with patch( - "agentic_cli.tools.executor.subprocess.run", return_value=mock_proc + "agentic_cli.tools.executor._run_command", return_value=mock_proc ) as mock_run, patch( "agentic_cli.tools.os_sandbox.get_os_sandbox", return_value=mock_os_sandbox, diff --git a/tests/tools/test_os_sandbox_protected.py b/tests/tools/test_os_sandbox_protected.py index 9a119119..6391af96 100644 --- a/tests/tools/test_os_sandbox_protected.py +++ b/tests/tools/test_os_sandbox_protected.py @@ -14,8 +14,10 @@ from __future__ import annotations +import os import subprocess import sys +import time from pathlib import Path import pytest @@ -196,14 +198,50 @@ def _sandbox(): def _run(code: str, cwd: Path, policy: OSSandboxPolicy) -> str: + """Run ``code`` in the sandbox as the executor runs a call.""" + from agentic_cli.tools.executor import _run_command + wrapped = _sandbox().wrap_python_command([sys.executable, "-c", code], cwd, policy) assert wrapped.success, wrapped.error - proc = subprocess.run( - wrapped.command, shell=True, cwd=cwd, capture_output=True, text=True, timeout=60 - ) + try: + proc = _run_command(wrapped.command, shell=True, input="", timeout=60, cwd=cwd) + finally: + wrapped.release() + # A sandbox that did not start ran no code: say why, rather than "". + assert proc.returncode == 0 or proc.stdout, proc.stderr return proc.stdout.strip() +def _processes() -> str: + """The shells, sandboxes and Pythons still running, for a failure to show + what survived.""" + columns = "pid,ppid,pgid,sid,stat,args" if sys.platform.startswith("linux") else ( + "pid,ppid,pgid,stat,args") + listing = subprocess.run(["ps", "-eo", columns], capture_output=True, text=True).stdout + lines = listing.splitlines() + keep = [line for line in lines[1:] if any(n in line for n in ("bwrap", "python", "sh "))] + return "still running:\n" + "\n".join(lines[:1] + keep) + + +def _bubblewrap_only() -> None: + if _sandbox().sandbox_type != "bubblewrap": + pytest.skip("needs bubblewrap's pid namespace") + + +#: Forks a process that closes its output and writes ``path`` ``delay`` +#: seconds later, then runs ``parent``. +LATE_WRITER = """ +import os, time +if os.fork() == 0: + for fd in (0, 1, 2): + os.close(fd) + time.sleep({delay}) + open({path!r}, "w").write("late") + os._exit(0) +{parent} +""" + + ATTEMPT = """ import sys try: @@ -235,10 +273,13 @@ def test_app_config_cannot_be_read(self, home, project): assert out == "blocked" def test_project_env_file_cannot_be_read(self, home, project): + """Seatbelt refuses the open; bubblewrap shows ``/dev/null`` in its + place, which reads empty. Either way the key never reaches the code.""" (project / ".env").write_text("NAME=value\n") - out = _run(ATTEMPT.format(action="open('.env').read()"), project, + out = _run(ATTEMPT.format(action="print(open('.env').read())"), project, OSSandboxPolicy(app_name=APP)) - assert out == "blocked" + assert out in ("blocked", "done") + assert "NAME=value" not in out def test_nothing_is_writable_when_started_from_home(self, home): out = _run(ATTEMPT.format(action="open('note.txt', 'w').write('x')"), home, @@ -246,6 +287,70 @@ def test_nothing_is_writable_when_started_from_home(self, home): assert out == "blocked" assert not (home / "note.txt").exists() + def test_a_missing_shell_start_up_file_cannot_be_created(self, home, project): + """The home holds none of the protected files: the sandbox still starts, + and the code cannot create one.""" + profile = home / ".bash_profile" + out = _run(ATTEMPT.format(action=f"open({str(profile)!r}, 'w').write('x')"), project, + OSSandboxPolicy(app_name=APP)) + assert out == "blocked" + assert not os.path.lexists(profile) + + def test_a_project_that_is_no_repository_cannot_be_made_one(self, home): + """``.git`` is missing in a folder the code may write: the code cannot + create it, and the run leaves nothing in its place.""" + plain = home / "plain" + plain.mkdir() + out = _run(ATTEMPT.format(action="__import__('os').makedirs('.git/hooks')"), plain, + OSSandboxPolicy(app_name=APP)) + assert out == "blocked" + assert os.listdir(plain) == [] + + def test_the_code_is_pid_1_of_its_sandbox(self, home, project): + """bubblewrap runs the code as its pid 1 (``--as-pid-1``): when it + exits, the kernel kills and reaps what it started before bwrap + returns, so a run that returned leaves no process behind.""" + _bubblewrap_only() + out = _run("import os; print(os.getpid())", project, OSSandboxPolicy(app_name=APP)) + assert out == "1" + + def test_what_the_code_forks_ends_with_it(self, home, project): + """In bubblewrap the sandbox ends with its pid 1, the code; on + seatbelt the fork stays in the run's process group, which the run's + end kills.""" + late = project / "late.txt" + out = _run(LATE_WRITER.format(path=str(late), delay=1, parent="print('forked')"), project, + OSSandboxPolicy(app_name=APP)) + assert out == "forked" + time.sleep(3) + assert not late.exists() + + @pytest.mark.parametrize("shell", ["as the executor runs it", "behind a shell that stays"]) + def test_a_timeout_stops_the_code_and_what_it_forked(self, home, project, shell): + """The executor's timeout kills the command's process group (the + shell, bubblewrap's monitor; on seatbelt the code and what it forked, + which stay in the group) and, on Linux, each child of the group that + left it: bubblewrap's sandbox init, in a session of its own + (``--new-session``), whose death ends every process in the sandbox. + The forked writer waits longer than the timeout, so a file means it + outlived it. Run as the executor runs the command (``sh -c`` may exec + bwrap) and behind a shell that must stay.""" + from agentic_cli.tools.executor import _run_command + + _sandbox() # skips without a backend + late = project / "late.txt" + code = LATE_WRITER.format(path=str(late), delay=5, parent="while True: pass") + wrapped = _sandbox().wrap_python_command([sys.executable, "-c", code], project, + OSSandboxPolicy(app_name=APP)) + assert wrapped.success, wrapped.error + command = wrapped.command if shell == "as the executor runs it" else ( + f"{wrapped.command}; exit $?") + with pytest.raises(subprocess.TimeoutExpired): + _run_command(command, shell=True, input="", timeout=2, cwd=project) + assert not late.exists(), _processes() + time.sleep(4) # past the writer's 5 s + assert not late.exists(), _processes() + def test_execute_python_runs_its_code_inside_the_sandbox(self, home, project, monkeypatch): """With the default settings the code runs inside the real sandbox, so a standard module that reaches the host (``json.codecs.open``) writes diff --git a/tests/workflow/test_memory_folders_protected.py b/tests/workflow/test_memory_folders_protected.py index 1d794b37..190e0a48 100644 --- a/tests/workflow/test_memory_folders_protected.py +++ b/tests/workflow/test_memory_folders_protected.py @@ -3,6 +3,7 @@ from __future__ import annotations import os +import shlex import shutil import stat import subprocess @@ -133,8 +134,9 @@ def test_executed_code_cannot_write_there_either(project): def test_a_folder_that_does_not_exist_is_not_made_a_mount_point(project, tmp_path): """A folder that cannot be made is still denied (seatbelt names a path, not - a mount) but is left out of the list bubblewrap binds, which would bind - /dev/null over it and can leave a stray file where the store makes its folder.""" + a mount) but is left out of the list bubblewrap binds, which covers a + missing path with a placeholder folder: none could be made there either, + and the run would be refused.""" (tmp_path / "blocker").write_text("a file, so nothing can be made below it") settings = BaseSettings(app_name="app", workspace_dir=tmp_path / "blocker" / "data") policy = _policy(settings) @@ -183,17 +185,22 @@ def test_the_rename_rule_follows_the_write_allows(self): def test_bubblewrap_binds_what_exists_and_needs_no_rename_rule(self, project): """A bind mount follows its directory when an ancestor is renamed, so a - write through the new name still meets the read-only mount.""" + write through the new name still meets the read-only mount. (The + memory folder, still empty, is held like a placeholder.)""" settings = _inside(project) policy = _policy(settings) - args = BubblewrapSandbox()._build_bwrap_args(project, policy) + first = BubblewrapSandbox().wrap_python_command(["python3"], project, policy) + args = shlex.split(first.command) root = (project / "data").resolve() binds = {(args[i + 1], args[i + 2]) for i in range(len(args)) if args[i] == "--ro-bind"} assert (str(root / "memory"), str(root / "memory")) in binds assert (str(root / "projects"), str(root / "projects")) in binds policy.deny_rename_paths.append(str(project / "data")) policy.deny_write_missing_paths.append(str(root / "never-made")) - assert BubblewrapSandbox()._build_bwrap_args(project, policy) == args + second = BubblewrapSandbox().wrap_python_command(["python3"], project, policy) + assert second.command == first.command + first.release() + second.release() @pytest.mark.skipif(