diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index a80eedc5..5ba9d46f 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -50,7 +50,7 @@ sign-off trailer on every commit in a PR. |---|---|---| | unit | `just test` | nothing | | race | `just race` | nothing | -| TUI model tests | `just test-pkg internal/tui` | nothing | +| TUI model tests | `just test-pkg tui` | nothing | | e2e | `just e2e` | KVM, network, ~15 minutes | A change to `internal/core`, `internal/sshx`, `internal/cloudinit`, diff --git a/README.md b/README.md index 66d154fd..468f6a80 100644 --- a/README.md +++ b/README.md @@ -1,143 +1,90 @@ # stoat -A terminal UI and CLI for running local QEMU VMs on Linux. No libvirt, no daemon, one binary. +Stoat is a CLI for running local QEMU virtual machines, built for people and +AI agents. Use its TUI for interactive work, JSON CLI output for automation, +or MCP tools from an agent. -## What it does +It manages disposable Alpine sessions and persistent Linux guests through +plain TOML files. One Go binary starts QEMU directly. -- Boots an Alpine live VM that comes up already networked and `ssh`-reachable: an apkovl overlay is baked into the boot, so there's no `setup-alpine` step to get in. -- Also runs Ubuntu, Debian, Fedora, and Arch cloud images, provisioned via cloud-init on first boot. -- Persistent disk VMs for anything else: install once with the guest's own installer, then boot straight to it. -- Ships a few ready-made recipes (XFCE, Docker, dev tools, Tailscale) to run post-boot over ssh or bake into a cloud-init seed. `stoat recipe new` scaffolds your own. -- QEMU processes are tracked by pidfile, not supervised: `stoat` can exit and the VM keeps running. -- A TUI for interactive use, and a scriptable CLI (`ls`, `up`, `down`, `ssh`, `apply`, `rm`, `recipe`, `guest`, `screenshot`, `logs`, `doctor`) covering the same operations for scripts and automation. Every command takes `--json` for one object per line, errors included. +[Get started](docs/getting-started/installation.md) · +[Documentation](docs/README.md) · +[CLI reference](docs/reference/cli.md) · +[Releases](https://github.com/NovusEdge/stoat/releases) -## What it looks like +![Stoat lists two running VMs and one stopped VM.](assets/tui-list.png) -The list screen, with one running VM, one stopped, and one whose `vm.toml` failed to parse (captured with `tmux capture-pane` against a throwaway `STOAT_HOME`): +## Install and check -``` - ███████╗████████╗ ██████╗ █████╗ ████████╗ - ██╔════╝╚══██╔══╝██╔═══██╗██╔══██╗╚══██╔══╝ - ███████╗ ██║ ██║ ██║███████║ ██║ - ╚════██║ ██║ ██║ ██║██╔══██║ ██║ - ███████║ ██║ ╚██████╔╝██║ ██║ ██║ - ╚══════╝ ╚═╝ ╚═════╝ ╚═╝ ╚═╝ ╚═╝ - - ╭────────────────────────────────────────────────────────────────╮ - │ │ - │ ❯ ○ alpine-live live 2048M 2c - │ - │ │ - │ ○ ubuntu-dev cloud 4096M 4c - │ - │ │ - │ ✗ old-vm broken: toml: line 1: expected '.' or │ - │ '=', but got 'i' instead │ - │ │ - ╰────────────────────────────────────────────────────────────────╯ - -↵ start/stop • →/l details • s ssh • p provision • / search • n new • r recipes • d delete • q quit • ? help -``` - -The same VMs from `stoat ls`: - -``` -$ stoat ls -NAME MODE STATE CPUS RAM SSH -alpine-live live stopped 2 2048 2200 -ubuntu-dev cloud stopped 4 4096 2201 -old-vm - broken - - - toml: line 1: expected '.' or, but got 'i' instead -``` - -## Requirements - -- Linux with KVM: `/dev/kvm` readable and writable by your user (member of the `kvm` group). -- `qemu-system-x86_64` and `qemu-img`. GTK+OpenGL support is needed only for the one VM that opens a window (`-display gtk,gl=on`, a disk VM mid-install); with no graphical session on the host, that console falls back to VNC and the VM still installs. -- `ssh`. -- `xorriso` (package `libisoburn`), only if you use a cloud-init image (Ubuntu/Debian/Fedora/Arch). Not needed for Alpine live or disk VMs. -- Go 1.26 to build from source (the version pinned in `go.mod`). - -## Install +Stoat runs on Linux with KVM, QEMU, OpenSSH, and `xorriso` for cloud images. +Building from source needs Go 1.26 or newer, Git, and Just. ```sh +git clone https://github.com/NovusEdge/stoat.git +cd stoat just setup ``` -Builds stoat, installs it to `~/.local/bin` (override with `$PREFIX`), offers to add that directory to your `PATH`, and reports which of `qemu-system-x86_64`, `qemu-img`, `ssh`, `xorriso`, and `/dev/kvm` still need attention, with the exact install command for your distro. - -For a non-interactive install (CI, scripts): +`just setup` builds Stoat, installs it under `~/.local/bin`, checks host +dependencies, and offers to add that directory to `PATH`. Open a new shell if +your shell configuration changed, then run: ```sh -just build && just install # or: make build && make install +stoat doctor ``` -`install` puts the binary at `$PREFIX/stoat` (default `~/.local/bin/stoat`); make sure that's on your `PATH`. +See [installation](docs/getting-started/installation.md) for host packages, +release archives, Nix, and KVM permission details. -Release tarballs and the Nix flake are covered in [docs/getting-started/installation.md](docs/getting-started/installation.md). +## Start a Debian VM -## Quick start - -``` -stoat +```sh +stoat pull debian-13 +stoat create dev --image debian-13 --ram 2048 --cpus 2 +stoat up dev +stoat wait dev --until reachable --timeout 2m +stoat exec dev -- cat /etc/debian_version +stoat down dev ``` -1. Press `n` for a new VM. -2. Pick an image and press SPACE to download it (not enter: enter tries to create the VM and tells you to download first). -3. Press ENTER to create the VM. -4. Back on the list, press ENTER to start it. -5. If you checked any recipes, stoat waits for sshd and then offers to run - them: `test1 is up, run xfce now? y/N`. Press `y`, or decline and press - `p` whenever you like. -6. Press `s` to ssh in. - -See [docs/getting-started/first-vm.md](docs/getting-started/first-vm.md) for the walkthrough with screenshots and troubleshooting. - -## Commands - -| Command | Purpose | -|---|---| -| `stoat` | launch the interactive TUI | -| `stoat ls` | list VMs, one line per VM | -| `stoat up ` | start a VM | -| `stoat down ` | stop a VM (graceful) | -| `stoat ssh ` | ssh into a VM, replacing this process | -| `stoat apply [--dry-run]` | run the VM's recipes, streaming output to stdout | -| `stoat rm [-y]` | delete a VM; refuses while running, confirms unless `-y` | -| `stoat screenshot [-o path]` | write the VM's screen to a PNG | -| `stoat recipe list` | list installed recipes and where they live | -| `stoat recipe new [--os alpine] [--backend cloudinit]` | scaffold a recipe in the recipes directory | -| `stoat guest ls` | list the guest OS definitions stoat knows | -| `stoat guest show ` | print one guest definition | -| `stoat logs [-n N]` | tail the stoat log (default 50 lines) | -| `stoat doctor` | check host prerequisites | -| `stoat version` | print the stoat version | - -Global flags: `-q`/`--quiet`/`--no-interactive` suppress progress chatter (results and errors still print). Exit codes: 0 success, 1 runtime failure, 2 usage error. Full reference: [docs/reference/cli.md](docs/reference/cli.md); TUI keys: [docs/reference/tui.md](docs/reference/tui.md). - -## How it works - -Every VM is one of three modes: - -- **live**: diskless, boots straight from an Alpine ISO, discards all state on stop. Provisioned by an **apkovl** overlay built fresh at every start. -- **disk**: a qcow2 that survives restarts. You run the guest's own installer once, then flip it to "installed". Provisioned by pushing recipes over **ssh**. -- **cloud**: a CoW overlay over a shared Ubuntu/Debian/Fedora/Arch base image. Provisioned by a **cloud-init** seed baked in once, at first boot only. +`create` writes the VM without starting it. `wait --until reachable` confirms +that SSH answers. `down` retains the disk; `stoat rm dev` deletes a stopped VM +after confirmation. [Your first VM](docs/getting-started/first-vm.md) covers +recipes, display access, and expected results. -Each VM is a directory under `~/.stoat` (override with `$STOAT_HOME`) holding a hand-editable `vm.toml` plus whatever state its mode keeps, nothing else is shared between VMs. See [docs/concepts/modes-and-backends.md](docs/concepts/modes-and-backends.md) and [docs/concepts/data-root.md](docs/concepts/data-root.md). +## Use the TUI -Recipes (the scripts and cloud-init fragments that install XFCE, Docker, etc.) are covered in [docs/recipes/overview.md](docs/recipes/overview.md), including how to write your own. A recipe script runs with a prelude of guest-neutral verbs (`stoat_pkg_install`, `stoat_svc_enable`, and the rest) rendered from the guest's own definition, so one script can serve several distros. +Run `stoat` with no subcommand. Press `n` to choose an image, **Space** to +download it, and **Enter** to create it. Press **Enter** on the list to +start or stop the selected VM, `s` to open SSH, `p` to apply recipes, and `r` +to edit the recipes directory in `$EDITOR`. Press `?` for help and `q` to quit. -Guest OS facts live in one TOML file per OS, bundled in the binary and overridable from `~/.stoat/guests/`. Adding an OS is a file, not a code change: see [docs/reference/guest.md](docs/reference/guest.md). +For repository-managed VMs, declare `[vms.]` in `stoat.toml` and commit +the file. `stoat up`, `status`, and `down` then reconcile and operate on the +declared VMs. See the [project file reference](docs/reference/project-file.md). -## Status +## Recipes, MCP, and scripts -Pre-1.0 and single-user: stoat assumes it's the only thing managing its `~/.stoat`, and offers no sandboxing beyond what QEMU/KVM already give a guest. The Alpine live path is the most exercised mode; the cloud-init backends and disk-mode installs are newer. `vm.toml`'s shape and the CLI's flags may still change before 1.0. +- [Recipe overview](docs/recipes/overview.md) explains targeting, parameters, + secrets, outputs, health checks, and persistence. +- [Sharing recipes](docs/recipes/sharing.md) covers remote refs, two scopes, + lock, sync, search, update, and removal. +- [MCP reference](docs/reference/mcp.md) covers client setup, transports, + project tools, and agent access levels. +- [JSON output](docs/reference/json.md) documents the CLI contract and DTOs. +- [Troubleshooting](docs/troubleshooting.md) covers boot, SSH, display, and + provisioning failures. -## Contributing +## See it running -`CONTRIBUTING.md` covers setup, the branch and PR flow, commit grammar, the -gates CI runs, and the test tiers. New recipes go in `~/.stoat/recipes/`; -the bundled set is closed. +The recording shows Stoat's TUI list and detail screens followed by an XFCE +desktop in a QEMU window. -## License +[Watch the MP4](assets/demo.mp4) · [Capture details](assets/README.md) -AGPL-3.0-or-later. Copyright (c) 2026 Aliasgar Khimani (NovusEdge). +Stoat stores VM configuration and state under `~/.stoat`; set `STOAT_HOME` to +use another data root. See [modes and backends](docs/concepts/modes-and-backends.md) +and [the data root](docs/concepts/data-root.md) for storage behavior. -Free to clone, run, and modify, including by automated agents and harnesses. Any distributed or network-hosted derivative must publish its source under the same license, so stoat cannot be forked into a proprietary product. +Stoat is pre-1.0. [Contributing](CONTRIBUTING.md) covers development setup +and checks. It is licensed under [AGPL-3.0-or-later](LICENSE). diff --git a/assets/README.md b/assets/README.md new file mode 100644 index 00000000..4bead528 --- /dev/null +++ b/assets/README.md @@ -0,0 +1,96 @@ +# Screenshots and recording + +These captures show Stoat built from commit `7f985bc` on Linux x86-64 with +QEMU 11.1.1. They were captured on 2026-09-06 in Europe/Helsinki time. +The guests use a separate `STOAT_HOME` and cached Alpine 3.24.1, Debian 13, +and Ubuntu 24.04 images. + +| File | Content | +|---|---| +| [tui-list.png](tui-list.png) | Main TUI in Konsole; two running VMs and one stopped VM | +| [tui-details.png](tui-details.png) | Alpine VM details and the completed XFCE apply log | +| [tui-create.png](tui-create.png) | New-VM form with a cached Alpine image | +| [qemu-xfce.png](qemu-xfce.png) | Actual QEMU window running Alpine XFCE | +| [alpine-xfce.png](alpine-xfce.png) | Earlier guest display captured through QMP with `stoat screenshot` | +| [demo.mp4](demo.mp4) | 23-second H.264 screen recording, 1280 × 1000, 15 fps, no audio | +| [demo.gif](demo.gif) | Animated copy of the screen recording, 10 fps | + +## Capture method + +The TUI images and video capture pixels from a Konsole window on KDE. +Konsole used its default color scheme, a 12-point monospace font, and an +opaque background. `NO_COLOR` was unset in the capture process. The main +image uses a 1280 × 768 window; the details and form use 1280 × 1000 so +the full content and key hints fit. + +FFmpeg captured the Konsole and QEMU Xwayland windows with `x11grab` and +`-draw_mouse 0`. The TUI hides its text cursor in the list and details views. +The guest terminal's text cursor was hidden for its demonstration command. +The QEMU clip has padding above and below to match the terminal clip's height. +The PNG files are window captures; `alpine-xfce.png` is a direct QMP capture. + +The video spends five seconds on the VM list, nine on VM details, and nine +on QEMU. The guest terminal opens during the QEMU segment and prints the +Alpine and kernel versions. The details show `xfce (applied unknown)`: the +recipe completed, and the bundled XFCE recipe has no health check. + +## Reproduce the guests + +Install the [host prerequisites](../docs/getting-started/installation.md), +then create a separate data root: + +```sh +export STOAT_HOME="$(mktemp -d "${TMPDIR:-/tmp}/stoat-docs.XXXXXX")" +stoat pull alpine-standard +stoat pull debian-13 +stoat pull ubuntu-24.04 + +stoat create alpine-desktop --image alpine-standard --ram 2048 --cpus 2 --recipes xfce +stoat create debian-dev --image debian-13 --ram 2048 --cpus 2 +stoat create ubuntu-lab --image ubuntu-24.04 --ram 2048 --cpus 2 + +STOAT_GRAPHICAL=1 stoat up alpine-desktop --json +STOAT_GRAPHICAL=0 stoat up debian-dev +stoat wait debian-dev --until reachable --timeout 2m +stoat exec debian-dev -- cat /etc/debian_version +``` + +Alpine opens a QEMU window and applies XFCE on its first start. Ubuntu stays +stopped. For the window capture, `xrandr` was installed in Alpine and the +display was set to 1280 × 768. The guest terminal ran `cat /etc/alpine-release` +and `uname -sr`. Future image and package versions can change the output. + +Open Stoat in Konsole, select `alpine-desktop`, and press `l` for details. +Press **Escape** to return, then `n` to capture the creation form. +**Escape** closes the form without creating another VM. + +```sh +env -u NO_COLOR TERM=xterm-256color COLORTERM=truecolor \ + QT_QPA_PLATFORM=xcb konsole -e stoat +``` + +On an X11/Xwayland window, find its ID with `xprop -root _NET_CLIENT_LIST` +and verify its title with `xprop -id _NET_WM_NAME`. Record that window: + +```sh +ffmpeg -f x11grab -draw_mouse 0 -framerate 15 -window_id "$WINDOW_ID" \ + -i "$DISPLAY" -c:v libx264 -crf 20 -pix_fmt yuv420p \ + -movflags +faststart capture.mp4 +``` + +Set `WINDOW_ID` to the chosen window's numeric ID. Use a new output path +for each capture. Pure Wayland windows require a compositor-supported recorder. + +## Stop the guests + +```sh +stoat down alpine-desktop +stoat wait alpine-desktop --until stopped --timeout 30s +stoat down debian-dev +stoat wait debian-dev --until stopped --timeout 30s +``` + +Stopping Alpine discards its live desktop. See the +[live recipe restart limitation](../docs/troubleshooting.md#a-live-vm-lost-everything-after-a-reboot) +before reusing a live VM. Debian's disk and the VM definitions remain in +the capture data root. diff --git a/assets/alpine-xfce.png b/assets/alpine-xfce.png new file mode 100644 index 00000000..40e6e6d0 Binary files /dev/null and b/assets/alpine-xfce.png differ diff --git a/assets/demo.gif b/assets/demo.gif new file mode 100644 index 00000000..81e81072 Binary files /dev/null and b/assets/demo.gif differ diff --git a/assets/demo.mp4 b/assets/demo.mp4 new file mode 100644 index 00000000..a68f51a5 Binary files /dev/null and b/assets/demo.mp4 differ diff --git a/assets/qemu-xfce.png b/assets/qemu-xfce.png new file mode 100644 index 00000000..d7893de7 Binary files /dev/null and b/assets/qemu-xfce.png differ diff --git a/assets/tui-create.png b/assets/tui-create.png new file mode 100644 index 00000000..bfb68f5f Binary files /dev/null and b/assets/tui-create.png differ diff --git a/assets/tui-details.png b/assets/tui-details.png new file mode 100644 index 00000000..4a908873 Binary files /dev/null and b/assets/tui-details.png differ diff --git a/assets/tui-list.png b/assets/tui-list.png new file mode 100644 index 00000000..83576bc3 Binary files /dev/null and b/assets/tui-list.png differ diff --git a/docs/README.md b/docs/README.md index 6ca4cdb6..fef0b828 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,4 +1,4 @@ -# stoat +# stoat documentation stoat is a terminal UI for running local QEMU virtual machines. It is Alpine-first, and it manages each VM through a small `vm.toml` file instead of @@ -10,15 +10,21 @@ stoat needs no libvirt, no background daemon, and no database, just a single Go binary that shells out to `qemu-system-x86_64`, `qemu-img`, and `ssh`, and a data root of plain files under `~/.stoat` (or `$STOAT_HOME`). -New here? Start with [Installation](getting-started/installation.md), then -[Your first VM](getting-started/first-vm.md). +Use the [installation guide](getting-started/installation.md) to prepare the +host, then follow [Your first VM](getting-started/first-vm.md). Alpine live is +the shortest path to a working shell. Alpine disk VMs install unattended and +retain their disk; cloud VMs use a first-boot cloud-init seed. -## Status +For repeatable development environments, commit a `stoat.toml` and its +`stoat.lock`, then use the [project workflow](guides/project-workflow.md). +For an agent client, use the [MCP workflow](guides/mcp-workflow.md). -stoat is pre-1.0 and single-user: it assumes it's the only thing managing its -data root, and there's no remote access or multi-tenant story. The Alpine -live-boot path (`apkovl` provisioning, no install to disk) is the most -exercised mode, so that's what to reach for first. Cloud-image provisioning -(`cloud-init` seeds) and installed disk VMs work but have seen less mileage. -Expect rough edges, and check [Troubleshooting](troubleshooting.md) if -something doesn't behave as documented. +The [concepts](SUMMARY.md#concepts) explain storage, access, modes, +networking, project shares, and provisioning. The [reference](SUMMARY.md) +contains the CLI, JSON, TUI, guest, project, and recipe details. + +stoat is pre-1.0 and single-user: it assumes it is the only process managing +its data root and does not manage remote VMs or provide multi-tenant +isolation. Cloud-image and installed-disk paths are implemented, but Alpine +live has received the most exercise. Start with +[Troubleshooting](troubleshooting.md) when a command reports an error. diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md index 1f60aaf4..69f34283 100644 --- a/docs/SUMMARY.md +++ b/docs/SUMMARY.md @@ -7,6 +7,11 @@ * [Installation](getting-started/installation.md) * [Your first VM](getting-started/first-vm.md) +## Workflows + +* [Project workflow](guides/project-workflow.md) +* [MCP workflow](guides/mcp-workflow.md) + ## Concepts * [Modes and backends](concepts/modes-and-backends.md) diff --git a/docs/concepts/access-and-auth.md b/docs/concepts/access-and-auth.md index 68e273bd..9f0ca34a 100644 --- a/docs/concepts/access-and-auth.md +++ b/docs/concepts/access-and-auth.md @@ -1,9 +1,10 @@ # Access and auth Every VM stoat manages is reached over SSH, forwarded to a loopback port on -the host. There is no console-based login flow for day-to-day use, no -password, and no separate stoat-level credential system. It's all built on -one SSH keypair stoat generates for itself. +the host. SSH is the day-to-day credential path; cloud images also receive a +password for their graphical console. There is no separate stoat-level +credential system. SSH access is built on one keypair stoat generates for +itself. ## The client keypair @@ -59,7 +60,7 @@ per image when a VM is created from the catalog: | Image | Backend | SSH user | |---|---|---| | `alpine-standard`, `alpine-virt` | `apkovl` | `root` | -| `ubuntu-24.04`, `debian-13`, `fedora-cloud`, `arch-cloud` | `cloudinit` | `stoat` | +| `ubuntu-24.04`, `debian-13`, `fedora-cloud`, `arch-cloud`, `alpine-cloud` | `cloudinit` | `stoat` | The `stoat` user is used for the cloud images specifically because that's the user stoat's own cloud-init seed creates and keys, not each distro's usual @@ -159,3 +160,27 @@ whatever it was created with. The detail screen says so plainly: To fix one, either recreate it, or set `console_password` in its `vm.toml` (`E` on the detail screen) and delete its `disk.qcow2` so the overlay and seed are rebuilt on the next start, which discards everything in that VM. + +## MCP agent access + +The `agent_access` field controls which operations an MCP client may perform +for a VM. The levels are cumulative: + +| Level | Additional operations allowed | +|---|---| +| `none` | host-side operations only: status, start, stop, snapshots, logs, forwards, and updates | +| `observe` | read guest files, directories, metadata, processes, service status, and logs | +| `manage` | write files, copy files, install packages, manage services, add users, and apply recipes | +| `exec` | run foreground or background guest commands and manage their jobs | + +New VMs default to `manage`. Set the level with `stoat create --agent-access` +or the project file's `agent_access` field. The CLI and TUI can raise or lower +the level. The MCP `update` tool can lower it but cannot raise it, so a human +must explicitly grant broader access. The older `allow_exec` field remains +understood for existing VM files; `true` maps to `exec` and `false` maps to +`manage`. + +MCP access is an authorization boundary for the server. It does not change +what a person can do with `stoat ssh` or `stoat exec` from the CLI. See the +[MCP workflow](../guides/mcp-workflow.md) for client setup and transport +details. diff --git a/docs/concepts/data-root.md b/docs/concepts/data-root.md index 63df1fe9..3d42aa7e 100644 --- a/docs/concepts/data-root.md +++ b/docs/concepts/data-root.md @@ -23,7 +23,8 @@ than an error. ├── guest_host_ed25519_key # stable sshd host key baked into live VMs ├── guest_host_ed25519_key.pub ├── isos/ # downloaded ISOs and cloud images -├── recipes/ # installed recipe scripts/fragments +├── recipes/ # global and bundled recipe scripts/fragments +├── stoat.lock # global recipe pins, when used outside a project ├── logs/ │ └── stoat.log # one shared log for the whole tool └── / # one directory per VM @@ -40,6 +41,12 @@ than an error. └── meta-data ``` +Project recipe state is stored beside the repository instead of in the data +root. `stoat.lock` contains project pins and `.stoat/recipes/` contains the +project recipe cache. `.stoat/secrets.toml` contains project recipe secrets and +must remain mode `0600`. `stoat init` adds `.stoat/` to `.gitignore` in a Git +checkout. A project cache and its secrets are separate from the global cache. + Two facts about that tree: - `isos/` holds both plain ISOs (Alpine) and downloaded cloud images @@ -48,8 +55,8 @@ Two facts about that tree: image itself is never copied per-VM, only referenced. - `ovl/` is reused for two unrelated purposes depending on mode: the Alpine overlay tarball for `live` VMs, or the cloud-init seed for `cloud` VMs. A - `disk` VM has no `ovl/` contents at all: there's nothing to inject before - you've installed the guest OS yourself. + `disk` VM has no `ovl/` contents until its first start builds the Alpine + installer overlay; a non-Alpine BYO disk has no injected installer. - `qemu.pid` and `monitor.sock` only exist while (or after) a VM has run at least once; they're not created at `vm.toml` save time. - The SSH keypair and the guest host key are **not** per-VM: they live at @@ -70,12 +77,21 @@ Each VM directory holds one `vm.toml`. Every field: | `cpus` | int | Virtual CPU count | | `disk` | string | Disk size, e.g. `"8G"` (`disk` mode only) | | `installed` | bool | `disk` mode only. Flips the QEMU boot order: `false` keeps the installer ISO attached and boot-forced on every start; `true` boots straight off `disk.qcow2` | -| `share` | string | Host directory exposed to the guest as `/mnt/host`; empty means no share | +| `share` | string | Legacy host directory exposed read-only to the guest as `/mnt/host`; empty means no share | | `sshport` | int | The host-side loopback port forwarded to the guest's port 22 | -| `recipes` | []string | Filenames (from `recipes/`) selected for this VM | +| `recipes` | []string | Recipe names selected for this VM and resolved through project, global, local, or bundled scopes | | `backend` | string | `"apkovl"`, `"cloudinit"`, or `"ssh"` (recorded at creation time, informational only afterward) | | `base` | string | Absolute path to the shared base image an overlay is created from (`cloud` mode only) | | `sshuser` | string | The account used for SSH access/provisioning; empty means `root` (never written explicitly for that default) | +| `params` | table | Non-secret recipe parameters, grouped as `params..` | +| `display` | string | Screen preference: empty/`auto`, `window`, or `vnc` | +| `forwards` | array | Declared host-to-guest TCP forwards | +| `console_password` | string | Graphical console password, primarily for cloud VMs; `random` is resolved when created | +| `allow_exec` | bool | Legacy per-VM permission for guest command and copy operations; new files use `agent_access` | +| `agent_access` | string | MCP access level: `none`, `observe`, `manage`, or `exec`; defaults to `manage` | +| `applied` | table | Recipe versions and health values written by stoat; do not edit | +| `project` | string | Absolute directory of the declaring `stoat.toml`; empty for a global VM | +| `shares` | array | Project directories exported under `/work`; stoat writes resolved paths and mount tags | ## What's safe to hand-edit @@ -83,8 +99,8 @@ Each VM directory holds one `vm.toml`. Every field: while the VM is stopped: stoat re-reads it fresh every time, there's no cache to invalidate. Some fields are safer to edit than others: -- **Safe-ish**: `ram`, `cpus`, `share`, `recipes` (as long as the filenames - still exist under `recipes/`), `sshuser`. +- **Safe-ish**: `ram`, `cpus`, `share`, `recipes` (as long as the names still + resolve in the active recipe scope), `sshuser`. - **Edit with care**: `sshport` (if you pick one another VM already has, both will try to bind it), `disk` (shrinking it doesn't shrink the underlying qcow2; you'd need a manual `qemu-img resize` and it can destroy data), @@ -120,5 +136,8 @@ for, a collision that's exactly how this safeguard came to exist. directory, never a shared ISO or cloud image other VMs might still be using. `recipes/` starts out populated with stoat's bundled recipes the first time -it runs, but that install step never overwrites a file that's already -there, so local edits to a recipe survive a stoat upgrade. +it runs, but that install step never overwrites a file that's already there, +so local edits to a recipe survive a stoat upgrade. A project cache under +`.stoat/recipes/` takes precedence over a global remote recipe with the same +name; project, global, local, and bundled entries follow recipe scope +resolution. diff --git a/docs/concepts/modes-and-backends.md b/docs/concepts/modes-and-backends.md index 6f3bfb2b..a9ed1f2f 100644 --- a/docs/concepts/modes-and-backends.md +++ b/docs/concepts/modes-and-backends.md @@ -8,12 +8,15 @@ related but not the same thing: happens to the guest's storage. It's what stoat's QEMU argument builder branches on when constructing the `qemu-system-x86_64` command line, and what its start logic checks before launching a VM. -- **Backend** (`vm.toml`'s `backend` field) records how the create form - picked recipes and provisioning at creation time. Once the VM exists, - nothing at runtime dispatches on this field: it's informational. +- **Backend** (`vm.toml`'s `backend` field) identifies how stoat prepares the + VM and provisions its selected recipes. It is chosen at creation time and + remains part of the VM's runtime configuration; edit it only as part of a + deliberate migration. -**`disk` mode gives you no SSH access until you install an OS yourself and -tell stoat so.** See the section below. +**`disk` mode's first boot depends on the image.** Alpine disk VMs install +unattended; a bring-your-own image needs its own console install and SSH key +setup. Stoat applies recipes only after the installed guest is ready. See the +section below. ## The three modes @@ -43,40 +46,29 @@ trade for SSH working immediately with zero setup. ### disk -A `disk` VM has a real, persistent `disk.qcow2`. But stoat does not install -anything onto it for you. The first time you start a `disk` VM, it boots the -ISO you attached, the same as a bare-metal install would, and you install -the guest OS yourself, interactively, in the QEMU console window. - -Until you do that, there is no operating system on the disk, which means -there is no sshd running and no key installed anywhere. **Provisioning -cannot work yet, because there is nothing on the other end of the SSH -connection.** Pressing the provision key against a fresh `disk` VM doesn't -time out mysteriously: stoat's `p` handler checks this up front and tells -you so directly, rather than waiting the full SSH-connect timeout to fail -with a generic "not reachable." - -Two things need to happen before a `disk` VM becomes provisionable: - -1. **Install the OS at the console**, then mark stoat's SSH keypair - (`~/.stoat/id_stoat.pub`) as an authorized key inside the guest yourself, - for whichever user `vm.toml`'s `sshuser` names (root, if unset); nothing - in stoat injects it automatically for `disk` mode the way `apkovl.Build` - does for `live`. -2. **Mark the VM installed**: press `i` on the VM's detail screen. This - flips the `installed` field in `vm.toml` and, from then on, changes the - QEMU boot order: with `installed = false`, the ISO stays attached and - forced first on *every* boot (so you can always get back to the - installer); once `installed = true`, the ISO is no longer attached at all - and the VM boots straight off `disk.qcow2`. - -`installed` only applies to `disk` mode; it's a no-op field for `live` and -`cloud` VMs. +A `disk` VM has a real, persistent `disk.qcow2`. Alpine disk VMs install +themselves on their first start. Stoat generates a `setup-alpine` answer file, +bakes it into the boot overlay, waits for the installer to power off, and +starts the VM again from the installed disk. The CLI waits up to 15 minutes +for this sequence. A failed install leaves the installer running so you can +inspect the console and logs. + +Until the Alpine install completes, the SSH service belongs to the temporary +installer environment. The TUI and CLI refuse to apply recipes to it. After +the next start sees enough data on the qcow2 disk, stoat sets `installed = +true` and stops attaching the ISO. The `i` key on the detail screen can still +toggle this field when automatic detection is wrong. + +A non-Alpine BYO ISO does not have Stoat's Alpine answer file. Install that OS +at its console, add `~/.stoat/id_stoat.pub` to the account named by +`vm.toml`'s `sshuser`, then stop and start the VM. Stoat can provision it only +after the OS and SSH access are ready. ### cloud A `cloud` VM starts from a downloaded cloud image (Ubuntu, Debian, Fedora, -Arch; see the image catalog) rather than an installer ISO. stoat creates a +Arch, or Alpine cloud; see the image catalog) rather than an installer ISO. +stoat creates a copy-on-write overlay backed by that shared base image, so the multi-hundred -megabyte download only happens once no matter how many `cloud` VMs you spin up from it. @@ -85,15 +77,16 @@ Alongside the overlay, stoat builds a small NoCloud cloud-init seed ISO (volume label `CIDATA`) containing: - a `stoat` user, password-less sudo, and stoat's public key -- any cloud-flavored recipes you selected, merged into the seed's - `packages:`/`runcmd:` lists +- the selected recipes' cloud-init fragments and script bodies, as separate + archive documents -Cloud-init applies all of this **at first boot only**: there is no ongoing -SSH-based provisioning step for `cloud` VMs the way there is for `live`/`disk`. -Pressing the provision key on a `cloud` VM is a deliberate no-op: it tells you -recipes already ran at first boot, and that changing them means recreating -the VM (the seed isn't rebuilt on later starts, since by then the overlay -holds real guest state you don't want thrown away). +Cloud-init applies the seed **at first boot only**. The seed carries selected +recipe fragments and scripts as cloud-init archive documents. Stoat may run a +post-boot apply pass to discover and record those results. The TUI's provision +key and `stoat apply` still use the normal recipe run policy, so a changed +script or a recipe with pending work can run over SSH later. Stoat does not +rebuild the seed on later starts. Changing the recipe list requires recreating +the VM so the new list is present in the seed. The host seed artifacts remain in the VM directory for inspection and later diagnosis. Stoat creates seed directories with mode `0700` and seed files with @@ -106,9 +99,9 @@ detach those artifacts after boot. |---|---|---|---| | Storage | none, RAM only | `disk.qcow2`, persistent | `disk.qcow2`, CoW overlay over a shared base image | | Survives reboot | no | yes | yes | -| SSH on first boot | yes, automatically | **no**, until installed | yes, via cloud-init (first boot only) | -| Who installs the OS | nobody, it's the live ISO | you, at the console | nobody, image is prebuilt | -| Recipes applied | over SSH, on demand (`p`) | over SSH, on demand (`p`), only once `installed` | baked into the cloud-init seed at first boot | +| SSH on first boot | yes, automatically | Alpine: after unattended install; BYO: after your install | yes, via cloud-init (first boot only) | +| Who installs the OS | nobody, it's the live ISO | Alpine: stoat; BYO: you, at the console | nobody, image is prebuilt | +| Recipes applied | over SSH, on demand (`p`) | over SSH, on demand (`p`), only once `installed` | cloud-init fragments at first boot; pending or changed work over SSH | | Rebuilt on every start | yes (the apkovl) | no | no (overlay created once) | | Typical backend | `apkovl` | `apkovl` (Alpine, undeployed) or `ssh` (any other ISO) | `cloudinit` | @@ -121,8 +114,8 @@ guess the right mode: - **`apkovl`**: Alpine only. Produces the `live`-mode overlay described above. An Alpine image can be run either as `live` or as `disk` mode (you choose at creation); either way, the backend is recorded as `apkovl`. -- **`cloudinit`**: any recognized cloud image (Ubuntu, Debian, Fedora, - Arch). Always resolves to `cloud` mode; there's no other option. +- **`cloudinit`**: any recognized cloud image (Ubuntu, Debian, Fedora, Arch, + or Alpine cloud). Always resolves to `cloud` mode; there's no other option. - **`ssh`**: the fallback for a bring-your-own image stoat doesn't recognize. Always resolves to `disk` mode: an unrecognized ISO is assumed to need a real install, followed by manual/SSH-based provisioning, since diff --git a/docs/concepts/networking-and-sharing.md b/docs/concepts/networking-and-sharing.md index 6b36fe95..8039d477 100644 --- a/docs/concepts/networking-and-sharing.md +++ b/docs/concepts/networking-and-sharing.md @@ -36,12 +36,12 @@ broken VM was using is very likely still committed to that VM's disk image, so treating "unparseable" as "claims nothing" would silently hand the same port to a second VM, exactly the collision this fallback exists to avoid. -## The `share` field +## The legacy `share` field Setting `share` on a VM exposes a host directory to the guest over 9p, as -`/mnt/host`. QEMU is passed `security_model=none` for this: the only -unprivileged option: `passthrough` needs root, and `mapped-xattr` needs host -filesystem xattr support that isn't guaranteed to be there. +`/mnt/host`. Stoat exports this directory read-only. The guest can remount it +with `rw`, but QEMU still rejects writes. Stoat uses `security_model=mapped-xattr` +so guest-created symlinks cannot escape into the host filesystem. Whether the share actually gets *mounted* inside the guest depends on mode: @@ -55,12 +55,25 @@ Whether the share actually gets *mounted* inside the guest depends on mode: ```sh mkdir -p /mnt/host - mount -t 9p -o trans=virtio,version=9p2000.L,rw host /mnt/host + mount -t 9p -o trans=virtio,version=9p2000.L,ro host /mnt/host ``` (add that to `/etc/fstab` yourself if you want it to persist across reboots on a `disk` VM). +## Project shares + +A project declaration can list `shares` under `[vms.]`. Stoat resolves +each entry relative to the directory containing `stoat.toml` and refuses a +path that leaves the project, including through a symlink. `.` mounts at +`/work`; a subdirectory mounts at `/work/`. These exports are +separate from the legacy `share` export and are read-write inside the guest. + +Debian cloud images do not mount project shares because their cloud kernel has +no 9p module. Ubuntu, Fedora, Arch, and Alpine use the guest's 9p support when +the definition permits it. See [The project file](../reference/project-file.md#shares) +for the declaration syntax. + ## Sharing a binary built on the host: the musl/glibc trap If you build a binary on a typical glibc-based Linux host (Arch, Fedora, diff --git a/docs/design/core-api.md b/docs/design/core-api.md index 4837e705..cc8604ca 100644 --- a/docs/design/core-api.md +++ b/docs/design/core-api.md @@ -1,6 +1,10 @@ # Core API: Operation Surface -**Status:** proposal for review. Branch `core-api`. Written 2026-08-02. +**Status:** original API proposal, written 2026-08-02, with later decisions +and implementation conventions appended. The proposed types and build order +below are historical. Sections 9 and 12 contain the error and CLI conventions +used by [Contributing](../../CONTRIBUTING.md). For the current public interface, +see the [CLI reference](../reference/cli.md) and [JSON contract](../reference/json.md). **Companion to:** [`guest-subsystem.md`](guest-subsystem.md) (§9 defines why this layer exists and where it sits). This document defines *what it does*. @@ -165,7 +169,7 @@ These are the "quick VM-based testing" features that make stoat pleasant to use. | **Snapshot** | `Snapshot(ctx, name, label) error` | `qemu-img snapshot -c` on a stopped VM; QMP `savevm` for live. "Set it up, snapshot, break it, restore" is the core testing loop, and it is the single feature that makes stoat *better* than re-creating a VM rather than merely faster. | | **Restore** | `Restore(ctx, name, label) error` | Reset to a known state without a rebuild. For an agent, this is how you get a clean environment per task without paying a full create. | | **Snapshots** | `Snapshots(ctx, name) ([]Snapshot, error)` | List with labels, sizes, timestamps. | -| **Forward** | `Forward(ctx, name, []PortForward) error` | Today only :22 is forwarded. Testing a web service in a VM means reaching :8080, currently impossible without hand-editing QEMU args. Applies at next start. | +| **Forward** | `Forward(ctx, name, []PortForward) error` | At the time of this proposal, only :22 was forwarded. Testing a web service in a VM meant reaching :8080 was impossible without hand-editing QEMU args. Applies at next start. | | **Images** | `Images(ctx) ([]Image, error)` | Catalog plus local, with download state and size. | | **DownloadImage** | `DownloadImage(ctx, id, progress chan<- Progress) error` | Cancellable: today `esc` leaves the goroutine running, a known open item. | | **Doctor** | `Doctor(ctx) ([]Check, error)` | Structured dependency/environment checks, not printed text. | diff --git a/docs/design/guest-subsystem.md b/docs/design/guest-subsystem.md index 38b54930..bde9378d 100644 --- a/docs/design/guest-subsystem.md +++ b/docs/design/guest-subsystem.md @@ -1,11 +1,13 @@ # Guest Subsystem: Design -**Update, 2026-09-05:** the guest registry this document designs is now data, -not Go: `internal/guest/bundled/*.toml` plus `~/.stoat/guests/*.toml` merged -over it, loaded through `internal/tomlx`. See `docs/reference/guest.md` for -the file format. +**Implementation status:** the guest registry uses bundled TOML files in +`internal/guest/bundled/` with overrides from `~/.stoat/guests/`, loaded +through `internal/tomlx`. See the [guest reference](../reference/guest.md) +for the current file format. The problem inventory and proposed types below +describe the original design context. -**Status:** accepted design, not yet a plan. Written 2026-08-02. The operation surface built on this layer is specified in [`core-api.md`](core-api.md). +**Status:** accepted design, written 2026-08-02. The original operation +surface is described in [Core API](core-api.md). **Why this exists:** guest-OS knowledge is scattered across 25 sites as ad-hoc string comparisons, provisioning has no contract at all, and the logic that creates a VM lives inside a Bubbletea form. Adding Alpine cloud support missed three of those sites and the feature silently did not work. This document defines the subsystem that makes that class of failure structural rather than a matter of remembering, and the API layer that lets something other than a keyboard drive stoat. @@ -281,9 +283,17 @@ For phase-2 recipes, progress comes from the declared `stages` plus emitted mark ### 9.1 The problem -**Creating a VM is only possible by driving a Bubbletea form.** `internal/tui/form.go`'s `build()` (:753+) resolves the image, infers the OS, picks the backend, allocates an SSH port via `config.FreePort()`, writes `vm.toml` and creates the qcow2. The CLI has `ls`, `up`, `down`, `ssh`, `provision`, `rm`, `recipe`, `logs`, `doctor`, and **no `create`**. - -So orchestration sits *above* the layer any programmatic caller would enter at. An MCP server would have to either re-implement `form.build()` (a second, drifting copy of the rules) or drive a TUI, which is absurd. The same is true of the CLI, which is why it has no `create` today. +**At the time of this proposal, creating a VM was only possible by driving a +Bubbletea form.** `internal/tui/form.go`'s `build()` (:753+) resolved the image, +inferred the OS, picked the backend, allocated an SSH port via +`config.FreePort()`, wrote `vm.toml` and created the qcow2. The CLI then had +`ls`, `up`, `down`, `ssh`, `provision`, `rm`, `recipe`, `logs`, `doctor`, and +**no `create`**. + +So orchestration sat *above* the layer any programmatic caller would enter at. +An MCP server would have had to either re-implement `form.build()` (a second, +drifting copy of the rules) or drive a TUI, which is absurd. The same was true +of the CLI, which is why it had no `create` command at that time. This is a layering defect, not a missing feature. diff --git a/docs/design/json-contract-draft.md b/docs/design/json-contract-draft.md index ea0a8f92..c34d0fc9 100644 --- a/docs/design/json-contract-draft.md +++ b/docs/design/json-contract-draft.md @@ -1,16 +1,20 @@ # stoat CLI structured output: the MCP API boundary -**Status:** proposal. Written 2026-08-04. No code changed. - -**Premise (settled, not relitigated):** the MCP server is Python + fastmcp in a -separate process. It reaches `internal/core` only by executing the `stoat` -binary and reading its output. Therefore this document specifies an **API**, -not a display format. Anything a Python caller has to regex, guess at, or -reconstruct from two streams is a defect in the API, not a rough edge. - -Read against: `internal/cli/cli.go` (18 subcommands today), -`internal/core/*.go` (14 typed errors, 9 return types), -`docs/design/core-api.md` §9/§10, `docs/reference/cli.md` (**stale, see §8.6**). +**Status:** historical proposal, written 2026-08-04. The Python MCP wrapper +described below has been replaced by the Go server in the Stoat binary. +Use the current [JSON contract](../reference/json.md) and +[MCP reference](../reference/mcp.md) when building integrations. + +**Historical premise (for the 2026-08-04 proposal):** the MCP server was Python +with fastmcp in a separate process. It reached `internal/core` only by executing +the `stoat` binary and reading its output. Therefore this document specifies an +**API**, not a display format. Anything a Python caller had to regex, guess at, +or reconstruct from two streams was a defect in the API, not a rough edge. + +The proposal was read against the 2026-08-04 tree: `internal/cli/cli.go` (18 +subcommands), `internal/core/*.go` (14 typed errors, 9 return types), +`docs/design/core-api.md` §9/§10, and the then-stale `docs/reference/cli.md` +(see §8.6). --- diff --git a/docs/design/mcp-server.md b/docs/design/mcp-server.md index a297be87..49bc42cf 100644 --- a/docs/design/mcp-server.md +++ b/docs/design/mcp-server.md @@ -4,7 +4,10 @@ Decisions are settled here. This document is the source of truth for the implementation; `core-api.md` §10 and `json-contract-draft.md` §7 hold the original reasoning and should be read first, not re-argued. -Contract: [../reference/json.md](../reference/json.md). +For setup, access levels, and available tools, use the +[MCP reference](../reference/mcp.md). The [JSON contract](../reference/json.md) +defines the shared result types. This document records implementation decisions +and the original validation plan. ## 0. Settled, do not relitigate diff --git a/docs/design/tui-migration-draft.md b/docs/design/tui-migration-draft.md index 39ba200b..97abc899 100644 --- a/docs/design/tui-migration-draft.md +++ b/docs/design/tui-migration-draft.md @@ -1,12 +1,17 @@ # Migrating internal/tui onto internal/core -Planning document. No code was changed. Written against the working tree of +Historical migration plan. The build failures and unfinished work listed here +describe the 2026-08-04 checkout. Use the [TUI reference](../reference/tui.md) +for current behavior and the [manual checks](../qa/tui-clickthrough.md) for +validation. + +The original plan was written against the working tree of 2026-08-04, which has a large in-flight comment sweep on top of `d940ff9` plus `c3ba57b` (sshx ctx). ## 0. Blocking precondition -`internal/tui` does not compile in the working tree right now: +In that 2026-08-04 working tree, `internal/tui` did not compile: ``` internal/tui/autoprov.go:30:26: not enough arguments in call to sshx.Wait diff --git a/docs/getting-started/first-vm.md b/docs/getting-started/first-vm.md index 2cb2ad34..5f389a40 100644 --- a/docs/getting-started/first-vm.md +++ b/docs/getting-started/first-vm.md @@ -1,6 +1,8 @@ # Your First VM -This walks through creating and using an Alpine **live** VM end to end: launch the TUI, download an image, create the VM, start it, provision it, and ssh in. Live is the mode to start with: it needs no manual steps inside the guest, unlike disk or cloud VMs (covered at the end of this page). +Create an Alpine **live** VM, apply a recipe, and connect over SSH. Live mode +needs no OS installation and discards the guest session when it stops. Disk +and cloud VMs are covered at the end of this page. Before you start, make sure `stoat doctor` reports `ok`, see [Installation](installation.md). @@ -10,32 +12,24 @@ Before you start, make sure `stoat doctor` reports `ok`, see [Installation](inst stoat ``` -On a fresh install you'll see stoat's banner over an empty list: +On a fresh install you'll see stoat's banner over an empty list. The footer +shows the list screen's keys: `↵ start/stop`, `→/l details`, `s ssh`, +`p apply`, `/ search`, `n new`, `d delete`, `q quit`, `? help`. -``` -no vms yet, press n to create one -``` +The example below shows the list after creating three VMs: -with a footer showing the list screen's keys: `↵ start/stop`, `→/l details`, `s ssh`, `p provision`, `/ search`, `n new`, `d delete`, `q quit`, `? help`. +![The VM list with two running guests and one stopped guest](../../assets/tui-list.png) ## 2. Press `n` for a new VM -This opens the "new vm" form. It starts you on the **name** field, and Alpine is preselected in the **image** row (it's the default catalog entry, and the only OS whose live mode works without an install step): - -``` -❯ name work - image alpine apkovl ⤓ download - mode (•) live ( ) disk - runs in RAM · ssh works now · reboot wipes it - - ram 4096 MB - cpus 4 - share ~/vms +This opens the "new vm" form. It starts you on the **name** field, and +Alpine is preselected in the **image** row. Alpine is the default catalog +entry and the only OS whose live mode works without an install step. - recipes [ ] devtools [ ] docker [ ] tailscale [ ] xfce -``` +Use `work` as the name for this walkthrough. `tab`/`↓` and `shift+tab`/`↑` +move between fields; `←`/`→` changes a picker's value. -Type a name (letters/digits, no spaces or slashes). `tab`/`↓` and `shift+tab`/`↑` move between fields; `←`/`→` changes a picker's value. +![The new VM form](../../assets/tui-create.png) ## 3. Download the image @@ -55,19 +49,25 @@ When it finishes, the status line reports `downloaded isos/alpine-standard-....i Press `enter` (from anywhere in the form) to create the VM. If the image hasn't finished downloading yet, stoat tells you instead of creating a broken VM (`press space to download alpine first`). -You're back on the list screen, with a `created ` status message and your new VM showing: - -``` -❯ ○ work live 4096M 4c - -``` +You're back on the list screen with a `created ` status message and your +new VM listed as stopped. ## 5. Start it -With the VM selected, press `enter`. stoat rebuilds the VM's Alpine overlay (this is what wires in the SSH key and networking) and launches QEMU. The status line reports ` started`, and the row's dot turns solid to show it's running, with an uptime and its forwarded ssh port: +With the VM selected, press `enter`. stoat rebuilds the VM's Alpine overlay +(this wires in the SSH key and networking) and launches QEMU. The status line +reports that it started. Press `l` to see its state, forwarded SSH port, +and apply log. This capture uses the example VM `alpine-desktop`: -``` -● work live 4096M 4c up 12s :2200 -``` +![VM detail screen](../../assets/tui-details.png) + +When the `xfce` recipe finishes on a graphical host, its desktop appears in a +QEMU window: + +![Alpine XFCE in a QEMU window](../../assets/qemu-xfce.png) + +The [23-second walkthrough video](../../assets/demo.mp4) shows the TUI list, +details, and the resulting XFCE window. **If you selected any recipes** in step 4, stoat watches for sshd in the background (you can keep using the TUI while it waits), and once the guest answers, it asks: @@ -75,7 +75,11 @@ With the VM selected, press `enter`. stoat rebuilds the VM's Alpine overlay (thi work is up, run xfce now? y/N ``` -Press `y` to run them (this streams into the same file the detail screen's "last provision" pane tails), or anything else to skip (`not provisioning work, press p when you want to`). The prompt reappears every time you (re)start a live VM, since a reboot wipes whatever ran before. +Press `y` to apply the selected recipes. The detail screen's **last apply** +pane shows their output. Press another key to skip; `p` starts an apply later. +On later boots, the host's recipe records can cause a `run = "once"` recipe +to be skipped after its guest files have disappeared. See the +[live restart limitation](../troubleshooting.md#a-live-vm-lost-everything-after-a-reboot). If you didn't select any recipes, there's nothing to offer, and you go straight to step 6. @@ -102,7 +106,9 @@ Stop it with `enter` again (or `stoat down `) whenever you're done: as a l ## Disk and cloud VMs -Live isn't the only mode: the other two exist because they keep guest state, at the cost of a manual step live doesn't need: +Live isn't the only mode: the other two keep guest state. + +- **Disk** VMs boot from the same kind of ISO and keep a qcow2 disk that survives restarts. An Alpine disk VM installs itself on its first start with a generated `setup-alpine` answer file. `stoat up` waits for the installer to power off, starts the VM from the new disk, and then offers or runs its recipes. The unattended install can take up to 15 minutes. The generated overlay carries stoat's SSH key into the installed system. For a non-Alpine BYO ISO, run that guest's installer at the console, add stoat's public key to the configured SSH account, then stop and start the VM. `i` on the detail screen still flips `installed` by hand when automatic detection is wrong. +- **Cloud** VMs (Ubuntu, Debian, Fedora, Arch, or Alpine cloud) use a prebuilt cloud image instead of an ISO. stoat writes a cloud-init seed on first start that creates a `stoat` user, installs stoat's key for it, and applies the selected recipe content during that first boot. After SSH becomes available, the normal apply pass can record the results and apply pending or changed recipe work over SSH. Pressing `p` runs that normal apply path; it does not rebuild the first-boot seed. SSH in as `stoat@127.0.0.1:` rather than `root`. Building that seed needs `xorriso` on your `PATH` (see [Installation](installation.md)). -- **Disk** VMs boot from the same kind of ISO but keep a qcow2 disk that survives restarts. You install the OS yourself at the QEMU console (`setup-alpine`, for an Alpine disk image) and then restart the VM: the next start sees an OS on the disk, marks the VM `installed` and stops booting the ISO. Until then, `s` and `p` refuse with a message pointing you at that same step. An Alpine disk VM carries the same apkovl as a live one *while installing*, so `setup-disk` copies your ssh key onto the installed system for you. `i` on the detail screen still flips `installed` by hand when the guess is wrong. -- **Cloud** VMs (Ubuntu, Debian, Fedora, Arch) use a prebuilt cloud image instead of an ISO. stoat writes a cloud-init seed on first start that creates a `stoat` user, installs stoat's key for it, and, unlike live/disk, applies any selected recipes automatically as part of that same first boot, with no offer or `p` needed (pressing `p` on a cloud VM just tells you so). SSH in as `stoat@127.0.0.1:` rather than `root`. Building that seed needs `xorriso` on your `PATH` (see [Installation](installation.md)). +For a repository with multiple VMs, use the [project workflow](../guides/project-workflow.md) instead of recreating each VM in the TUI. diff --git a/docs/getting-started/installation.md b/docs/getting-started/installation.md index 5bef534d..45864370 100644 --- a/docs/getting-started/installation.md +++ b/docs/getting-started/installation.md @@ -9,7 +9,7 @@ stoat is a single Go binary that shells out to `qemu-system-x86_64`, `qemu-img`, | Linux with KVM | stoat opens `/dev/kvm` directly and runs QEMU with hardware acceleration | | `qemu-system-x86_64` and `qemu-img` | run and create VMs | | `ssh` (an OpenSSH client) | connect into a running VM, and provision it | -| `xorriso` | **only** if you'll create Ubuntu/Debian/Fedora/Arch cloud VMs: it builds the cloud-init seed image | +| `xorriso` | used when you create Ubuntu/Debian/Fedora/Arch/Alpine cloud VMs: it builds the cloud-init seed image | | Go 1.26 | **only** if you're building from source (the exact version pinned in `go.mod`) | ### QEMU and SSH @@ -20,7 +20,9 @@ On Arch: sudo pacman -S --needed qemu-full openssh ``` -(`qemu-desktop` also works and is smaller; either package provides `qemu-system-x86_64` and `qemu-img` with GTK/OpenGL display support, see below.) +(`qemu-desktop` also works and is smaller. Choose a QEMU package that provides +`qemu-system-x86_64`, `qemu-img`, and the GTK/OpenGL display backend described +below.) On Debian/Ubuntu: @@ -50,15 +52,18 @@ sudo usermod -aG kvm "$USER" # then log out and back in ### GPU/display -A VM opens a real QEMU window by default. That window is `-display gtk,gl=on`, so your QEMU build needs GTK and OpenGL support (the `qemu-full`/`qemu-desktop` Arch packages and the Debian/Ubuntu packages above provide this). Set `display = "vnc"` on a VM to keep it headless with its screen on a VNC socket instead. +A VM opens a real QEMU window by default on a host with a graphical session. +Stoat passes `-display gtk,gl=on`, so the QEMU binary on your `PATH` must have +GTK and OpenGL support. Set `display = "vnc"` in `vm.toml` to keep a VM +headless with its screen on a VNC socket. **On a host with no graphical session** (a server, an ssh session with no forwarding) stoat does not ask for a window at all: every VM's screen goes to a VNC socket and stoat prints how to attach, so a disk VM can still be installed from another machine. It detects this from `DISPLAY`, `WAYLAND_DISPLAY` and `$XDG_RUNTIME_DIR/wayland-0`. If a VM still fails to start with a display or GL error, your host has a session QEMU cannot draw on. Set `STOAT_GRAPHICAL=0` to take the window out of play; see [troubleshooting](../troubleshooting.md). No source edit and no rebuild. -### xorriso (cloud VMs only) +### xorriso (cloud VMs) -Provisioning an Ubuntu, Debian, Fedora, or Arch cloud image builds a small ISO9660 seed via `xorriso`. Alpine VMs (live or disk) never need it. If it's missing, creating or starting a cloud VM fails with: +Provisioning an Ubuntu, Debian, Fedora, Arch, or Alpine cloud image builds a small ISO9660 seed via `xorriso`. Alpine live and disk VMs do not need it. If it is missing, creating or starting a cloud VM fails with: ``` xorriso is required for cloud-init provisioning; install libisoburn @@ -70,7 +75,10 @@ On Debian/Ubuntu the package is `xorriso`; on Arch it's `libisoburn` (which prov Requires Go 1.26. -There's both a `justfile` and a `Makefile` exposing the same targets, use whichever you have: +The repository has a `justfile` and a smaller `Makefile`. They share `build`, +`install`, `hooks`, and `test`; the `justfile` also provides `setup`, `check`, +`lint`, `e2e`, and other development targets. Use the `justfile` for the +installer and host checks: ```sh just build @@ -84,11 +92,14 @@ make build make install ``` -`install` builds the binary and copies it to `~/.local/bin/stoat` (with `just`, override the destination via `PREFIX`). Make sure `~/.local/bin` is on your `PATH`. +`just install` builds the binary and copies it to `~/.local/bin/stoat` by +default. Set `PREFIX` to change that destination. The Makefile's `install` +target always uses `~/.local/bin`. Make sure the destination is on your +`PATH`. -> If your shell aliases `make` to `just`, `make build`/`make install` still work: the alias resolves to the justfile, which has the same targets. To reach the real Makefile explicitly, use `command make ...`. - -Other useful `just`/`make` targets: `just test` runs the test suite, `just check` runs the same `gofmt`/`go vet`/`go build` checks as the pre-commit hook, and `just hooks` points git at `.githooks` to enable it. +Other useful targets: `just test` runs the test suite, `just check` runs the +same `gofmt`/`go vet`/`go build` checks as the pre-commit hook, and `just hooks` +points Git at `.githooks` to enable it. ## Install from a release tarball @@ -110,7 +121,11 @@ nix run # build and run in one step nix develop # a dev shell with go, just, qemu and openssh ``` -The `vendorHash` is pinned and the build works as-is, no hash dance required. +The development shell provides the build tools and QEMU/OpenSSH packages. Add +`xorriso` to the shell or host `PATH` when you use cloud VMs. The flake includes +a pinned `vendorHash` for the current `go.mod` and +`go.sum`. A dependency change requires updating that hash from the value Nix +prints in its mismatch error. ### If `nix build` fails before it starts building @@ -165,7 +180,9 @@ Once the binary is installed, check that your host is ready: stoat doctor ``` -This checks that `qemu-system-x86_64` is on your `PATH`, that `/dev/kvm` is usable, and that `ssh` is on your `PATH`. On success it prints: +This checks `qemu-system-x86_64`, `qemu-img`, `ssh`, `xorriso`, Git (optional), +and `/dev/kvm`. It prints failed checks and a suggested command for each. A +missing required check exits with status 1. On a ready host it prints: ``` ok diff --git a/docs/guides/mcp-workflow.md b/docs/guides/mcp-workflow.md new file mode 100644 index 00000000..2041a2cc --- /dev/null +++ b/docs/guides/mcp-workflow.md @@ -0,0 +1,109 @@ +# MCP workflow + +The Go binary serves the Model Context Protocol over stdio by default. An MCP +client launches `stoat mcp` as a subprocess, so the client needs only the +installed binary. The server uses the process working directory as project +scope. + +## Install a client entry + +From the project directory when the client should use that project's +`stoat.toml`, run one of: + +```sh +stoat mcp install claude-code +stoat mcp install claude-desktop +stoat mcp install cursor +stoat mcp install vscode +``` + +`claude-code`, `claude-desktop`, and `cursor` use their normal user config +files. VS Code uses `.vscode/mcp.json` in the current directory. Use +`--project` with Claude Code to write `.mcp.json` in the current directory. +`--print` prints the JSON entry without writing a file. An existing `stoat` +entry is replaced; other client entries remain. + +Check the installed entry and server contract with: + +```sh +stoat mcp doctor +``` + +The report includes contract version, transport, and client-entry status. The +server advertises the same contract used by `stoat --json`; see [JSON output](../reference/json.md) +for the shared data types and error rules. + +## Choose a transport + +Stdio is the normal transport: + +```sh +stoat mcp +``` + +For a client that cannot launch a subprocess, serve streamable HTTP on a +loopback address: + +```sh +stoat mcp serve --http 127.0.0.1:7777 +``` + +The HTTP server rejects non-loopback addresses and has no authentication. Keep +it on loopback and use an authenticated tunnel if another machine must reach +it. The default per-tool limit is a burst of 30 calls with a refill of 0.5 +calls per second. The shared server limit is a burst of 60 with a refill of 2 +calls per second. Override them on `mcp serve` when the client needs another +bound. + +## Set the VM access level + +Set `agent_access` when creating a VM or in its project declaration: + +```sh +stoat create --image ubuntu-24.04 --agent-access observe lab +``` + +The levels are cumulative. `none` allows host-side VM operations. `observe` +adds guest reads, process listings, service status, and log reads. `manage` +adds file writes, file copies, package installation, service changes, user +creation, and recipe application. `exec` adds foreground and background +commands plus job management. New VMs default to `manage`. + +An MCP `update` can lower a VM's level but cannot raise it. Raise a level with +the CLI or TUI so a person grants that capability explicitly. The older +`allow_exec` field is still read for existing files; `true` maps to `exec` and +`false` maps to `manage`. + +## Use project tools + +The server registers project tools in every process. They work when its fixed +working directory contains a `stoat.toml`; outside a project they return an +error explaining that scope. The project tools are: + +| Tool | Effect | +|---|---| +| `project_status` | Read every declaration's state, health, and drift | +| `project_up` | Reconcile and start every declared VM in order | +| `project_down` | Stop every declared VM in order | +| `project_apply` | Apply every declared VM's recipes in order | +| `project_wait` | Wait for every declared VM to answer on SSH in order | + +These tools use the server's fixed working directory. They stop at the first +failure and mark later entries as skipped. Without a project file, use +`start`, `stop`, `apply_recipes`, and `wait` with a VM name. The server does +not walk up to find a parent `stoat.toml`. + +## Plan before applying + +Use `plan_recipes` before `apply_recipes` to see which recipes will run, which +will be skipped, and why. `apply_recipes` requires `agent_access = manage` or +`exec`, runs the recipe scripts over SSH, and writes the same apply state and +log that the CLI uses. `wait` can wait for `reachable`, `applied`, or +`stopped`; `healthy` waits for SSH and every applied recipe's health check. + +Read-only tools include VM and image listings, recipe applicability and +schemas, guest definitions, logs, and host checks. Guest read tools require +`observe`; guest writes, package, service, and user tools require `manage`; +command and job tools require `exec`. The server validates absolute guest +paths, caps large reads and listings, redacts secret values, and passes command +arguments without joining them into a shell string. diff --git a/docs/guides/project-workflow.md b/docs/guides/project-workflow.md new file mode 100644 index 00000000..978d101c --- /dev/null +++ b/docs/guides/project-workflow.md @@ -0,0 +1,135 @@ +# Project workflow + +Use a project file when a repository needs the same VM definitions for every +checkout. `stoat.toml` is committed. The `.stoat/` cache and its secrets are +ignored. The project lock file is committed with the declaration. + +## Create the declaration + +From the repository root, run: + +```sh +stoat init +``` + +This writes `stoat.toml` with one example VM. It refuses to overwrite an +existing file. In a Git checkout it also adds `.stoat/` to `.gitignore`. +Edit the image, VM key, resources, recipes, and shares. `image` is required; +the other fields use the same defaults as `stoat create` when omitted. + +For example: + +```toml +schema = 1 + +[project] +name = "myrepo" + +[vms.dev] +image = "ubuntu-24.04" +cpus = 4 +ram = 4096 +disk = "20G" +recipes = ["docker"] +shares = ["."] +agent_access = "manage" + +[vms.dev.params.docker] +user = "dev" +``` + +The key `dev` resolves to `myrepo-dev` unless `name` overrides it. A command +typed in the project directory resolves a bare key first, then a declared +global name. Scope is current-directory only; stoat does not search parent +directories. See [The project file](../reference/project-file.md) for all +fields and validation rules. + +## Pin and cache recipes + +Declare a recipe in `[recipes]`, then resolve it to a commit: + +```sh +stoat recipe add https://github.com/OWNER/REPOSITORY.git@TAG +stoat recipe lock +stoat recipe sync +``` + +Replace the URL and tag with a repository that contains a valid `recipe.toml`. +The repository's current `index.toml` has no published entries, so an index +name cannot be resolved from this checkout. Bundled recipes such as `docker` +are already available and do not need `recipe add`. + +An index name can be used when the configured index contains it. A Git URL can +be passed instead, with an optional `@tag` or branch. `recipe add` writes the +project declaration, lock entry, and cache entry as one operation. Run +`recipe lock` after editing `[recipes]`; it updates `stoat.lock` but does not +populate the cache. Run `recipe sync` to make `.stoat/recipes/` match the lock. + +The project lock is `./stoat.lock`. A command outside project scope uses the +global lock at `~/.stoat/stoat.lock` and the global cache at +`~/.stoat/recipes/`; pass `--global` to force that scope from a project +directory. Do not edit a lock entry by hand. Commit `stoat.lock` so another +checkout runs the same recipe commits. + +## Keep secrets out of Git + +Non-secret recipe parameters belong in `[vms..params.]`. Put +secret parameters in `.stoat/secrets.toml`, which stoat writes with mode +`0600` and never includes in status output. Its keys use the declaration key: + +```toml +[dev.tailscale] +authkey = "tskey-..." +``` + +Do not put the secret in `stoat.toml`, `stoat.lock`, or a command line. A +missing required secret fails when stoat validates the recipe before it runs. + +## Start and reconcile + +Check the declaration and existing VM state without changing anything: + +```sh +stoat status +``` + +Create missing VMs, reconcile mutable fields, and start every declaration in +file order: + +```sh +stoat up +``` + +`up` creates a missing VM from its declaration. For an existing VM it applies +CPU, memory, recipe, parameter, share, and agent-access changes through the +same validation as `stoat update`. Image and disk changes are immutable; stop, +remove, and recreate that VM when those fields change. CPU, memory, and shares +are saved immediately but take effect at the next down and up. + +`up` waits for Alpine disk auto-installation to finish. It then waits for SSH +and applies pending recipes. Use `--no-apply` when the VM should start without +the post-boot recipe pass. A named invocation operates on one declaration: + +```sh +stoat up dev +stoat wait dev --healthy +``` + +For a cloud VM, changing the recipe list changes the declaration used by later +reconciliation, but it does not rebuild the first-boot seed. Recreate the VM +when the new list must be present in cloud-init. Existing recipe scripts can +still run through the normal apply policy after SSH is ready. + +The no-argument project commands `up`, `down`, `apply`, `wait`, and `rm` stop +at the first failure and report later declarations as skipped. `rm` requires +`-y` when used without an interactive confirmation. + +## Inspect drift and resolve failures + +`stoat status` reports each declaration's global name, state, health, and +drift. A missing VM is reported as `missing`. An image or disk change reports +the remove-and-recreate command. A share outside the repository, including a +symlink that resolves outside it, is rejected before a VM is changed. + +Use the [troubleshooting guide](../troubleshooting.md) for readiness, apply, +recipe lock, and project-scope errors. diff --git a/docs/qa/tui-clickthrough.md b/docs/qa/tui-clickthrough.md index c79fb8dd..1094b9a2 100644 --- a/docs/qa/tui-clickthrough.md +++ b/docs/qa/tui-clickthrough.md @@ -1,53 +1,52 @@ # TUI click-through: manual smoke test -The checks an agent cannot run, because they need a real Alpine boot and a human -reading the screen. Run these after any change to the list or detail screens. +Run these checks with real guests after changes to the list or detail screens. +Inspect the rendered screen as well as the command output. -Data root is `~/.stoat` (or `$STOAT_HOME`). Each VM is a directory -`~/.stoat//` that holds a `vm.toml`. Build and launch the TUI with `just run` -or `go run ./cmd/stoat`. +Use a separate data root for these checks. They modify VM configuration, +delete test VMs, and restore snapshots. Set `STOAT_HOME` to a scratch directory +before creating the guests. Build and launch the TUI with `just run` or +`go run ./cmd/stoat`. Paths below use `$STOAT_HOME`. Keys, for reference. List screen: `enter` start/stop, `l`/`→` details, `s` ssh, `p` provision, `d` delete, `n` new, `r` edit recipes, `/` search. Detail screen: `e` edit form, `E` raw vm.toml in `$EDITOR`, `i` installed toggle, `s` ssh, `p` provision, `L` console log, `S` snapshots, `esc`/`h`/`q` back. -## 1. VM start: row flip and uptime tick +## 1. VM start and uptime display -Verifies the running dot and the `time.Since(StartedAt)` uptime that ticks with -no list refresh. +Verify the running indicator and elapsed uptime. The current list has no +periodic refresh: uptime changes when input or another event causes a render. +The idle refresh check below records that limitation. 1. On the list, select a stopped VM. The dot is the stopped glyph, the row shows `-`. 2. Press `enter`. The VM starts. -3. Watch the row. The dot flips to the running glyph. The row reads - `up 1s :PORT`, then `up 2s`, then `up 3s`. The count climbs every second on - its own. -4. Confirm the seconds keep climbing without any keypress. A frozen number means - the uptime regressed to a stored duration. -5. Press `enter` again to stop it. The dot returns to stopped and the uptime - drops to `-`. +3. Watch the row. The dot changes to the running glyph and shows `up ... :PORT`. +4. Wait a few seconds without input and record whether uptime changes. Press + `j` or `k` and confirm that the next render shows the elapsed time. Return + the selection to the VM from step 1 and confirm its name before continuing. +5. Press `enter` again to stop that VM. The dot returns to stopped and the + uptime drops to `-`. ## 2. Corrupt vm.toml: shows broken, deletes with d+y Verifies a broken VM still lists and deletes, keyed by its directory. 1. Stop the TUI. Pick a directory name that sorts mid-list, e.g. `mmm-broken`. - Create `~/.stoat/mmm-broken/vm.toml` with garbage: `not = valid = toml`. + Create `$STOAT_HOME/mmm-broken/vm.toml` with garbage: `not = valid = toml`. 2. Launch the TUI. `mmm-broken` appears in alphabetical position, marked broken. -3. Select it and press `l`. The detail screen still opens; `L` serves any - console log the directory holds. -4. Back on the list, press `d`. A `y/N` confirmation appears. -5. Press `y`. The row is removed and `~/.stoat/mmm-broken/` is gone. -6. Repeat step 4. Press any key other than `y`. The TUI cancels the delete and - the row stays. +3. Select it and press `l`. The TUI stays on the list and reports that + `vm.toml` is broken. +4. Press `d`, then `n`. The confirmation closes and the row remains. +5. Press `d`, then `y`. The row and `$STOAT_HOME/mmm-broken/` are removed. ## 3. vm.toml name differs from directory Verifies every operation keys off the directory, never the `name` field. This identity bug recurred before. It must stay fixed. -1. Stop the TUI. In a working VM at `~/.stoat/realdir/vm.toml`, change the `name` +1. Stop the TUI. In a test VM at `$STOAT_HOME/realdir/vm.toml`, change the `name` field to `wrongname` (leave the directory `realdir`). 2. Launch the TUI. The row shows `realdir`, the directory, not `wrongname`. 3. Press `l` for details. Run each key below. Each one must act on this VM, and @@ -56,7 +55,7 @@ identity bug recurred before. It must stay fixed. - `L` shows this VM's console log. - `s` opens an ssh session to it (VM must be running). - `p` starts a provision run against it. -4. Restore the `name` field afterward if you care about it. +4. Restore the original `name` field after the check. ## 4. Snapshots modal (S), four states @@ -67,13 +66,16 @@ need a disk or cloud VM, not live. opens and shows an empty-state line, not a blank box. 2. Press `d`, then `r`. Neither does anything on an empty list. `esc` closes the modal. -3. Take a snapshot first (`stoat snapshot ` from a shell), then reopen - the detail screen and press `S`. The snapshot lists by its tag and date. +3. Take a snapshot named `clean` first (`stoat snapshot clean` from a + shell), then reopen the detail screen and press `S`. The snapshot lists by + its tag and date. 4. Press `d`. A `delete clean? y/N` line appears. Press a non-`y` key. The TUI deletes nothing. Press `d` then `y`. The TUI removes the snapshot and refreshes the list. -5. Take another snapshot. Press `r`. A `restore clean? y/N` line appears. Confirm - with `y` and the VM's disk rolls back to that snapshot. -6. Run steps 1-5 once with the VM stopped and once running. Both must behave the - same. A running VM's `info snapshots` prints `--` for the ID. The modal keys - everything off the tag for that reason. +5. Take another snapshot named `clean`. Press `r`. A `restore clean? y/N` line + appears. Confirm with `y` and verify that the VM returns to that snapshot. +6. Run steps 1-5 once with the VM stopped and once running. The modal and + confirmation behavior must match. Snapshot semantics differ: a stopped + snapshot stores disk state, while a running snapshot also stores VM memory + and restores in place. A running VM's `info snapshots` prints `--` for the + ID. The modal keys everything off the tag for that reason. diff --git a/docs/recipe-authoring-spec.md b/docs/recipe-authoring-spec.md index b2671926..310a2d42 100644 --- a/docs/recipe-authoring-spec.md +++ b/docs/recipe-authoring-spec.md @@ -1,27 +1,29 @@ # Spec: authoring recipes in stoat -Status: **proposal, not built.** Written 2026-07-31 in response to "a selector -or some kinda recipe creator if that's something we can spec out". +Status: **historical proposal.** Written 2026-07-31. The authoring workflow +described here predates the shipped directory manifests and +`stoat recipe new`; keep it as design history, not as a current interface. +For current behavior, see [Recipes](recipes/overview.md) and [Writing your +own recipe](recipes/writing-your-own.md). -## What a recipe is today +## What a recipe was in this proposal's context -A file in `~/.stoat/recipes/`, named `..sh` or `.cloud.yaml`. -Shell recipes are piped into `sh -s` over ssh; cloud fragments are merged into -the cloud-init seed. `recipes.List(os, backend)` filters by the suffix, so the -picker only ever offers files that can run on the selected image. +The proposal assumed a file in `~/.stoat/recipes/`, named `..sh` or +`.cloud.yaml`. That description is historical. Current recipes are +directories containing `recipe.toml` and scripts; the manifest declares target +guests, requirements, runtime, parameters, outputs, and health checks. -There are 7 files and 441 lines today. `Install()` copies the bundled ones into -the data root and **never overwrites**, so editing one in place already works -and survives upgrades. +## Historical motivation -## The thing worth noticing first +The proposal explored how to make authoring discoverable and how to avoid +duplicating per-OS shell files. Its options and estimates below are retained +for design history. They are not a list of missing current features. -**Authoring a recipe is already supported.** Drop a file in `~/.stoat/recipes/` -named `mything.alpine.sh` and it appears in the picker for Alpine VMs. No code -required. Any "creator" competes with `$EDITOR` on a path that already works. +The shipped path is `stoat recipe new `, followed by editing the +generated manifest and scripts. See the current authoring guide for the exact +command and validation rules. -The question is what a creator does that `vim ~/.stoat/recipes/x.alpine.sh` -doesn't. Candidates below, smallest first. +The alternatives below describe the proposal, smallest first. --- @@ -91,7 +93,7 @@ A pane that lists recipes, with new/edit/delete and a text area. --- -## Recommendation +## Recommendation in the historical proposal **Option A, plus two things worth more than any of them:** @@ -105,7 +107,7 @@ A pane that lists recipes, with new/edit/delete and a text area. Option B only becomes worth it past roughly a dozen recipes across four OSes. At 7 files it would be more machinery than the thing it manages. -## What this does NOT propose +## What this historical proposal did not cover - A DSL. Phase 4 already rejected one: "per-OS templates = per-OS files, no DSL". - Fetching recipes from a registry. That is a supply-chain question, not an diff --git a/docs/recipe-spec-v2.md b/docs/recipe-spec-v2.md index 05ae3484..0d4e5931 100644 --- a/docs/recipe-spec-v2.md +++ b/docs/recipe-spec-v2.md @@ -1,10 +1,22 @@ -# Recipe Spec v2 +# Recipe Spec v2 (historical draft) -Status: **draft** +Status: **historical proposal**. This document records an earlier design and +does not define all current behavior. For the shipped authoring workflow, see +[Recipes](recipes/overview.md), [Writing your own recipe](recipes/writing-your-own.md), +and the [current sample](reference/samples/recipe.toml). + +The implementation currently accepts schema 2 and schema 3 manifests. Schema +3 adds parameters, secrets, outputs, and health checks. The current CLI also +provides `stoat recipe new`, remote recipe lock/sync, and project recipe +scopes; those shipped details are documented in the links above and in +[Sharing recipes](recipes/sharing.md). Statements below describe the draft +unless they are explicitly marked as current behavior. ## Summary -Recipes become directories containing a `recipe.toml` manifest and one or more shell scripts. One format, one execution model, works on every backend. +The draft proposed directories containing a `recipe.toml` manifest and one or +more shell scripts. The directory and manifest format is now shipped, but the +draft's backend and stage details are historical. ## Directory Structure @@ -48,7 +60,7 @@ fedora = "install-fedora.sh" # unlisted OSes fall back to `script` ``` -### Fields +### Fields in the draft | Field | Type | Required | Description | |-------|------|----------|-------------| @@ -70,7 +82,7 @@ A recipe declaring `requires = ["systemd"]` is not offered to Alpine. Stoat reso ## Stages -### `provision` (default) +### `provision` (default, draft model) Runs after the VM is booted and reachable over SSH. This is the common case: install packages, configure services, etc. @@ -78,7 +90,7 @@ Runs after the VM is booted and reachable over SSH. This is the common case: ins A non-`sh` runtime is bootstrapped first: stoat checks the guest for it and installs it with the guest's package manager if missing, over a separate SSH call, before piping the recipe body. - **cloudinit backend**: Wrapped into a cloud-config `runcmd` block at VM creation, runs at first boot. -### `install` +### `install` (draft model) Reserved for initial disk setup, before the first real boot. Alpine disk-mode VMs automate `setup-alpine` unattended without needing a recipe (see below); install-stage recipe bodies are not executed today. @@ -88,7 +100,7 @@ Reserved for initial disk setup, before the first real boot. Alpine disk-mode VM ## Execution Model -### For apkovl/ssh backends +### For apkovl/ssh backends (draft model) ``` VM boots @@ -97,7 +109,7 @@ VM boots -> marks them applied in vm.toml ``` -### For cloudinit backend +### For cloudinit backend (draft model) ``` VM creation @@ -152,7 +164,7 @@ This lets the TUI show: - Applied: recipe in both - Stale: recipe in `applied` but removed from `recipes` -## Migration +## Migration (historical proposal) ### Bundled recipes @@ -166,11 +178,14 @@ Convert existing files: One `xfce/` directory replaces 6+ files. -### User recipes +### User recipes (historical proposal) -Old-format recipes (`.sh` files in recipes root) continue to work during a deprecation period. Stoat logs a warning suggesting migration. +The proposal suggested a deprecation period for old-format recipes. Current +Stoat reads directory manifests; old flat recipe files are not a supported +authoring format. Use `stoat recipe new` or convert a recipe to the current +directory layout. -## Decisions +## Decisions recorded by the historical proposal 1. **Auto-provision**: Toggle-able via a field in recipe.toml. Recipes can declare `auto = true` to run automatically when the VM becomes reachable for the first time. Default is `false` (require explicit Apply). diff --git a/docs/recipes/overview.md b/docs/recipes/overview.md index 80d30302..826bbe18 100644 --- a/docs/recipes/overview.md +++ b/docs/recipes/overview.md @@ -1,114 +1,102 @@ # Recipes -A recipe is a small, named script that sets up something inside a guest -(XFCE, Docker, a dev toolchain, Tailscale) so you don't type the same install -commands into every VM you spin up. Recipes live as plain files on disk, in -`~/.stoat/recipes/` (or `$STOAT_HOME/recipes` if you've set that), and stoat's -picker offers you whichever ones make sense for the VM you're building. +A recipe is a named directory with a `recipe.toml` manifest and one or more +scripts. Stoat selects recipes by the guest OS and the capabilities declared by +that guest. The same recipe can run over SSH on an already booted VM or from a +cloud-init seed at first boot. -## Two kinds +## Recipe directories -**Shell recipes** run as `root` over ssh, piped into `sh -s` on an -already-booted guest. They're the right shape for the apkovl (live Alpine) and -plain-ssh (installed disk) backends, where a real shell is sitting there -waiting for you. +User recipes live under `~/.stoat/recipes/`, or under `$STOAT_HOME/recipes/` +when `STOAT_HOME` is set. A recipe directory must contain `recipe.toml` and +the script named by its `script` field. The manifest's `[scripts]` table can +select another script for a particular OS or one of that OS's aliases. -**Cloud-config fragments** are `#cloud-config` YAML merged into the seed ISO -that cloud-init reads on a cloud image's first boot. They're the only option -for the cloudinit backend: a cloud image's `packages:`/`runcmd:` machinery runs -once, at first boot, before stoat ever gets an ssh session, there's no "run -this over ssh" step to hook into. - -## Naming, and how the picker filters - -The filename is the whole contract. `internal/recipes/recipes.go` reads it -straight off the directory listing: there's no manifest, no registration -step: - -| Pattern | Kind | Offered to | -|---|---|---| -| `..sh` | shell recipe | that exact OS, on the apkovl/ssh backends | -| `.cloud.yaml` | shared cloud fragment | the OSes in the shared set (`ubuntu`, `debian`, `arch`), on the cloudinit backend | -| `..cloud.yaml` | per-OS cloud fragment | that exact OS, on the cloudinit backend; takes the place of the shared fragment for that OS | - -`List(osName, backend)` does the filtering: a shell recipe only ever shows up -for its exact OS, and cloud fragments only ever show up on the cloudinit -backend. `List("ubuntu", "cloudinit")` will never return `xfce.ubuntu.sh`, and -`List("alpine", "apkovl")` will never return `xfce.cloud.yaml`. If both a -per-OS fragment and the shared fragment could technically apply to one OS, -only the per-OS one is offered: you never see two "xfce" entries for the same -image. - -`Install()` copies the bundled recipes into the data root the first time stoat -needs them, and (this matters if you ever edit one) **it never overwrites an -existing file**. Drop your own version in place of a bundled recipe, or add a -new one, and a stoat upgrade leaves it alone. +Stoat installs bundled recipes into this directory. It records checksums in +`.manifest`; an unchanged bundled file can be refreshed by an upgrade, while a +hand-edited file is preserved. Remote recipes use a project cache at +`.stoat/recipes/` or a global cache under `~/.stoat/recipes/`. See +[Sharing recipes](sharing.md). ## When recipes run -- **Manually**, with `p` on the list or detail screen, against a VM's already- - selected recipes (chosen when you created the VM, or later, see the edit - form). -- **Offered automatically** after a start, once ssh comes up: stoat asks - ` is up, run , now? y/N` rather than running anything - unasked. A live VM gets asked every time (its root is wiped on every - reboot, so nothing survives to make asking again redundant); a disk or cloud - VM is only asked again if the last run didn't finish cleanly. -- **At first boot**, automatically, for cloud VMs: cloud-init applies the - merged `packages:`/`runcmd:` fragment before you ever get an ssh prompt. - There's nothing for `p` to do on a cloud VM; pressing it just tells you so. - -A disk VM with no OS installed yet has no ssh to provision over at all: -`p` refuses with a reminder to run the installer at the QEMU console first. -See [troubleshooting](../troubleshooting.md) if you hit that. +`stoat apply ` runs the VM's selected recipes after the VM is running and +reachable over SSH. `--only` restricts the run to names already present in the +VM's recipe list. `stoat apply --dry-run` reports the run or skip decision for +each selected recipe without contacting the guest. + +The TUI offers provisioning after SSH becomes reachable. A live VM is offered +again after every reboot because its root filesystem is temporary. A disk VM +is offered while it needs provisioning. Cloud-init recipes run from the seed +at first boot; stoat discovers their marker files if a later `apply` needs to +reconcile state. + +## Targeting and execution + +The manifest uses `os` to restrict a recipe to named guests and `requires` to +require guest capabilities. An empty `os` applies to every loaded guest. +Stoat must satisfy every entry in `requires`. The guest's `init` value is also +available as a capability. `stoat guest show ` displays the values. + +The default `stage` is `provision`. An `install` stage is accepted in the +manifest but is not executable; `stoat check-recipes` reports +`install-stage recipes are not yet supported`. Alpine disk installation is +handled by Stoat's unattended installer instead. + +The default `runtime` is `sh`, invoked as `sh -s`. `runtime = "python3"` is +invoked as `python3 -`; Stoat installs `python3` with the guest package +manager first when the guest does not provide it. The only accepted runtimes +are `sh` and `python3`. + +`depends` names recipes that must run first. The dependency must be in the VM's +recipe list, unless it was already applied. Stoat orders the run and rejects +cycles or unsatisfied dependencies. The TUI can add missing dependencies when +you select a recipe; the CLI requires you to include them in the recipe list. + +`run` defaults to `once`. `once` skips a recipe whose current script and +resolved non-secret parameters match its applied record. A changed script or +parameter runs again. `always` runs on every apply. `manual` runs only when its +name appears in `stoat apply --only `. The `auto` manifest field is +stored and shown by the manifest parser but does not control the TUI's +auto-provision decision; the TUI decides from VM mode and applied state. + +Set `reboot = true` when a recipe needs a disk VM to restart before its effect +is visible. Stoat performs one reboot after all recipes in the apply run have +succeeded. It does not reboot live VMs because a live root is temporary. ## Bundled recipes -| Recipe | Files | What it does | +The bundled set is closed and currently contains these four recipes: + +| Recipe | Supported guests | Purpose | |---|---|---| -| **xfce** | `xfce.alpine.sh`, `xfce.arch.sh`, `xfce.ubuntu.sh`, `xfce.debian.sh`, `xfce.cloud.yaml`, `xfce.fedora.cloud.yaml` | Installs an XFCE desktop. The shell recipes autologin root on tty1 and start X; the cloud fragments install `lightdm` and give you a graphical login screen instead, since a cloud image has a real user account (`stoat`), so there is someone to log in *as*. | -| **docker** | `docker.alpine.sh` | Installs Docker plus the compose plugin (`docker-cli-compose`, a separate package from `docker` on Alpine) and starts the daemon. | -| **devtools** | `devtools.alpine.sh`, `devtools.cloud.yaml` | The baseline for a throwaway VM: `git`, `curl`, `ca-certificates`, `vim`, `tmux`, `less`. The Alpine version also installs `bash` (Alpine defaults to `ash`) and `build-base`, its gcc/make/libc-dev meta-package. | -| **tailscale** | `tailscale.alpine.sh` | Installs Tailscale and starts `tailscaled`. | - -Two recipes have a cloud-config side, `xfce` and `devtools`. A shared -`.cloud.yaml` covers Ubuntu, Debian and Arch, because cloud-init's -`packages:` list is handed straight to the guest's own package manager with no -per-distro syntax, so a shared fragment only works where the names happen to -match on both apt and pacman. - -`devtools.cloud.yaml` deliberately drops the compiler toolchain that -`devtools.alpine.sh` installs: it is `build-essential` on apt, `base-devel` on -pacman and `@development-tools` on dnf: three names and three shapes for one -thing. Install your distro's own if you need it. - -Fedora is the recurring exception and stays out of the shared set entirely: - -- For xfce it has no package literally named `xfce4`, so it gets its own - `xfce.fedora.cloud.yaml` using the comps group `@xfce-desktop`. -- For devtools, its `vim` is packaged as `vim-enhanced`, so even the shared - fragment's names don't hold. - -Worth knowing if you're writing your own: **Arch has no `xfce4` package -either**: `xfce4` is a package *group* there. It works only because -`pacman -S` accepts a group name where apt would want a real package. Do not -assume a name that resolves on two distros resolves the same way on both. - -See [writing your own](writing-your-own.md) for the details if you're adding -another cross-distro recipe. - -### Tailscale installs but does not authenticate - -`tailscale.alpine.sh` installs the package and starts `tailscaled`, that's -where it stops. It does not run `tailscale up`, and it does not carry an auth -key. That's deliberate: stoat has nowhere to keep an auth key that isn't worse -than not having one. Put it in `vm.toml` and it sits in plaintext in the data -root; bake it into the recipe file and it ends up committed to git the first -time someone shares their recipes directory. Neither is acceptable, so the -recipe stops short and tells you the one command to run yourself: - -``` -tailscale installed and tailscaled running. -To join your tailnet, ssh in and run: tailscale up -(stoat does not store auth keys, see this recipe's header for why.) -``` +| `devtools` | Alpine, Ubuntu, Debian, Fedora, Arch | Git, compiler tools, editor and basic fetch tools | +| `docker` | Alpine, Ubuntu, Debian, Fedora, Arch | Docker engine and compose plugin; schema 3 parameter `user`, output `socket`, health check `docker info` | +| `tailscale` | Alpine, Ubuntu, Debian, Fedora, Arch | Install and start `tailscaled`; schema 3 required secret `authkey`, health check `tailscale version` | +| `xfce` | Alpine, Ubuntu, Debian, Arch | XFCE desktop with autologin startx on tty1; requests a disk-VM reboot | + +The scripts are in `internal/recipes/bundled/` in the source tree. Each +recipe has a manifest. Docker, devtools and Tailscale use OS-specific script +overrides because package names and repository setup differ. XFCE uses one +script with guest prelude verbs and therefore has no `[scripts]` table. + +Cloud images use their own package manager and a cloud-init seed. Cloud-init +currently wraps recipe bodies in shell commands; it does not perform the SSH +runtime bootstrap. Use `runtime = "sh"` for a cloud recipe. A `python3` +manifest is not installed or invoked by cloud-init. A shared package name must +resolve on every guest listed in `os`; use an OS-specific script when it does +not. + +## Recipe state + +After a successful apply, Stoat records the recipe version, script and input +hashes, outputs, health result and timestamp in the VM's `[applied]` table. +Secret values never enter that table or JSON output. A secret is represented as +`` or `` when Stoat displays state. + +Run a declared health check after applying a recipe with `stoat wait +--healthy`. A check exits with status 0 for healthy. If `health.timeout` is +omitted, Stoat uses 30 seconds. A recipe with no `[health]` table has no check. + +Continue with [Writing your own recipe](writing-your-own.md) or +[Sharing recipes](sharing.md). diff --git a/docs/recipes/sharing.md b/docs/recipes/sharing.md index 5b31dbd4..93968d92 100644 --- a/docs/recipes/sharing.md +++ b/docs/recipes/sharing.md @@ -1,63 +1,62 @@ # Sharing recipes -Remote recipes come from a Git repository with a `recipe.toml` at its root. -Stoat validates the manifest, pins the resolved commit in `stoat.lock`, and -keeps the checkout in `.stoat/recipes/` for a project or `~/.stoat/recipes/` -for the global scope. Git must be installed on the host. +A remote recipe is a Git repository with `recipe.toml` at its root. Stoat +validates the manifest and scripts, resolves a ref to a full commit, and +records that commit in `stoat.lock`. Git must be installed on the host. -## Add and search +## Find and add a recipe -Search the configured index by name or description: +The curated index is `index.toml` at the root of the Stoat repository. Stoat +clones it into its data root and refreshes the clone after 24 hours. Search by +name or description: ```sh -stoat recipe search my-tools +stoat recipe search docker +stoat recipe search --refresh ``` -Add an index entry by name. An index name does not prompt for confirmation: +Add an index entry by name. An optional `@ref` selects a tag or branch: ```sh stoat recipe add my-tools +stoat recipe add my-tools@v1.2 ``` -Add directly from a repository URL when the source is not in the index. A TTY -shows the manifest name, target OSes, requirements, and parameters before it -asks for confirmation. `-y` skips that prompt: +You can add a repository URL when it is not in the index. Stoat previews the +manifest in a terminal and asks for confirmation; `-y` skips that prompt: ```sh stoat recipe add https://github.com/example/stoat-my-tools@main -y ``` -The default index is `index.toml` at the root of the stoat repository, -`https://github.com/NovusEdge/stoat`, fetched as a shallow clone. To add a -recipe to it, open a pull request that adds an entry under `[recipes]`. Set -`STOAT_INDEX` to any Git repository or local directory that holds an -`index.toml` to use your own: +`--global` forces the home scope from inside a project. `--force` permits a +name collision with a bundled, local, or remote recipe. Use it only when the +replacement is intentional. -```sh -export STOAT_INDEX=/path/to/my-index -stoat recipe search my-tools -``` - -Index refreshes are cached for 24 hours; `--refresh` forces a new fetch. - -List installed recipes and their scope and short commit pin: +List installed recipes, their scope, and the short commit pin: ```sh stoat recipe list ``` -Search reads the configured index. `update` addresses an existing remote pin -by its plain name and does not search the index again. This also applies to a -recipe added from a URL: +The visible roots have these labels: -```sh -stoat recipe update my-tools -``` +| Scope | Location and ownership | +|---|---| +| `project` | `./.stoat/recipes/`, when the current directory contains `stoat.toml` | +| `global` | remote recipes pinned by `~/.stoat/stoat.lock` | +| `local` | user directories under `~/.stoat/recipes/` that are not bundled or globally pinned | +| `bundled` | recipes shipped by Stoat and recorded in `.manifest` | + +Project recipes shadow global, local and bundled recipes with the same name. +Within the home directory, a globally pinned remote recipe shadows a local or +bundled recipe. `stoat recipe show ` displays the visible contract. ## Project and global scopes If the current directory contains `stoat.toml`, recipe commands use project -scope. The declaration lives in its `[recipes]` table: +scope. Stoat does not search parent directories. Declare an index recipe by +ref, or declare a repository explicitly: ```toml [recipes] @@ -66,55 +65,70 @@ other-tools = { source = "https://github.com/example/stoat-other-tools", ref = " ``` Project scope writes `./stoat.lock` and caches checkouts under -`./.stoat/recipes/`. Stoat adds `.stoat/` to `.gitignore` when the directory -is a Git checkout. Commit both `stoat.toml` and `stoat.lock` so another -checkout can reproduce the same recipe commits. +`./.stoat/recipes/`. In a Git checkout Stoat adds `.stoat/` to `.gitignore`. +Commit both `stoat.toml` and `stoat.lock` so another checkout uses the same +commits. Pass `--global` to `add`, `lock`, `sync`, or `rm` to force global +scope. `update` has no `--global` flag and uses the active scope. -Without `stoat.toml`, commands use the global lock at `~/.stoat/stoat.lock` -and cache at `~/.stoat/recipes/`. Pass `--global` to force global scope from a -project directory. Stoat does not search parent directories for a project -file. +Without a project file, the global lock is `~/.stoat/stoat.lock` and the cache +is `~/.stoat/recipes/`. Global scope has no declaration file; its lock entries +are the source of truth. -## Lock, sync, update, and remove +## Lock and synchronize -Resolve every project declaration to a commit without changing the cache: +In project scope, `lock` resolves each declaration to a fresh commit and +writes only the lock. It does not populate the cache: ```sh stoat recipe lock ``` -Populate the cache from the lock, removing project cache entries no longer in -the lock: +`sync` makes the cache match the lock. It validates every checkout, replaces +missing or stale clean checkouts, and removes project cache directories that +are absent from the lock: ```sh stoat recipe sync ``` -Fetch refs again and repin one recipe, or every remote recipe when no name is -given: +An existing checkout with uncommitted changes is refused. Copy it to a local +recipe before changing it. Global sync leaves unrelated local recipes in the +home recipes directory alone. + +`apply`, `apply --dry-run`, and recipe listing take a coordinated snapshot. +For a project, a stale declaration or lock reports an instruction to run +`stoat recipe lock`; a missing or mismatched clean cache is repaired from the +lock before apply. A dirty checkout remains an error. + +## Update and remove + +`update` fetches the ref stored in the lock and repins it. It accepts one or +more plain names, or no names to update every remote recipe: ```sh stoat recipe update my-tools stoat recipe update ``` -Remove a remote recipe after checking that no VM uses it: +It does not search the index again. A checkout with local changes is refused. + +`rm` removes a remote declaration, lock entry, and checkout. It asks for +confirmation unless `-y` is present. It refuses when a VM lists the recipe; +`--force` removes it despite those references: ```sh stoat recipe rm my-tools -y +stoat recipe rm my-tools --force -y ``` -If an add would replace a bundled, local, or same-scope recipe, it refuses -unless the replacement is intentional: - -```sh -stoat recipe add my-tools --global --force -``` +`rm` is scoped to the current project when one is active. The MCP +`remove_recipe` tool has no force option and refuses a recipe still used by a +VM. -For removal, use `--force` only when intentionally removing a recipe listed by -a VM. A recipe checkout with local changes is never overwritten by update or -sync; copy it to a local recipe first. +## Lock file shape -`apply` in project scope checks that declarations, lock entries, and cache -checkouts agree. A stale declaration reports a repair instruction to run -`stoat recipe lock`; a missing or changed cache is synchronized before apply. +`stoat.lock` is generated by Stoat. It contains schema 1 and one entry per +remote recipe. Each entry stores `source`, the requested `ref`, the resolved +40-character hexadecimal `commit`, and the original `added` timestamp. +Treat it as generated data and commit it; do not hand-edit it to change a +recipe ref. Change the declaration, then run `stoat recipe lock`. diff --git a/docs/recipes/writing-your-own.md b/docs/recipes/writing-your-own.md index fe217baa..249c5c75 100644 --- a/docs/recipes/writing-your-own.md +++ b/docs/recipes/writing-your-own.md @@ -1,152 +1,229 @@ # Writing your own recipe -Authoring a recipe needs no code change. Drop a correctly-named file in -`~/.stoat/recipes/` and it appears in the picker the next time you open it: -`recipes.List` just reads the directory. Because `Install()` never overwrites -an existing file, your recipe (and any edit you make to a bundled one) survives -a stoat upgrade. +Use `stoat recipe new ` to create a recipe directory from the annotated +sample. It creates `recipe.toml`, the default script, and scripts named by the +sample's `[scripts]` table. Edit the files, then select the recipe for a VM +whose guest matches its `os` and `requires` fields. -See [the overview](overview.md#naming-and-how-the-picker-filters) for the full -naming table. The short version for a shell recipe: name it -`..sh`, e.g. `~/.stoat/recipes/postgres.alpine.sh`, and it's offered -to Alpine VMs on the apkovl/ssh backends. +```sh +stoat recipe new tools --os alpine +stoat recipe show tools +stoat recipes --os alpine +``` -## The required shape +The generated directory is under `~/.stoat/recipes/` (or the `recipes/` +directory below `STOAT_HOME`). Project caches are reserved for remote recipes +managed by `recipe lock` and `recipe sync`. `recipe new` accepts `--backend +cloudinit` for compatibility, but recipes always use the manifest directory +format and shell scripts. + +## Manifest + +The manifest needs `name` and `script`. `schema` defaults to 2. Set +`schema = 3` to use parameters, outputs, or health checks. + +```toml +schema = 3 +name = "tools" +description = "Install the tools used by this VM" +os = ["alpine", "ubuntu"] +requires = [] +stage = "provision" +script = "install.sh" +runtime = "sh" +run = "once" +reboot = false +depends = [] + +[params.editor] +type = "string" +default = "vi" +help = "editor command to install" + +[params.port] +type = "int" +default = 8080 + +[params.debug] +type = "bool" +default = false + +[params.channel] +type = "enum" +values = ["stable", "test"] +default = "stable" + +[params.token] +type = "secret" +required = true +help = "token used by the installer" + +[outputs] +installed_path = "path of the installed tools" + +[health] +check = "tools --version" +timeout = "30s" +``` -Every bundled shell recipe starts the same way, and yours should too: +Manifest rules: + +| Field | Rule | +|---|---| +| `name` | Required recipe identifier. Use a simple name without spaces, slashes, backslashes, a leading dot, `.` or `..`; Stoat also rejects an empty name. `recipe new` additionally rejects dots in the name. | +| `description` | Optional text shown by `recipes` and `recipe show`. | +| `schema` | 2 or 3. A missing value means 2. Parameters, outputs and health require 3. | +| `os` | Optional list of guest names. Empty means every loaded guest. | +| `requires` | Optional capability list. Every capability must match the guest. | +| `stage` | `provision` (default) is supported. `install` is parsed but cannot run. | +| `script` | Required path relative to the recipe directory. | +| `scripts` | Optional OS or guest-alias overrides of `script`. | +| `runtime` | `sh` (default) or `python3`; SSH provisioning invokes `sh -s` or `python3 -`. | +| `run` | `once` (default), `always`, or `manual`. | +| `auto` | Accepted manifest metadata. The current TUI auto-provision decision does not read it. | +| `reboot` | Reboot a disk VM once after the apply run succeeds. Live VMs are not rebooted. | +| `depends` | Recipe names that must run first and must be in the VM's recipe list or already applied. | + +## Parameters and secrets + +Parameter names match `[a-z][a-z0-9_]*`. Supported types are `string`, `int`, +`bool`, `enum`, and `secret`. Every non-secret parameter needs a `default` or +`required = true`. An enum needs a non-empty `values` list, and its default +must be one of those values. A secret cannot have a default. + +Stoat resolves a value from the VM's stored non-secret override or the +manifest default. It reads secrets from the VM's `secrets.toml`; for a +project-declared VM, the project source is `.stoat/secrets.toml` and Stoat +reconciles those values into the VM data directory. Secret files are mode +0600 and values are never included in `vm.toml`, the applied record, or JSON +output. A required value with no value fails before the script runs. + +The script receives resolved values as environment variables named +`STOAT_PARAM_`, plus `STOAT_RECIPE`. For example, `editor` +becomes `STOAT_PARAM_EDITOR`. Secret parameters are also passed to the guest +through that environment, so do not print them. + +Set values when creating or updating a VM: ```sh -#!/bin/sh -set -e +stoat create tools-vm --image alpine-virt --recipes tools \ + --set tools.editor=vim --secret tools.token +stoat update tools-vm --set tools.port=9090 +stoat update tools-vm --unset tools.port ``` -`set -e` matters more here than in a script you'd run locally: a recipe runs -unattended, streamed into `sh -s` over ssh -(`internal/sshx/sshx.go`'s `Provision`), with its output going straight to -`last-provision.log`. Without `set -e`, a failed package install just scrolls -past and the recipe reports success anyway. +Interactive `--secret` reads a value without displaying it. In JSON mode, +provide `STOAT_SECRET_TOOLS_TOKEN` in the environment because JSON mode never +prompts. `--unset` clears a non-secret override and restores its manifest +default; it does not clear a secret. -The recipe runs as `root`: there's no `sudo` to reach for, and none is -installed on Alpine by default. +## Outputs and health -On Alpine, enable the community repository before installing anything from it, -most non-base packages (Docker, Tailscale, `build-base`) live there, not in -main: +Stoat creates `STOAT_OUTPUT` for each recipe run. Write one `name=value` line +per output. Stoat records declared and undeclared output names after a +successful run; undeclared names are reported in the apply log. It removes +the temporary guest output file after reading it. ```sh -setup-apkrepos -c -1 +#!/bin/sh +set -e + +stoat_pkg_setup +stoat_pkg_install htop +printf 'installed_path=%s\n' /usr/bin/htop >> "$STOAT_OUTPUT" ``` -`-c` turns on community; `-1` picks the fastest mirror and refreshes the -package indexes in the same step, so a separate `apk update` afterward would -just be redundant work that widens the window for a transient network drop to -kill the whole recipe under `set -e`. +`[health] check` runs after the recipe's apply step. The command exits 0 for a +healthy result. `timeout` is a positive Go duration such as `30s`; it defaults +to 30 seconds when a check exists. `stoat wait --healthy` waits for every +applied recipe that declares a check and records `ok`, `failed`, or `unknown` +health in the VM status. + +## Guest prelude + +For a loaded guest, Stoat prepends a shell prelude. Use these portable verbs +when one script supports several package managers or init systems: + +| Name | Effect | +|---|---| +| `stoat_pkg_setup` | Refreshes the package index when the guest defines a setup command. | +| `stoat_pkg_install ` | Installs packages with the guest's package manager. | +| `stoat_svc_enable ` | Enables a service at boot. | +| `stoat_svc_start`, `stoat_svc_stop`, `stoat_svc_restart`, `stoat_svc_status` | Controls one service. | +| `STOAT_OS` | Loaded guest name. | +| `STOAT_INIT` | Guest init value, such as `systemd` or `openrc`. | +| `STOAT_PKGMGR` | Package manager basename, such as `apk` or `apt-get`. | -## The live-vs-disk honesty block +Recipes run as the VM's SSH user, with the guest's escalation command when +that user is not root. Bundled guests provide the appropriate commands. A +recipe that needs root should use the service and package verbs instead of +assuming `sudo` exists. -This is the one thing every bundled shell recipe carries, and -`recipes_test.go` enforces it (`TestShellRecipesAreHonestAboutLiveVsDiskPersistence`) -for every file in the bundle. It exists because of a real trap: a **live** -Alpine VM boots into a diskless mode where the root filesystem is a -`tmpfs`/`overlay` mount in RAM. Anything your recipe installs or edits there, -packages, `/etc` files, `/root/.profile`, is gone the moment the VM reboots. -A **disk** VM, by contrast, has a real block-device root that persists -normally. The same recipe file runs against both, over the same kind of ssh -session, so it has to check which one it landed on rather than assume: +## Persistence and scripts for several guests + +A live Alpine VM stores its root in a temporary `tmpfs` or `overlay`; package +changes and files disappear after reboot. A disk VM keeps those changes. State +the behavior in a recipe that changes the guest: ```sh root_fstype=$(awk '$2 == "/" { print $3 }' /proc/mounts) - case "$root_fstype" in tmpfs | overlay) - echo "NOTE: this is a live VM (root is $root_fstype, in RAM). Everything installed above is gone after a reboot; rebooting will NOT bring it back. Use a disk VM to keep it." + echo "NOTE: this is a live VM (root is $root_fstype, in RAM); changes are lost after reboot." ;; *) - echo "installed on a disk VM (root is $root_fstype): this survives a reboot." + echo "installed on a disk VM (root is $root_fstype); changes survive reboot." ;; esac ``` -Skip this and your recipe can end up promising something false: "reboot to -get your desktop" is a lie on a VM whose root is tmpfs. If your recipe does -something that needs to appear *right now* on a live VM rather than after a -reboot that will never come (the way `xfce.alpine.sh` sends `kill -HUP 1` to -make init respawn tty1 immediately), put that in the `tmpfs | overlay` branch -too, see that file for the full pattern. - -## A complete example +Use `[scripts]` when package names or setup differ: -A recipe that installs `htop` and `ncdu` on Alpine, following the same shape -as the bundled ones: +```toml +os = ["alpine", "ubuntu", "debian", "fedora", "arch"] +script = "install.sh" -```sh -#!/bin/sh -# Installs htop and ncdu. Runs as root over ssh on a booted Alpine VM. -set -e - -setup-apkrepos -c -1 -apk add htop ncdu +[scripts] +alpine = "install-alpine.sh" +ubuntu = "install-debian.sh" +debian = "install-debian.sh" +fedora = "install-fedora.sh" +arch = "install-arch.sh" +``` -root_fstype=$(awk '$2 == "/" { print $3 }' /proc/mounts) +`stoat recipes --os ` lists matching recipes. `stoat check-recipes` +explains why a named recipe does not match. The `--backend` flag is accepted +for compatibility; recipe applicability is determined by the guest OS and +manifest requirements. The VM backend determines how the selected script is +executed, not whether its manifest matches. -case "$root_fstype" in -tmpfs | overlay) - echo "NOTE: this is a live VM (root is $root_fstype, in RAM). Everything installed above is gone after a reboot; rebooting will NOT bring it back. Use a disk VM to keep it." - ;; -*) - echo "installed on a disk VM (root is $root_fstype): this survives a reboot." - ;; -esac -``` +## Cloud-init recipes -Save that as `~/.stoat/recipes/tools.alpine.sh` and it shows up in the picker -as `tools` (the picker strips the `.alpine.sh` suffix, see -`internal/tui/labels.go`'s `recipeLabel`) for any Alpine VM on the apkovl or -plain-ssh backend. +Cloud-mode VMs run selected recipe scripts from their first-boot NoCloud seed. +Stoat writes each script and executes the scripts in selection order, then +writes a marker after each successful script. A later `apply` can discover +those markers and populate the VM's applied state. -## Cloud fragments +Cloud-init currently wraps recipe bodies in a shell command and does not run +the SSH runtime bootstrap. Use `runtime = "sh"` for a cloud recipe; a +`python3` manifest does not install Python or invoke `python3` in cloud mode. +Cloud-init secrets are written to a temporary mode-0600 file in the guest and +removed after the seed commands finish. -A cloud fragment is a `#cloud-config` document, not a shell script. It's -merged into the seed's `user-data` (`internal/cloudinit/cloudinit.go`), which -splices out just the `packages:` and `runcmd:` lists from each selected -fragment and concatenates them: that's the only shape it understands, so stick -to those two top-level keys: +## Check and apply -```yaml -#cloud-config -packages: - - htop - - ncdu +Inspect the contract, syntax-check a script with the guest shell, then plan an +apply before running it: -runcmd: - - echo "tools installed" +```sh +stoat recipe show tools +sh -n ~/.stoat/recipes/tools/install.sh +stoat apply tools-vm --dry-run +stoat apply tools-vm --only tools +stoat wait tools-vm --healthy ``` -Name it `tools.cloud.yaml` for the shared cross-distro fragment, or -`tools..cloud.yaml` for one that targets a single OS. - -### The trap: one fragment doesn't always cover every distro - -`packages:` is handed straight to the image's native package manager (`apt` -on Ubuntu/Debian, `pacman` on Arch) with no per-distro syntax. A shared -fragment only works if the package name happens to be spelled identically -everywhere it's offered. `xfce.cloud.yaml` covers Ubuntu, Debian, and Arch -this way because `xfce4` is a real package (or group name) on both apt and -pacman. Fedora broke that: it has no package literally named `xfce4` at all, -the desktop is the comps group `@xfce-desktop-environment`, installed via -`dnf install @xfce-desktop-environment`. Cramming an `@group` token into the -shared fragment would've worked on Fedora and silently failed everywhere else -it's offered, so Fedora gets its own file, `xfce.fedora.cloud.yaml`, and -`recipes.List` offers it *instead of* the shared one for Fedora specifically, -see [the naming table](overview.md#naming-and-how-the-picker-filters). - -If you're writing a cross-distro fragment of your own, check every OS you're -offering it to against that OS's actual package manager before assuming one -name works everywhere. - -## Tooling for this is still just a proposal - -There's no `stoat recipe new` or manifest format yet: writing a file by hand -in `$EDITOR` is the whole workflow today. A design for scaffolding/validating -recipes has been sketched but not built; see -[../recipe-authoring-spec.md](../recipe-authoring-spec.md) for that proposal. +`stoat apply` requires a running VM. It stops at the first failed recipe and +does not mark that recipe as applied. A dependency cycle, missing manifest, +invalid parameter, dirty remote checkout, or stale project lock fails before +the guest run begins. diff --git a/docs/reference/cli.md b/docs/reference/cli.md index b1c91105..0c1c1540 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -10,7 +10,7 @@ usage: stoat [flags] ## Global flags -- **`--json`** turns on machine output: one JSON object per line on stdout, errors included, never prose. It implies `--quiet` and never prompts (so `rm` without `-y` fails instead of asking). It is recognized anywhere in argv before kong ever parses flags, with one exception: for `exec`, only before the VM name, so `stoat exec work ls --json` still sends `--json` to the guest. This file only covers the human-facing CLI; the JSON shapes themselves are in [json.md](json.md). +- **`--json`** turns on machine output for a named VM operation: one JSON object per line on stdout, errors included, never prose. It implies `--quiet` and never prompts (so `rm` without `-y` fails instead of asking). Project fan-out commands currently have a known limitation: no-name `up`, `down`, and `apply` can print progress prose before their terminal JSON result. Use a named VM command when a clean stream is required. The flag is recognized anywhere in argv before kong ever parses flags, with one exception: for `exec`, only before the VM name, so `stoat exec work ls --json` still sends `--json` to the guest. This file only covers the human-facing CLI; the JSON shapes themselves are in [json.md](json.md). - **`-q`, `--quiet`, `--no-interactive`** are three names for one flag, present on every subcommand. Where it has an effect, it suppresses "in-progress" chatter (`starting work...`, `provisioning work...`, ...); final results and all errors print regardless. - **`-h`, `--help`** prints the command's usage and flags and exits 0. `stoat help` (the subcommand) prints the same top-level text `stoat --help` does. - **`-v`, `--version`** prints `stoat ` and exits 0. It is matched as the **first argument only**, before any parsing (`cmd/stoat/main.go`), which has two consequences worth knowing: `stoat -v --json` prints plain text and ignores `--json`, and `stoat --json --version` is a usage error because `-v` is no longer first. **Scripts and machine consumers should use the `version` subcommand**, which behaves normally under `--json`. @@ -38,7 +38,7 @@ order, when given no VM argument. A bare VM argument resolves against | [`wait`](#stoat-wait-name) | Block until a VM reaches a state | 0, 1, 2 | | [`rm`](#stoat-rm-name--y) | Delete a VM | 0, 1 | | [`clone`](#stoat-clone-source-name) | Copy a VM: overlay disk, fresh ssh port, no forwards | 0, 1 | -| [`exec`](#stoat-exec-name-command-) | Run a command in a VM, verbatim | 0-255, see below | +| [`exec`](#stoat-exec-name-command) | Run a command in a VM, verbatim | 0-255, see below | | [`ssh`](#stoat-ssh-name) | ssh into a VM, replacing this process | 0, 1, 2 | | [`ssh-command`](#stoat-ssh-command-name) | Print the ssh argv instead of running it | 0, 1 | | [`cp`](#stoat-cp-source-dest) | Copy a file in or out; one side is `:` | 0, 1, 2 | @@ -53,6 +53,12 @@ order, when given no VM argument. A bare VM argument resolves against | [`recipe list`](#stoat-recipe-list) | List installed recipes and where they live | 0, 1 | | [`recipe show`](#stoat-recipe-show-name) | Show one recipe's parameter and output contract | 0, 1 | | [`recipe new`](#stoat-recipe-new-name) | Scaffold a recipe in the recipes directory | 0, 1 | +| [`recipe search`](#stoat-recipe-search-term) | Search the curated remote recipe index | 0, 1 | +| [`recipe add`](#stoat-recipe-add-ref) | Add a remote recipe to the active scope | 0, 1 | +| [`recipe lock`](#stoat-recipe-lock---global) | Resolve project recipe refs to commits | 0, 1 | +| [`recipe sync`](#stoat-recipe-sync---global) | Synchronize a recipe cache to its lock | 0, 1 | +| [`recipe update`](#stoat-recipe-update-names---global) | Repin remote recipes to current refs | 0, 1 | +| [`recipe rm`](#stoat-recipe-rm-name--y) | Remove a remote recipe | 0, 1 | | [`guest ls`](#stoat-guest-ls) | List loaded guest OS definitions | 0 | | [`guest show`](#stoat-guest-show-name) | Print one guest's merged definition | 0, 1 | | [`logs`](#stoat-logs-name--n-n) | Tail a VM's log, or stoat's own | 0, 1 | @@ -147,7 +153,7 @@ created work (alpine, live, ssh port 2222) start it with: stoat up work ``` -Flags: `--image` (required; catalog id or a path to your own image), `--os`, `--backend` (override what a bring-your-own image's filename would otherwise infer), `--mode` (`live` or `disk`; only meaningful for the alpine iso, every other image has one mode), `--ram` (MB), `--cpus`, `--disk` (absolute size, e.g. `8G`), `--share` (host directory to expose), `--console-password` (`random` generates one), `--recipes` (comma-separated or repeated), `--set recipe.param=value` (set a non-secret recipe parameter), `--secret recipe.param` (read a secret from the environment or prompt), `--allow-exec` (default true; `--allow-exec=false` opts this VM out of `exec`/`copy_to`/`copy_from`, enforced by the MCP server rather than stoat itself). +Flags: `--image` (required; catalog id or a path to your own image), `--os`, `--backend` (override what a bring-your-own image's filename would otherwise infer), `--mode` (`live` or `disk`; only meaningful for the alpine iso, every other image has one mode), `--ram` (MB), `--cpus`, `--disk` (absolute size, e.g. `8G`), `--share` (host directory to expose), `--console-password` (`random` generates one), `--recipes` (comma-separated or repeated), `--set recipe.param=value` (set a non-secret recipe parameter), `--secret recipe.param` (read a secret from the environment or prompt), `--agent-access` (`none`, `observe`, `manage`, or `exec`; default `manage`, controls MCP guest access). The hidden `--allow-exec` flag remains as a compatibility alias: true maps to `exec`, false to `manage`. `create` (alias `new`) refuses at project scope: `a stoat.toml is present; declare the VM there and run stoat up, or pass --global`. `--global` creates the VM outside the project. @@ -251,6 +257,11 @@ work stopped At project scope, `` is optional: with no name, every declared VM is stopped in declaration order, and a failure stops the run and reports every later VM as skipped. +The stop request can return while QEMU is still exiting. Use +`stoat wait --until stopped` when a script needs confirmed termination. +Under project fan-out, add the VM name to keep the JSON output machine-readable; +no-name `down --json` can include progress prose before its result. + **Exit codes:** 0 on success; 1 if the VM can't be loaded, is broken, isn't running, or fails to stop. ## `stoat wait ` @@ -477,13 +488,17 @@ work: recipes applied At project scope, `` is optional: with no name, every declared VM's recipes run in turn, in declaration order, and a failure stops the run. -**Exit codes:** 0 on success; 1 if the VM can't be loaded, the run fails, or (for a cloud-mode VM) recipes were already applied at boot rather than by this command. +**Exit codes:** 0 on success; 1 if the VM cannot be loaded, is not running, or the apply fails. Cloud VMs support the same apply path over SSH; recipe run modes determine which scripts run or are skipped. `provision` is a hidden alias of `apply`: `stoat provision work` behaves exactly like `stoat apply work`, and reports `"cmd":"apply"` under `--json`. ## `stoat recipes` -Lists recipes, optionally filtered to ones applicable to a guest OS and/or backend. +Lists recipes, optionally filtered to ones applicable to a guest OS and/or +backend. Applicability is determined by the guest OS, manifest `os`, and +manifest `requires`; the backend controls execution after selection. The +`--backend` flag remains accepted for compatibility with older callers but is +not an additional manifest-match condition. ``` $ stoat recipes --os alpine --backend apkovl @@ -494,7 +509,8 @@ tailscale Tailscale daemon, installed and started (join man xfce XFCE desktop with autologin startx on tty1 ``` -`--os` alone means "what that OS gets" (its own backend is inferred); `--backend` alone means "every OS on that backend"; both together is the exact filter; neither is the full catalog. +`--os` selects a guest OS. `--backend` is accepted but does not narrow the +manifest match. With neither flag, the command lists the full catalog. **Exit codes:** 0 on success; 1 if the recipe list can't be built. @@ -503,29 +519,36 @@ xfce XFCE desktop with autologin startx on tty1 Reports, for each named recipe, why it would **not** apply to the given OS/backend; an empty result means every one of them would. ``` +$ stoat check-recipes xfce --os fedora --backend cloudinit +xfce: xfce is not offered to fedora/cloudinit $ stoat check-recipes docker xfce --os debian --backend cloudinit -docker: docker is not offered to debian/cloudinit -$ stoat check-recipes xfce --os alpine --backend apkovl all applicable ``` -`--os` is required; `--backend` narrows further. +`--os` is required. `--backend` is accepted for compatibility and is included +in the diagnostic wording, but it does not add a manifest applicability filter. **Exit codes:** 0 on success, whether or not any recipe turned out inapplicable (an inapplicable recipe is a valid answer, not a failure); 1 if the check itself fails; 2 if no recipe names are given. ## `stoat recipe list` -Lists recipes installed under stoat's recipes directory and prints where that directory is. +Lists every visible recipe in shadow order and prints each recipe's scope and +remote commit when available. Project scope is active only when the current +directory contains `stoat.toml`. The JSON form also includes the roots in +search order. ``` $ stoat recipe list -/home/user/.stoat/recipes - devtools - docker - tailscale - xfce +NAME SCOPE COMMIT DESCRIPTION +devtools bundled git, a compiler, an editor and basic fetch tools +docker bundled Docker engine and the compose plugin +tailscale bundled Tailscale daemon, installed and started (join manually) +xfce bundled XFCE desktop with autologin startx on tty1 ``` +The bundled index currently has no remote entries. Use `recipe search` to +inspect the curated index before adding a remote recipe by name. + **Exit codes:** 0 on success; 1 if the directory can't be read. ## `stoat recipe show ` @@ -574,6 +597,68 @@ VM and guest samples are [here](samples/vm.toml) and **Exit codes:** 0 on success; 1 if the recipe can't be created (e.g. the name is already taken). +## `stoat recipe search [term...]` + +Searches the curated remote index by recipe name and description. With no +terms it lists every index entry. `--refresh` refreshes the local index clone +before searching; the clone is otherwise reused for 24 hours. + +```sh +stoat recipe search docker +stoat recipe search --refresh +``` + +The current shipped index may be empty. An index name can be passed to +`recipe add`; a Git URL is also accepted by `recipe add` but is not accepted by +the MCP `add_recipe` tool. + +## `stoat recipe add ` + +Adds a remote recipe to the active scope. `` is an index name, an index +name with `@tag-or-branch`, or a Git URL with an optional `@ref`. + +```sh +stoat recipe add my-tools +stoat recipe add my-tools@v1.2 +stoat recipe add https://github.com/example/stoat-my-tools@main -y +``` + +An index name does not prompt. A Git URL previews the manifest and asks for +confirmation on a terminal; use `-y` for a non-interactive call. `--global` +selects the global lock and cache from a project. `--force` allows a remote +recipe to replace an existing bundled, local, or remote name. + +In project scope, the declaration is written to `stoat.toml` and the lock is +updated by `recipe lock`; `recipe sync` then populates `.stoat/recipes/`. +Without a project file, global scope records the source in +`~/.stoat/stoat.lock` and checks out under `~/.stoat/recipes/`. + +## `stoat recipe lock [--global]` + +Resolves each active project declaration to a full 40-character commit and +writes `stoat.lock`. It does not populate the recipe cache. In global scope, +the existing global lock is repinned. A stale or missing project declaration +must be locked before project recipe operations can proceed. + +## `stoat recipe sync [--global]` + +Makes the active cache match its lock. Missing or mismatched clean checkouts +are replaced, and project cache directories absent from the lock are removed. +A dirty checkout is refused; copy it to a local recipe before editing. + +## `stoat recipe update [names...] [--global]` + +Fetches the stored ref and repins it to a new commit. With no names, all +remote recipes in the active lock are updated. It does not search the index +again and refuses a dirty checkout. + +## `stoat recipe rm [-y]` + +Removes a remote recipe's declaration, lock entry, and checkout. It refuses a +recipe still selected by a VM unless `--force` is supplied. Confirmation is +required unless `-y` is supplied; `--json` also requires `-y` because it never +reads stdin. + ## `stoat guest ls` Lists every loaded guest OS: bundled definitions from `internal/guest/bundled/*.toml`, plus any `~/.stoat/guests/*.toml` merged over them. @@ -632,7 +717,7 @@ $ stoat logs -n 20 ## `stoat screenshot [-o path]` -Writes the VM's screen to a PNG and prints the path, the pixel size and the byte count. qemu dumps its own framebuffer over the monitor socket, so the image is the same whether the display is a GTK window or a VNC socket, and a VM stuck at a boot prompt still answers. +Writes the VM's screen to a PNG and prints the path, pixel size, and byte count. QEMU sends its framebuffer through the QMP socket with `screendump`. This works with a GTK window or VNC display, including when the guest is at a boot prompt. ``` $ stoat screenshot work @@ -687,12 +772,12 @@ Prints the full usage message (subcommands, global flags, exit codes) to stdout. | `1` | Runtime failure | Unknown VM name, VM already stopped for `down`, VM running for `rm`, ssh unreachable during provision, `doctor` found an issue, `rm` confirmation declined | | `2` | Usage error | Unknown subcommand, missing/extra arguments, an unparseable flag, `update` given no flags, `check-recipes` given no names | -A usage error (2) always prints both the specific complaint and the full usage text to stderr; a runtime failure (1) prints only `stoat: : ` to stderr. `exec` is the one command whose exit code, without `--json`, is neither: it is the guest's own status, 0-255 (see [`stoat exec`](#stoat-exec-name-command-)). +A usage error (2) always prints both the specific complaint and the full usage text to stderr; a runtime failure (1) prints only `stoat: : ` to stderr. `exec` is the one command whose exit code, without `--json`, is neither: it is the guest's own status, 0-255 (see [`stoat exec`](#stoat-exec-name-command)). ## Scripting - **`-q`, `--quiet`, `--no-interactive`** are three names for the same flag, present on every subcommand. Where it has an effect, it suppresses the "in-progress" chatter (`starting work...`, `provisioning work...`, ...); final results and all errors print regardless of this flag. - **`rm`** also treats `--no-interactive`/`-q`/`--json` as "there is no one to answer a confirmation prompt": without `-y` it refuses rather than blocking on stdin. -- **`--json`** is the machine-readable mode: one JSON object per line on stdout, errors included, implying `--quiet` and never prompting. See [json.md](json.md) for the wire format. +- **`--json`** is the machine-readable mode for named VM commands: one JSON object per line on stdout, errors included, implying `--quiet` and never prompting. Project fan-out `up`, `down`, and `apply` can currently add progress prose before the result; see [json.md](json.md) for the wire format and workaround. - **`NO_COLOR`** (any non-empty value) disables ANSI color in `ls`'s output. - Color is also **disabled automatically whenever stdout is not a terminal** (checked via `os.ModeCharDevice`), so piping `stoat ls` into `awk`, `grep`, or a file never carries escape codes even without setting `NO_COLOR`. Only `ls`'s `STATE` column is ever colored. diff --git a/docs/reference/json.md b/docs/reference/json.md index 43fda11f..492ba31c 100644 --- a/docs/reference/json.md +++ b/docs/reference/json.md @@ -1,8 +1,10 @@ # JSON Output Reference -`--json` turns any subcommand into a machine interface, so everything a +`--json` turns a named VM command into a machine interface, so everything a caller would otherwise regex, guess at, or reconstruct is defined here -instead. +instead. Project fan-out currently has one exception: no-name `up`, `down`, +and `apply` can write progress prose before their terminal JSON result. Use a +named VM invocation when the complete stdout stream must be JSON. This document is the contract. The human-facing CLI is documented in [cli.md](cli.md). @@ -18,7 +20,7 @@ stoat --json ls {"v":3,"type":"result","cmd":"ls","ok":true,"data":{"vms":[...]}} ``` -## The consumer contract, in one page +## The consumer contract for a named VM command ```python proc = subprocess.run(argv, stdout=PIPE, stderr=DEVNULL) @@ -37,7 +39,7 @@ return result["data"] Note what this does **not** do: branch on the exit code for anything except "was there a result at all". That is the intended shape. -Four rules make it work: +Four rules make it work for a named VM command: 1. **Every line of stdout is one JSON object.** Nothing else is ever written to stdout in `--json` mode, including a recipe's own output, which is @@ -50,6 +52,15 @@ Four rules make it work: 4. **Either you get a result line, or the process died.** The second case is detectable as "exit code is nonzero and no result line was seen". +Project fan-out is a current limitation. With a project `stoat.toml` and no +VM name, `up`, `down`, and `apply` force their internal fan-out path and may +write human progress lines before the final `ProjectRun` result. A parser that +must use fan-out should capture the final result line only after handling this +known defect; prefer `stoat --json` for a clean stream. A +single-VM `down` result can also report `state: "running"` while QEMU is still +exiting; follow it with `stoat wait --until stopped --json` when +termination must be confirmed. + Rule 3 is not a preference. A consumer that must merge two pipes to reconstruct one result will eventually interleave them wrong, and a naive `subprocess` read of two pipes in sequence deadlocks when either buffer fills. @@ -195,7 +206,7 @@ VM {"name":"work","os":"alpine","mode":"cloud","backend":"cloudinit", "share":"/home/u/src","recipes":["xfce"], "ssh_port":2200,"ssh_user":"stoat","installed":false, "forwards":[{"host_port":8080,"guest_port":80}], - "allow_exec":true,"agent_access":"manage","display":"vnc", + "allow_exec":false,"agent_access":"manage","display":"vnc", "error":"only on a broken VM", "project":"/home/u/myrepo","key":"dev","project_missing":false} @@ -243,7 +254,7 @@ RecipeRemoved {"name":"tailscale","scope":"global"} IndexEntry {"name":"tailscale","source":"https://github.com/x/stoat-tailscale", "description":"join a tailnet on boot","os":["alpine"]} -RecipeIssue {"name":"docker","reason":"docker is not offered to debian/cloudinit"} +RecipeIssue {"name":"xfce","reason":"xfce is not offered to fedora/cloudinit"} ApplyPlan {"name":"xfce","action":"run","reason":"never applied", "version":"1.2"} @@ -594,9 +605,10 @@ The same version also adds `agent_access` to `VM`, additive alongside neither bumped the version on its own; they are noted here only because they landed in the same branch as the `recipe list` change. -The project-file plan adds `init` and `status` commands, `--project` on `ls`, -and a no-argument fan-out on `up`, `down`, `apply`, `wait` and `rm` at project -scope, plus `project`, `key` and `project_missing` on `VM`, and five new MCP -tools (`project_status`, `project_up`, `project_down`, `project_apply`, -`project_wait`) alongside `start`, `stop`, `apply_recipes` and `wait`, which -keep their existing inputs and outputs. All additions; the contract stays 3. +The project-file release added `init` and `status` commands, `--project` on +`ls`, and a no-argument fan-out on `up`, `down`, `apply`, `wait` and `rm` at +project scope. It also added `project`, `key` and `project_missing` on `VM`, +and five MCP tools (`project_status`, `project_up`, `project_down`, +`project_apply`, `project_wait`) alongside `start`, `stop`, `apply_recipes` +and `wait`, which keep their existing inputs and outputs. All additions; the +contract stays 3. diff --git a/docs/reference/mcp.md b/docs/reference/mcp.md new file mode 100644 index 00000000..8fbe1b0c --- /dev/null +++ b/docs/reference/mcp.md @@ -0,0 +1,150 @@ +# MCP server + +`stoat mcp` serves Stoat's Model Context Protocol server over stdio. MCP +clients launch it as a subprocess, normally with the working directory of the +project they should manage. The server uses the same Go wire types as +`--json`; see [JSON output](json.md). + +## Install a client entry + +Stoat can write the `stoat` entry for these clients: + +```sh +stoat mcp install claude-code +stoat mcp install claude-desktop +stoat mcp install cursor +stoat mcp install vscode +``` + +The command preserves other entries in the client's JSON file. Use +`--print` to print the entry without writing a file. Use `--project` to write +the current directory's project entry where the client supports it: + +```sh +stoat mcp install claude-code --project +stoat mcp install vscode --project +``` + +The installed entry includes the absolute Stoat executable, the `mcp` +argument, and the current working directory. The working directory determines +which `stoat.toml` the server reads. Claude Code and VS Code support a project +entry; Claude Desktop and Cursor use their user configuration only. + +Check the contract, transport, executable, and client entries with: + +```sh +stoat mcp doctor +``` + +`mcp doctor` marks an entry as stale when its command points to a different +Stoat executable. Re-run `mcp install` after changing the installation path. + +## Transport + +The default transport is stdio: + +```sh +stoat mcp +# Equivalent explicit form: +stoat mcp serve +``` + +Streamable HTTP is available for a client that cannot launch a subprocess: + +```sh +stoat mcp --http 127.0.0.1:7777 +``` + +The HTTP address must be loopback (`127.0.0.1`, `::1`, or `localhost`). The +server has no authentication, so it refuses addresses that bind another +interface. + +The default rate limits are 30 calls in the per-tool burst with a refill of +0.5 calls per second, and 60 calls in the shared burst with a refill of 2 +calls per second. Change them with `--tool-burst`, `--tool-rate`, `--burst`, +and `--rate` when starting the server. + +## VM access levels + +`agent_access` in a VM's `vm.toml` controls guest access. Levels include all +permissions below them: + +| Level | Guest operations | +|---|---| +| `none` | No guest operation. Host-side status, lifecycle, snapshots, logs, forwarding, recipe management and project operations remain available. | +| `observe` | `read_file`, `list_dir`, `stat`, `ps`, `svc_status`, and `tail_log`. | +| `manage` | Observe operations plus `write_file`, `copy_to`, `copy_from`, `pkg_install`, `svc`, `useradd`, and `apply_recipes`. | +| `exec` | Manage operations plus `exec`, `exec_bg`, `job_status`, `job_output`, `job_kill`, and `list_jobs`. | + +`create` defaults to `manage`. The CLI and TUI can raise or lower a VM's +level. The MCP `update` tool can lower a level but refuses to raise it. The +level is checked by the MCP server; `stoat exec` and `stoat cp` do not enforce +it when called directly by a person. + +## Tools + +The server registers these tools. Most inspection tools return data without +changing the guest. `recipe_schema` may refresh bundled recipe files, and +`search_recipes` may refresh the local index clone; treat those host-side +caches as mutable. Other tools may change VM, recipe, or guest state; tools +that run guest code are marked by their required level. + +### Host and recipe tools + +| Tool | Purpose | +|---|---| +| `list_vms`, `vm_status` | List VMs or inspect one VM, including recipe state and health. | +| `list_images` | List catalog and downloaded images. | +| `list_recipes`, `check_recipes`, `recipe_schema`, `search_recipes` | Inspect recipe availability, contract, or index entries. | +| `plan_recipes` | Show what `apply_recipes` would run or skip without running it. | +| `add_recipe`, `update_recipe`, `remove_recipe` | Add, repin, or remove remote recipes. `add_recipe` accepts curated index names only; it refuses Git URLs. `remove_recipe` has no force option. | +| `list_guests`, `guest_info` | Inspect loaded guest definitions and their package/service commands. | +| `doctor`, `logs` | Check host prerequisites or tail a VM's console/apply log. | +| `create`, `start`, `stop`, `destroy`, `update`, `clone` | Manage VM definitions and lifecycle. `destroy` deletes the VM and its disk. | +| `snapshot`, `restore`, `forward`, `wait`, `prune` | Manage disk snapshots, port forwards, state waits, and stale files. `prune` is dry-run unless `apply=true`. | +| `project_status`, `project_up`, `project_down`, `project_apply`, `project_wait` | Inspect or operate on every VM declared by the server working directory's `stoat.toml`, in declaration order. A failure stops the run and later VMs are marked skipped. | + +`wait` accepts `reachable`, `applied`, or `stopped`; `healthy=true` waits for +the applied recipes' health checks. Its `timeout_seconds` is a count of +seconds capped at 600, not a duration string. + +### Guest tools + +| Tool | Required level | Purpose | +|---|---:|---| +| `apply_recipes` | `manage` | Run the VM's configured recipe scripts, optionally with a named subset. | +| `read_file`, `list_dir`, `stat`, `ps`, `svc_status`, `tail_log` | `observe` | Read guest files, directories, process state, service state, or logs. Guest paths are absolute. | +| `write_file`, `copy_to`, `copy_from`, `pkg_install`, `svc`, `useradd` | `manage` | Modify files, copy data under the VM's shared directory, install packages, manage services, or add a user. | +| `exec`, `exec_bg`, `job_kill` | `exec` | Run or signal arbitrary guest commands. `argv` is an argument array, not a shell string. | +| `job_status`, `job_output`, `list_jobs` | `exec` | Inspect background jobs created by `exec_bg`. A guest reboot clears the job files and a job can become `unknown`. | + +Guest operations require a running VM. `copy_to` and `copy_from` restrict the +host side to the VM's configured shared directory. `pkg_install` uses the +loaded guest definition's package manager. `svc` uses its service templates. + +## Project tools and scope + +Project tools use the directory in which the server was started. They do not +accept a project path in a tool argument. Start the server from the project, +or install a project entry that records that working directory. If the server +has no `stoat.toml`, project tools fail with an instruction to run `stoat init` +or use a named VM tool. + +Project recipe state follows the same rules as the CLI: `stoat.toml` declares +recipes, `stoat.lock` pins commits, and `.stoat/recipes/` holds checkouts. A +project operation repairs a missing or stale clean checkout from the lock and +refuses a stale declaration or dirty checkout. + +## Output and limits + +MCP tool results use the DTOs described in [JSON output](json.md). Errors are +returned as tool errors with the same human-readable message as the CLI; a +caller should branch on the documented operation and level rather than parse +that message. Binary file content is base64 encoded. Read, directory, process, +log, and command output sizes are capped by the server; the tool schema states +the cap for each input. + +Recipe and guest tools read manifests and definitions on the host. They do not +run a recipe unless the caller selects `apply_recipes`, a project apply tool, +or a lifecycle operation whose documented behavior starts a VM and applies +its configured recipes. diff --git a/docs/reference/project-file.md b/docs/reference/project-file.md index fdd47858..7625ba37 100644 --- a/docs/reference/project-file.md +++ b/docs/reference/project-file.md @@ -87,10 +87,11 @@ step there. Ubuntu cloud VMs mount shares as usual. ## Secrets -Secrets live in `.stoat/secrets.toml`, mode 0600, keyed -`..`. `stoat init` adds `.stoat/` to `.gitignore` in -a git checkout. Every reader renders a secret as `` or ``, -never as its value. +Secrets declared by a project live in `.stoat/secrets.toml`, mode 0600, keyed +`..`. `stoat init` adds `.stoat/` to `.gitignore` in a git +checkout. Every reader renders a secret as `` or ``, never as its +value. A VM created outside a project keeps secrets in its own data-directory +`secrets.toml`. ## Commands @@ -101,6 +102,13 @@ never as its value. | `stoat ls --project` | filters the VM list to the current project | | `stoat up`, `down`, `apply`, `wait`, `rm` with no VM argument | act on every declared VM, in declaration order | +Project fan-out currently has a JSON limitation: `stoat up --json`, +`down --json`, and `apply --json` can emit human progress lines before the +terminal JSON result. Use a named VM command such as `stoat up dev --json` +when a clean JSON stream is required. After `down`, use +`stoat wait --until stopped` when a caller needs confirmed termination; +the stop request can return while QEMU is still exiting. + ## Errors | Condition | Message | diff --git a/docs/reference/samples/vm.toml b/docs/reference/samples/vm.toml index 688e789c..6b457c62 100644 --- a/docs/reference/samples/vm.toml +++ b/docs/reference/samples/vm.toml @@ -13,11 +13,12 @@ share = "~/src" # string path; default empty; user/TUI and st sshport = 2200 # int; default an allocated free port; stoat writes the host forward to guest sshd. recipes = ["docker"] # string[]; default []; user/TUI and stoat create/update write the selection. display = "auto" # string; default "auto"; user/TUI writes: auto, window, or vnc. -backend = "cloudinit" # string; default inferred from image; stoat writes: apkovl, cloudinit, or ssh. +backend = "apkovl" # string; default inferred from image; stoat writes: apkovl, cloudinit, or ssh. base = "" # string path; default empty; stoat writes the absolute shared base-image path. -sshuser = "stoat" # string; default guest-defined user (empty means root); stoat writes it. +sshuser = "root" # string; default guest-defined user (empty means root); stoat writes it. console_password = "" # string; default "stoat" for cloud VMs, empty otherwise; stoat writes, never ssh. -allow_exec = true # bool; default true; user/TUI and stoat create write the MCP exec/copy opt-in. +allow_exec = false # bool; derived compatibility flag; agent_access=manage does not grant guest exec. +agent_access = "manage" # string; default manage; none, observe, manage, or exec for MCP guest access. [[forwards]] # table[]; default []; user/TUI and `stoat forward` write extra forwards. hostport = 8080 # int; default none; user writes the host port. @@ -25,7 +26,6 @@ guestport = 80 # int; default none; user writes the guest por [params.docker] # table; default {}; stoat writes parameter values; do not edit by hand. user = "dev" # string; default recipe value "dev"; stoat writes the non-secret override. -channel = "stable" # string; default recipe value; stoat writes the non-secret override. # Written by stoat; do not edit. [applied.docker] diff --git a/docs/reference/tui.md b/docs/reference/tui.md index 99d8463a..ff283abc 100644 --- a/docs/reference/tui.md +++ b/docs/reference/tui.md @@ -25,7 +25,7 @@ The list shows every VM directory under the data root as one sequence: valid VMs ### Row format -A good VM's row is `name mode RAMMc cpus` followed by either a dim `-` (stopped) or `up :` (running): +A good VM's row is `name mode RAMMc cpus` followed by either a dim `-` (stopped) or `up :` (running). The elapsed value is rendered when the list redraws; opening the list does not start a separate uptime timer: ``` ❯ ● work live 4096M 4c up 2h15m :2222 @@ -48,6 +48,7 @@ A good VM's row is `name mode RAMMc cpus` followed by either a dim `-` (stopp | `p` | Provision | Runs the same guard chain as the detail screen's `p`, see [Provisioning](#provisioning-progress). | | `/` | Search | Opens the filter input; typing filters the list by VM name. | | `n` | New VM | Opens the new-VM form. If a download is still in flight from a previous visit to the form, that same form (and its download) is reused instead of being reset. | +| `r` | Edit recipes | Opens the recipes directory in `$EDITOR` (or `vi` when `$EDITOR` is unset). A recipe added there appears the next time the new-VM form opens. | | `d` | Delete | On a stopped VM or a broken entry, arms a `delete ? y/N` prompt. On a running VM, refuses with "stop `` first". | | `j`/`↓`, `k`/`↑`, `pgup`, `pgdown`, `home`, `end`, `g`, `G` | Move / page | Forwarded to the underlying list component. | | `esc` | Clear search, or cancel a pending prompt | See ordering note below. | @@ -112,7 +113,9 @@ The image picker offers every catalog entry (Alpine, etc.) plus any file already **Validation on `enter`:** name is required, must contain no spaces or slashes, and must not already exist; an image must be selected and already downloaded; RAM must be a number ≥ 256 (MB); CPUs must be ≥ 1. The ssh port is picked automatically from the first free port. -Recipes offered are filtered to those matching the selected image's OS/backend, and the selection resets whenever the image changes. +Recipes offered are filtered to those matching the selected image's guest OS +and manifest requirements. The compatibility `--backend` filter is not used +for manifest applicability. The selection resets whenever the image changes. While a download is running, a progress block appears under the fields, see [Download progress](#download-progress). @@ -151,7 +154,7 @@ The in-TUI editor for a VM that already exists: everything the raw `$EDITOR` rou |---|---|---| | `tab` / `↓` | Next field | | | `shift+tab` / `↑` | Previous field | | -| `←` / `→` | Change the focused field | Mode row: cycle `live` / `disk` / `cloud` (also resyncs the recipe list, since recipes are per-backend). Recipes row: move the sub-cursor. | +| `←` / `→` | Change the focused field | Mode row: cycle `live` / `disk` / `cloud` (also resyncs the recipe list for the selected guest). Recipes row: move the sub-cursor. | | `space` | Toggle a recipe | Recipes row only. | | `enter` | Save | Runs the guard chain below; on success writes `vm.toml` and returns to the detail screen. | | `esc` | Cancel | Back to the detail screen without saving. | diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index fa2006ce..25edfe50 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -134,8 +134,9 @@ over on its own. ## A binary on `/mnt/host` won't run: confusing "not found" You built a binary on the host, it shows up fine on the shared -`/mnt/host` (backed by `-virtfs local,...,security_model=none`, -`internal/qemu/args.go`), but running it inside the guest fails with something +`/mnt/host` (backed by QEMU's read-only `-virtfs` export with +`security_model=mapped-xattr`, `internal/qemu/args.go`), but running it inside +the guest fails with something like: ``` @@ -194,6 +195,13 @@ is gone the moment the VM restarts: only the apkovl itself (rebuilt fresh on every `Start`, per `internal/qemu/run.go`) survives, and it can't carry installed packages. +The host still keeps the VM's applied recipe records. A matching recipe with +`run = "once"` can therefore be skipped after a live restart even though its +package or files disappeared with the guest. For work that must run on every +boot, copy the recipe into a project or local scope, set `run = "always"` in +its manifest, and select that copy for the VM. Existing bundled once recipes +may report a successful no-op after a restart. + **If you want state to survive a reboot, live mode is the wrong mode.** Create a **disk** VM instead (install the OS once, then it persists like a normal machine) or a **cloud** VM (a prebuilt image where cloud-init's `packages:`/ @@ -229,3 +237,106 @@ delete a VM out from under a live QEMU process. **Fix:** stop the VM first (`stoat down `, or `d` in the TUI once it's stopped), then `rm` it. + +## `stoat up` says the project declaration is immutable + +Project scope reconciles an existing VM with `stoat.toml`. CPU, memory, +recipes, parameters, shares, and `agent_access` are mutable. The image and +disk size identify the VM's backing storage and cannot be changed in place. + +**Fix:** if the image or disk declaration changed intentionally, stop and +remove that VM, then run `stoat up ` to create it again. Removing a VM +deletes its directory and persistent disk. A project command without a VM +name uses declaration order; use the key or global name when repairing one +entry. + +## `stoat up` says `no VM ... in stoat.toml or ~/.stoat/vms` + +Project scope is active only when `stoat.toml` exists in the current +directory. Stoat does not walk up to a parent directory. A bare VM argument +must be a declaration key, a declared `name`, or a global VM that exists in +the data root. + +**Fix:** change to the directory containing the intended `stoat.toml`, use the +declaration key, or create a global VM with `stoat create --global ...`. + +## `stoat recipe lock` says the lock is out of date + +`stoat.toml` declares project recipe refs. `stoat.lock` records the commit to +use, and `.stoat/recipes/` is the local checkout. Editing `[recipes]` without +locking leaves the declaration unpinned. + +**Fix:** run the following from the project directory: + +```sh +stoat recipe lock +stoat recipe sync +``` + +Commit `stoat.toml` and `stoat.lock`. Keep `.stoat/` ignored. `recipe lock` +resolves refs but does not populate the cache; `recipe sync` makes the cache +match the lock. If the configured index has no matching name, pass a Git URL +for a repository containing `recipe.toml` or configure an index that publishes +the recipe. + +## `stoat apply` says `not running` or `recipe not applicable` + +`apply` sends the VM's selected recipe scripts over SSH. The VM must be +running, and the recipe must be in that VM's own recipe list and applicable to +its guest OS and backend. `apply --dry-run` computes the plan without SSH and +does not require a running VM. + +**Fix:** start the VM, then inspect the plan: + +```sh +stoat up --no-apply +stoat apply --dry-run +stoat apply +``` + +Use `--only ` only for a recipe already listed on the VM. A recipe can +run again when its script changed or its manifest declares `run = "always"`; +`run = "once"` skips a matching applied version, and `run = "manual"` needs +an explicit `--only` selection. + +## `stoat wait --healthy` times out + +`wait --healthy` first waits for SSH, then checks every applied recipe that +declares a health command. A recipe without a health command contributes no +verdict. A failed check includes the recipe name and its last diagnostic line +when available. + +**Fix:** inspect the apply log and VM state: + +```sh +stoat logs --which apply -n 200 +stoat get +stoat wait --until reachable +``` + +Run the failing health command yourself with `stoat exec` only when the VM's +agent access policy and your workflow permit it. The CLI timeout is a Go +duration and defaults to two minutes; the MCP `wait` tool uses +`timeout_seconds` and caps it at 600 seconds. + +## An MCP client cannot find `stoat` or project VMs + +MCP clients launch the binary recorded by their configuration entry. The +entry also records its working directory, which determines project scope. + +**Fix:** reinstall the entry from the intended directory and inspect it: + +```sh +stoat mcp install claude-code +stoat mcp doctor +``` + +Use `stoat mcp install claude-code --project` when Claude Code should read a +project-local `.mcp.json`. The server uses stdio by default. For HTTP, use +`stoat mcp serve --http 127.0.0.1:7777`; non-loopback addresses are refused +because the server has no authentication. + +If a tool reports an access refusal, raise `agent_access` with the CLI or TUI. +The MCP `update` tool can lower a VM's level but cannot raise it. `observe` is +needed for guest reads, `manage` for writes and recipe application, and `exec` +for command and job tools. diff --git a/docs/writing-recipes.md b/docs/writing-recipes.md index 4c80182e..a603e3f5 100644 --- a/docs/writing-recipes.md +++ b/docs/writing-recipes.md @@ -1,9 +1,9 @@ # Writing recipes A recipe is a directory with a `recipe.toml` manifest and one or more shell -scripts. Stoat runs the right script over ssh (or bakes it into cloud-init) -depending on the guest and backend. This is the v2 format; see -`docs/recipe-spec-v2.md` for the full spec this guide summarizes. +scripts. Stoat runs the selected script over SSH or places it in a cloud-init +seed, depending on the VM backend. This is the shipped directory-manifest +format; the [v2 spec](recipe-spec-v2.md) is historical design context. ## Directory structure @@ -32,7 +32,6 @@ everything else is shell scripts referenced from it. name = "docker" description = "Docker engine and the compose plugin" os = ["alpine"] -requires = ["apk", "openrc"] stage = "provision" script = "install.sh" ``` @@ -52,7 +51,7 @@ script = "install.sh" | Field | Type | Required | Default | Description | |-------|------|----------|---------|-------------| -| `name` | string | yes | | Identifier, matches the directory name | +| `name` | string | yes | | Recipe identifier; keep it a safe path component | | `description` | string | no | | One-line human description | | `version` | string | no | | Semver; tracked per-VM alongside applied state | | `os` | string[] | no | all OSes | Guest OSes this recipe applies to | @@ -61,7 +60,7 @@ script = "install.sh" | `script` | string | yes | | Default script, path relative to the recipe dir | | `scripts` | table | no | | Per-OS script overrides; unlisted OSes fall back to `script` | | `run` | string | no | `"once"` | `"once"`, `"always"`, or `"manual"` | -| `auto` | bool | no | `false` | Run automatically the first time the VM becomes reachable | +| `auto` | bool | no | `false` | Stored manifest metadata; current TUI auto-provision does not read it | | `runtime` | string | no | `"sh"` | Interpreter the script runs under: `"sh"` or `"python3"` | `name` and `script` are the only required fields; a manifest missing either @@ -82,11 +81,9 @@ guests that can't satisfy it: | `dnf` | dnf package manager | fedora | | `pacman` | pacman package manager | arch | -`docker/recipe.toml` uses `requires = ["apk", "openrc"]` instead of -`os = ["alpine"]` alone. Both narrow the recipe to Alpine today, but -`requires` also documents *why*: if another OpenRC/apk distro shows up later, -the recipe already applies to it with no manifest change. A recipe with both -`os` and `requires` must satisfy both. +The bundled recipes use `os` to name their supported guests. A recipe may also +use `requires` to require every named guest capability and its init value. A +recipe with both `os` and `requires` must satisfy both filters. A recipe requiring `systemd` is never offered to an Alpine VM; stoat resolves this against `guest.OS` at list/check time, before ssh is even involved. @@ -103,6 +100,10 @@ case: install packages, enable services, write config. - **cloudinit backend**: the script is wrapped into the cloud-config `runcmd` block at VM creation and runs at first boot. +Cloud-init currently executes recipe bodies through a shell command and does +not perform the SSH runtime bootstrap. Use `runtime = "sh"` for a cloud +recipe; `runtime = "python3"` is not installed or invoked by cloud-init. + ## Runtime `runtime` picks the interpreter a `provision`-stage script runs under. It @@ -150,9 +151,9 @@ table in `vm.toml`: - `manual`: never runs automatically; only via an explicit `stoat apply --only `. -`auto = true` is a separate toggle: it makes the recipe run automatically the -first time the VM becomes reachable, without waiting for an explicit -`stoat apply`. Default is `false`. +`auto` is accepted and stored as manifest metadata. The current TUI decides +whether to offer provisioning from VM mode and applied state; it does not read +`auto`. ## Verbs @@ -236,8 +237,9 @@ Two conventions worth copying from the bundled scripts: ## Testing a recipe manually -`stoat recipes` lists what's applicable to a given OS/backend, filtered by -`os`/`requires`: +`stoat recipes` lists what's applicable to a given guest OS, filtered by the +manifest's `os` and `requires` fields. The accepted `--backend` flag is a +compatibility argument and does not add another manifest filter: ``` stoat recipes --os alpine @@ -281,6 +283,6 @@ hand-edited recipe is left alone. Drop a new directory with its own `recipe.toml` anywhere under `~/.stoat/recipes/` and it's picked up the same way, no code changes or registration needed. -Old-format flat files (`..sh`, `.cloud.yaml`) still work -during the v1-to-v2 deprecation period, but stoat logs a warning whenever one -is offered, suggesting migration to a `recipe.toml` directory. +Old-format flat files (`..sh`, `.cloud.yaml`) are not a +supported authoring format. Convert them to a directory with `recipe.toml`, +or start with `stoat recipe new `. diff --git a/internal/recipes/samples/recipe.toml b/internal/recipes/samples/recipe.toml index 578613a0..ba56c524 100644 --- a/internal/recipes/samples/recipe.toml +++ b/internal/recipes/samples/recipe.toml @@ -9,7 +9,7 @@ os = ["alpine"] # string[]; default []; author writes. Empty me requires = ["apk"] # string[]; default []; author writes. Capabilities from the guest file. stage = "provision" # string; default "provision"; author writes: "install" or "provision". script = "install.sh" # string; required; author writes. Relative to this directory. -auto = false # bool; default false; author writes. Offered pre-checked in the TUI picker. +auto = false # bool; default false; accepted metadata. The TUI does not use it to select recipes. run = "once" # string; default "once"; author writes: "once", "always", or "manual". reboot = false # bool; default false; author writes. Reboot a disk VM after this recipe. runtime = "sh" # string; default "sh"; author writes: "sh" or "python3".