diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md index 2810e4d1..873d66da 100644 --- a/docs/SUMMARY.md +++ b/docs/SUMMARY.md @@ -19,6 +19,7 @@ * [TUI](reference/tui.md) * [CLI](reference/cli.md) * [JSON output](reference/json.md) +* [Guest definitions](reference/guest.md) ## Recipes diff --git a/docs/design/core-api.md b/docs/design/core-api.md index 34d98da7..d18bfb85 100644 --- a/docs/design/core-api.md +++ b/docs/design/core-api.md @@ -238,6 +238,12 @@ ErrBroken ErrDiskShrink Each carries the specific subject (which field, which recipe, which dependency) rather than only a message. +The CLI reports these two ways, and the choice is not a style question. `a.fail` takes an +error `core` returned and maps its sentinel to the JSON error code. `a.failMsg` takes a +sentinel the CLI picked itself, for a condition `core` never saw: a bad flag combination, an +unknown subcommand. A CLI-detected condition sent through `a.fail` would carry whatever code +the wrapped error happened to hold. + --- ## 10. Security model diff --git a/docs/design/guest-subsystem.md b/docs/design/guest-subsystem.md index 3cd1b1ff..38b54930 100644 --- a/docs/design/guest-subsystem.md +++ b/docs/design/guest-subsystem.md @@ -1,5 +1,10 @@ # Guest Subsystem: Design +**Update, 2026-09-05:** the guest registry this document designs is now data, +not Go: `internal/guest/bundled/*.toml` plus `~/.stoat/guests/*.toml` merged +over it, loaded through `internal/tomlx`. See `docs/reference/guest.md` for +the file format. + **Status:** accepted design, not yet a plan. Written 2026-08-02. The operation surface built on this layer is specified in [`core-api.md`](core-api.md). **Why this exists:** guest-OS knowledge is scattered across 25 sites as ad-hoc string comparisons, provisioning has no contract at all, and the logic that creates a VM lives inside a Bubbletea form. Adding Alpine cloud support missed three of those sites and the feature silently did not work. This document defines the subsystem that makes that class of failure structural rather than a matter of remembering, and the API layer that lets something other than a keyboard drive stoat. diff --git a/docs/recipe-spec-v2.md b/docs/recipe-spec-v2.md index 47594da0..05ae3484 100644 --- a/docs/recipe-spec-v2.md +++ b/docs/recipe-spec-v2.md @@ -63,16 +63,8 @@ fedora = "install-fedora.sh" ### Capabilities -Capabilities are facts about a guest that a recipe can require: - -| Capability | Meaning | OSes | -|------------|---------|------| -| `systemd` | Uses systemd init | ubuntu, debian, arch, fedora | -| `openrc` | Uses OpenRC init | alpine | -| `apt` | Has apt package manager | ubuntu, debian | -| `apk` | Has apk package manager | alpine | -| `dnf` | Has dnf package manager | fedora | -| `pacman` | Has pacman | arch | +Capabilities are facts about a guest that a recipe can require: every +`capabilities` entry and `init` of a loaded guest; see `stoat guest show`. A recipe declaring `requires = ["systemd"]` is not offered to Alpine. Stoat resolves these against `guest.OS` at list/check time. diff --git a/docs/reference/cli.md b/docs/reference/cli.md index b7218267..5281d7d5 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -38,11 +38,12 @@ usage: stoat [flags] | [`snapshot`](#stoat-snapshot-name-tag) | List, save, restore or delete a snapshot | 0, 1, 2 | | [`prune`](#stoat-prune) | Report, or with `--apply` remove, stale files | 0, 1 | | [`apply`](#stoat-apply-name) | Run the VM's recipes, streaming output | 0, 1 | -| [`provision`](#stoat-provision-name) | Run recipes, streaming output to stdout | 0, 1 | | [`recipes`](#stoat-recipes) | List recipes, optionally only applicable ones | 0, 1 | | [`check-recipes`](#stoat-check-recipes-names---osos) | Report why a recipe would not apply | 0, 1, 2 | | [`recipe list`](#stoat-recipe-list) | List installed recipes and where they live | 0, 1 | | [`recipe new`](#stoat-recipe-new-name) | Scaffold a recipe in the recipes directory | 0, 1 | +| [`guest ls`](#stoat-guest-ls) | List loaded guest OS definitions | 0 | +| [`guest show`](#stoat-guest-show-name) | Print one guest's merged definition | 0, 1 | | [`logs`](#stoat-logs-name--n-n) | Tail a VM's log, or stoat's own | 0, 1 | | [`doctor`](#stoat-doctor) | Check host prerequisites | 0, 1 | | [`version`](#stoat-version) | Print the stoat version | 0 | @@ -420,29 +421,7 @@ work: recipes applied **Exit codes:** 0 on success; 1 if the VM can't be loaded, the run fails, or (for a cloud-mode VM) recipes were already applied at boot rather than by this command. -## `stoat provision ` - -Runs the VM's recipes over ssh, streaming `last-provision.log` to stdout as it's written (polled every 150ms), the same log the TUI's detail screen tails, so there is no separate provisioning path to keep in sync. - -``` -$ stoat provision work -provisioning work... -=== recipe xfce === -Unpacking libx11-data... -... -work provisioned -``` - -A **cloud-mode VM short-circuits**: cloud-init applies its recipes once, automatically, at first boot, baked into the seed when the VM's overlay was created, there is nothing left for ssh-based provisioning to do, and piping a cloud recipe (`#cloud-config` YAML, not a shell script) into `sh -s` would just fail. Instead it prints an explanatory line and exits 0 without touching ssh: - -``` -$ stoat provision cloudvm -cloudvm is a cloud VM: recipes are applied automatically via cloud-init at first boot; recreate the VM to change them. -``` - -`-q` suppresses the `provisioning ...` line only; the streamed log and the final line still print. - -**Exit codes:** 0 on success (including the cloud short-circuit); 1 if the VM can't be loaded or the provision run itself fails. +`provision` is a hidden alias of `apply`: `stoat provision work` behaves exactly like `stoat apply work`, and reports `"cmd":"apply"` under `--json`. ## `stoat recipes` @@ -505,6 +484,47 @@ edit it, then pick it in the new-vm form for a matching vm **Exit codes:** 0 on success; 1 if the recipe can't be created (e.g. the name is already taken). +## `stoat guest ls` + +Lists every loaded guest OS: bundled definitions from `internal/guest/bundled/*.toml`, plus any `~/.stoat/guests/*.toml` merged over them. + +``` +$ stoat guest ls +NAME INIT PKG BACKEND SOURCE +alpine openrc apk apkovl bundled +arch systemd pacman cloudinit bundled +debian systemd apt-get cloudinit bundled +fedora systemd dnf cloudinit bundled +ubuntu systemd apt-get cloudinit bundled +``` + +**Exit codes:** always 0. + +## `stoat guest show ` + +Prints one guest's merged definition: init system, shell, default backend and ssh user, escalate argv, capabilities, aliases, and the `pkg`/`svc` tables. See `docs/reference/guest.md` for what each field means. + +``` +$ stoat guest show alpine +name: alpine (bundled) +init: openrc +shell: /bin/ash +default backend: apkovl +default ssh user: root +escalate: [sudo -n] +capabilities: [apk openrc] +aliases: [] +pkg setup: apk update +pkg install: [apk --wait 60 add] +svc enable: rc-update add {name} default +svc start: rc-service {name} start +svc stop: rc-service {name} stop +svc restart: rc-service {name} restart +svc status: rc-service {name} status +``` + +**Exit codes:** 0 on success; 1 if no guest by that name is loaded. + ## `stoat logs [name] [-n N]` With a VM name, tails that VM's own log (`--which console` for the qemu console, the default, or `--which apply` for its apply log). With no name, tails stoat's own log file. diff --git a/docs/reference/guest.md b/docs/reference/guest.md new file mode 100644 index 00000000..111d7e97 --- /dev/null +++ b/docs/reference/guest.md @@ -0,0 +1,81 @@ +# Guest definitions + +A guest OS's facts (its init system, package manager, service commands...) +live in one `guest.toml` file, not in Go. Five come bundled +(`internal/guest/bundled/*.toml`: alpine, ubuntu, debian, fedora, arch). +Add or override one by dropping a file in `~/.stoat/guests/.toml`; see +`stoat guest ls` and `stoat guest show `. + +```toml +schema = 1 +name = "freebsd" +init = "rc" # systemd | openrc | rc +shell = "/bin/sh" +installer = "" # the interactive install command; "" means "the installer" +default_backend = "cloudinit" # seeds the VM field at create time; the VM field always wins after +default_ssh_user = "freebsd" # same +escalate = ["sudo", "-n"] # argv; applied only when the VM's ssh user is not root +capabilities = ["pkg"] # feeds recipe.toml's `requires`; the loader appends init +aliases = ["bsd"] # extra keys a recipe's [scripts] map may use for this OS +filename_hints = ["FreeBSD-"] # recognise this OS in a BYO image filename +seed_packages = ["sudo"] # packages the cloud-init seed assumes but the image may lack + +[pkg] +setup = "pkg update" # prelude's stoat_pkg_setup; empty means no refresh needed +install = ["pkg", "install", "-y"] # argv; prelude's stoat_pkg_install appends packages +scaffold_setup = "" # comment text `recipe new` writes into a scaffolded script +scaffold_install = "pkg install -y " # same, for the install line +runtime_packages = { python3 = "python3" } # maps a recipe runtime to the package that provides it + +[svc] +enable = "sysrc {name}_enable=YES" +start = "service {name} start" +stop = "service {name} stop" +restart = "service {name} restart" +status = "service {name} status" + +[backend.cloudinit] # opaque to the loader; the cloudinit package decodes it +skip_9p = true +``` + +## Field rules + +- Required: every top-level scalar and list except `installer`, `aliases`, + `filename_hints`; every `[pkg]` and `[svc]` key except `scaffold_setup`. A + missing one is `guest.toml: : missing `. +- Unknown keys are an error: `: unknown key ""`. +- `schema` must be present and equal to 1. +- The loader appends `init` to `capabilities`. A file whose `capabilities` + names a different init system (`systemd`, `openrc`, `rc`) is rejected. +- `[backend.*]` loads as an opaque table per backend name. The backend + package that owns a name decodes its own keys; `internal/guest` never + reads inside. +- `default_backend` and `default_ssh_user` only seed a new VM's fields at + create time. Every code path reads the VM's own field afterward, never + these, so a catalog entry or a user's explicit choice always wins. +- `escalate` is an OS fact, not a VM fact: Alpine and OpenBSD ship `doas`, + not `sudo`. `sshx` applies it only when the VM's ssh user is not root. +- Every `[svc]` value is a template: `{name}` renders to `"$1"`; a template + without `{name}` gets `"$@"` appended instead. A template may not contain + a single quote: the python prelude wraps each one in a single-quoted + literal. +- `STOAT_PKGMGR` (available in the recipe prelude) is the basename of + `pkg.install[0]`. + +## Merging a user file over a bundled one + +A user file whose `name` matches a bundled guest merges over it: scalars and +lists in the file replace the bundled value, an absent field keeps the +bundled one, and each `[backend.x]` table replaces whole (not merged key by +key). A user file with a new `name` needs every required field; nothing is +inherited. + +`stoat guest show ` reports `source`: `bundled`, `user`, or +`bundled+user` for a merged file. + +## Unknown guest + +A `vm.toml` whose `os` names no loaded guest makes that VM broken (`stoat ls` +shows `unknown guest ""; run stoat guest ls`), the same as an +unparseable `vm.toml`. An empty `os` is not checked: it keeps the existing +fallbacks for a VM saved before the OS field existed. diff --git a/docs/reference/json.md b/docs/reference/json.md index bfbe5f55..adc83808 100644 --- a/docs/reference/json.md +++ b/docs/reference/json.md @@ -195,6 +195,17 @@ RecipeIssue {"name":"docker","reason":"docker is not offered to debian/cloudinit ApplyPlan {"name":"xfce","action":"run","reason":"never applied", "version":"1.2"} + +Guest {"name":"fedora","init":"systemd","shell":"/bin/bash", + "installer":"","default_backend":"cloudinit", + "default_ssh_user":"stoat","escalate":["sudo","-n"], + "capabilities":["dnf","systemd"],"aliases":["rpm-family"], + "filename_hints":["fedora"],"seed_packages":[], + "pkg":{"setup":"","install":["dnf","install","-y"], + "env":{},"runtime_packages":{"python3":"python3"}}, + "svc":{"enable":"systemctl enable {name}", ...}, + "cmd":{},"backend":{"cloudinit":{"skip_9p":false}}, + "source":"bundled"} ``` `state` is one of `stopped`, `running`, `broken`. `error` appears only on a @@ -261,6 +272,13 @@ agent cannot run a GUI viewer anyway. A consumer that needs to tell a human where to look runs `stoat get ` without `--json`, which prints the socket and an attach command for a viewer installed on that machine. +`Guest.source` is `"bundled"`, `"user"`, or `"bundled+user"` for a user file +merged over a bundled one. `Guest.svc` and `Guest.cmd` are template strings, +not commands to run directly: `{name}` renders to the service/argument, see +`docs/reference/guest.md`. `Guest.backend` passes each `[backend.]` +table through opaque; only the backend package that owns `` defines its +keys. + ### What is deliberately absent - **Host paths.** `core.VM` carries six absolute host paths (its disk, console @@ -302,11 +320,14 @@ so a leak fails the build rather than shipping. | `apply --dry-run` | `{"vm":"work","dry_run":true,"plan":[ApplyPlan,...]}` | | `recipes` | `{"recipes":[Recipe,...]}` | | `check-recipes` | `{"applicable":false,"issues":[RecipeIssue,...]}` | +| `guest ls` | `{"guests":[Guest,...]}` | +| `guest show` | `{"guest":Guest}` | | `recipe list` | `{"dir":"...","recipes":["xfce"]}`, see note below | | `recipe new` | `{"path":"/home/u/.stoat/recipes/foo.alpine.sh"}` | -Both `recipe` subcommands report `"cmd":"recipe"`, not `"cmd":"recipe list"`. -Distinguish them by which fields `data` carries. +Both `recipe` subcommands report `"cmd":"recipe"`, not `"cmd":"recipe list"`, +and both `guest` subcommands report `"cmd":"guest"`. Distinguish them by which +fields `data` carries. `recipe list` is "every file in the recipes directory", which is not the same as "every recipe you can use": it currently includes the `.bak` files the diff --git a/docs/writing-recipes.md b/docs/writing-recipes.md index 77f350d2..4c80182e 100644 --- a/docs/writing-recipes.md +++ b/docs/writing-recipes.md @@ -12,12 +12,13 @@ depending on the guest and backend. This is the v2 format; see xfce/ recipe.toml install.sh - install-alpine.sh - install-debian.sh - install-arch.sh docker/ recipe.toml install.sh + install-alpine.sh + install-debian.sh + install-fedora.sh + install-arch.sh ``` One directory per recipe, named after the recipe. `recipe.toml` is required; @@ -36,7 +37,8 @@ stage = "provision" script = "install.sh" ``` -`internal/recipes/bundled/xfce/recipe.toml`, with per-OS overrides: +`internal/recipes/bundled/xfce/recipe.toml`, one script covering every OS by +branching on the guest verbs (see Verbs, below): ```toml name = "xfce" @@ -44,11 +46,6 @@ description = "XFCE desktop with autologin startx on tty1" os = ["alpine", "ubuntu", "debian", "arch"] stage = "provision" script = "install.sh" - -[scripts] -alpine = "install-alpine.sh" -debian = "install-debian.sh" -arch = "install-arch.sh" ``` ### Fields @@ -157,29 +154,63 @@ table in `vm.toml`: first time the VM becomes reachable, without waiting for an explicit `stoat apply`. Default is `false`. +## Verbs + +A recipe script does not need to know each guest's package manager or init +system by name: the prelude `stoat` renders in front of every script body +defines a small set of shell functions and variables over the guest's +`guest.toml` facts (`docs/reference/guest.md`), the same functions for every +OS. + +| Name | What it does | +|------|--------------| +| `stoat_pkg_setup` | Refreshes the package index (a no-op where none is needed, e.g. dnf) | +| `stoat_pkg_install ` | Installs packages with the guest's package manager | +| `stoat_svc_enable ` | Enables a service to start at boot | +| `stoat_svc_start`/`stop`/`restart`/`status` `` | Controls a running service | +| `STOAT_OS` | The guest name, e.g. `alpine` | +| `STOAT_INIT` | `systemd`, `openrc`, or `rc` | +| `STOAT_PKGMGR` | The package manager binary, e.g. `apk`, `apt-get`, `dnf`, `pacman` | + +`internal/recipes/bundled/xfce/install.sh` is one script for four OSes, +branching on `STOAT_PKGMGR` for package names and `STOAT_INIT` for the +autologin mechanism (systemd's `getty@.service` drop-in vs. Alpine's busybox +`/etc/inittab`): + +```sh +stoat_pkg_setup + +case "$STOAT_PKGMGR" in +apk) stoat_pkg_install xfce4 xfce4-terminal dbus-x11 ;; +apt-get) stoat_pkg_install xfce4 xfce4-terminal dbus-x11 xinit xserver-xorg ;; +pacman) stoat_pkg_install xfce4 xfce4-terminal dbus xorg-xinit ;; +esac + +stoat_svc_enable dbus +``` + +`[scripts]` per-OS overrides (below) still exist for a recipe whose install +differs by more than package names and a service call, `docker`'s GPG key +setup and repo URL being a different string per distro family. + ## OS-specific script overrides -`script` is the fallback every OS not listed in `[scripts]` uses. `xfce` -needs different logic per init system and package manager, so it overrides -three of its four supported OSes and leaves `ubuntu` on the default -`install.sh`: +`script` is the fallback every OS not listed in `[scripts]` uses. `docker` +needs a different script per distro family (repo URL, GPG key path), so it +overrides every OS it supports: ```toml -os = ["alpine", "ubuntu", "debian", "arch"] +os = ["alpine", "ubuntu", "debian", "fedora", "arch"] script = "install.sh" [scripts] alpine = "install-alpine.sh" +ubuntu = "install-debian.sh" debian = "install-debian.sh" +fedora = "install-fedora.sh" arch = "install-arch.sh" ``` -Compare `install.sh` (ubuntu/debian, `apt-get` + systemd `getty@.service` -override) against `install-alpine.sh` (`apk` + busybox `/etc/inittab` -autologin, no systemd unit exists to override). The packages and init -mechanism differ enough per OS that a single script branching internally -would be harder to read than one script per OS family. - Two conventions worth copying from the bundled scripts: - **`set -e` at the top**, so the script stops on the first real failure diff --git a/internal/backend/backend.go b/internal/backend/backend.go index ffbb67f0..914c85ef 100644 --- a/internal/backend/backend.go +++ b/internal/backend/backend.go @@ -1,4 +1,4 @@ -// Package backend turns guest.OS.Backend from a string into an interface. +// Package backend turns guest.OS.DefaultBackend from a string into an interface. // It is the only package allowed to assume "this is Alpine" or "this is // cloud-init". internal/qemu no longer needs that assumption. // @@ -15,7 +15,7 @@ import ( "github.com/novusedge/stoat/internal/guest" ) -// Backend is the behaviour a guest.OS.Backend name stands for: how to build +// Backend is the behaviour a guest.OS.DefaultBackend name stands for: how to build // the pre-boot artifact this OS needs, and what QEMU arguments that artifact // requires. Only Prepare and Args are implemented here. Ready (today: // sshx.Wait plus cloud-init polling) and Provision (today: @@ -64,7 +64,7 @@ func For(v *config.VM) Backend { name := v.Backend if name == "" { if o, ok := guest.Lookup(v.OS); ok { - name = o.Backend + name = o.DefaultBackend } } switch name { diff --git a/internal/backend/cloudinit.go b/internal/backend/cloudinit.go index cb5b18d8..40b792a7 100644 --- a/internal/backend/cloudinit.go +++ b/internal/backend/cloudinit.go @@ -9,6 +9,7 @@ import ( "github.com/novusedge/stoat/internal/cloudinit" "github.com/novusedge/stoat/internal/config" + "github.com/novusedge/stoat/internal/guest" "github.com/novusedge/stoat/internal/keys" "github.com/novusedge/stoat/internal/recipes" ) @@ -75,8 +76,16 @@ func (cloudinitBackend) Prepare(v *config.VM) error { // WrapScripts renders every recipe into one write_files+runcmd fragment // cloud-init runs in order at first boot. An empty selection wraps to "", // which Seed must not carry as a document; pass no bodies instead. + // + // prelude is sh only: the cloud-init path has no runtime bootstrap step + // (that is sshx.Provision's job), so a non-sh recipe here runs with a + // bare body, same as before this feature. + var prelude string + if o, ok := guest.Lookup(v.OS); ok { + prelude = guest.Prelude(o, "sh") + } var bodies []string - if frag := cloudinit.WrapScripts(scripts); frag != "" { + if frag := cloudinit.WrapScripts(scripts, prelude); frag != "" { bodies = []string{frag} } if _, err := cloudinit.Seed(v, pub, bodies); err != nil { diff --git a/internal/cli/cli.go b/internal/cli/cli.go index cf9f342c..05674abc 100644 --- a/internal/cli/cli.go +++ b/internal/cli/cli.go @@ -18,6 +18,7 @@ import ( "bytes" "fmt" "io" + "path/filepath" "strings" "time" @@ -26,6 +27,7 @@ import ( "github.com/novusedge/stoat/internal/cli/wire" "github.com/novusedge/stoat/internal/config" "github.com/novusedge/stoat/internal/core" + "github.com/novusedge/stoat/internal/guest" "github.com/novusedge/stoat/internal/keys" "github.com/novusedge/stoat/internal/logx" "github.com/novusedge/stoat/internal/recipes" @@ -358,6 +360,15 @@ func Main(args []string, version string, stdin io.Reader, stdout, stderr io.Writ defer logx.Close() logx.L().Debug("cli", "cmd", a.Cmd, "vm", a.VM) + if err := guest.Load(filepath.Join(config.Root(), "guests")); err != nil { + if a.JSON { + _ = wire.NewEmitter(stdout).ResultErr(a.Cmd, wire.UsageError(err.Error())) + return ExitUsage + } + fmt.Fprintln(stderr, "stoat:", err) + return ExitUsage + } + switch a.Cmd { case "ls": return runLS(a, stdout, stderr) @@ -385,12 +396,12 @@ func Main(args []string, version string, stdin io.Reader, stdout, stderr io.Writ return runDown(a, stdout, stderr) case "ssh": return runSSH(a, stdout, stderr) - case "provision": - return runApply(a, stdout, stderr) case "rm": return runRM(a, stdin, stdout, stderr) case "recipe": return runRecipe(a, stdout, stderr) + case "guest": + return runGuest(a, stdout, stderr) case "logs": return runLogs(a, stdout, stderr) case "get": diff --git a/internal/cli/cli_test.go b/internal/cli/cli_test.go index 3ea46624..448026fe 100644 --- a/internal/cli/cli_test.go +++ b/internal/cli/cli_test.go @@ -82,9 +82,10 @@ func TestParse(t *testing.T) { {"ssh", []string{"ssh", "alpine"}, &Args{Cmd: "ssh", VM: "alpine"}, false}, {"ssh missing name", []string{"ssh"}, nil, true}, - {"provision", []string{"provision", "alpine"}, &Args{Cmd: "provision", VM: "alpine"}, false}, + // provision is a kong alias of apply, so it parses to Cmd "apply". + {"provision", []string{"provision", "alpine"}, &Args{Cmd: "apply", VM: "alpine"}, false}, {"provision missing name", []string{"provision"}, nil, true}, - {"provision quiet alias", []string{"provision", "--no-interactive", "alpine"}, &Args{Cmd: "provision", VM: "alpine", Quiet: true}, false}, + {"provision quiet alias", []string{"provision", "--no-interactive", "alpine"}, &Args{Cmd: "apply", VM: "alpine", Quiet: true}, false}, {"rm", []string{"rm", "alpine"}, &Args{Cmd: "rm", VM: "alpine"}, false}, {"rm -y", []string{"rm", "-y", "alpine"}, &Args{Cmd: "rm", VM: "alpine", Yes: true}, false}, diff --git a/internal/cli/grammar.go b/internal/cli/grammar.go index 31d758c7..8e550384 100644 --- a/internal/cli/grammar.go +++ b/internal/cli/grammar.go @@ -51,11 +51,11 @@ type grammar struct { Snapshot snapshotCmd `cmd:"" help:"list, save, restore or delete a snapshot"` Prune pruneCmd `cmd:"" help:"report, or with --apply remove, stale files"` - Apply applyCmd `cmd:"" help:"run the VM's recipes, streaming output"` - Provision applyCmd `cmd:"" hidden:"" name:"provision" help:"alias of apply"` + Apply applyCmd `cmd:"" aliases:"provision" help:"run the VM's recipes, streaming output"` Recipes recipesCmd `cmd:"" help:"list recipes, optionally only applicable ones"` CheckRecipes checkRecipesCmd `cmd:"" help:"report why a recipe would not apply"` Recipe recipeCmd `cmd:"" help:"author recipes"` + Guest recipeGuestCmd `cmd:"" help:"list or show guest OS definitions"` Logs logsCmd `cmd:"" help:"tail a VM's log, or stoat's own"` Doctor doctorCmd `cmd:"" help:"check host prerequisites"` @@ -227,6 +227,17 @@ type recipeNewCmd struct { Backend string `help:"\"cloudinit\" for a cloud-init fragment; shell otherwise"` } +type recipeGuestCmd struct { + LS guestLsCmd `cmd:"" name:"ls" help:"one line per guest: name, init, package manager, backend, source"` + Show guestShowCmd `cmd:"" help:"the merged definition of one guest"` +} + +type guestLsCmd struct{} + +type guestShowCmd struct { + Name string `arg:"" help:"guest name (see guest ls)"` +} + type logsCmd struct { VM string `arg:"" optional:"" help:"vm name; omit to tail stoat's own log"` N int `short:"n" default:"50" help:"number of lines to tail"` @@ -386,9 +397,6 @@ func (g *grammar) toArgs(path string) (*Args, error) { case "apply": a.VM, a.Only, a.DryRun = g.Apply.VM, trimList(g.Apply.Only), g.Apply.DryRun - case "provision": - a.VM, a.Only, a.DryRun = g.Provision.VM, trimList(g.Provision.Only), g.Provision.DryRun - case "recipes": a.OS, a.Backend = g.Recipes.OS, g.Recipes.Backend @@ -406,6 +414,13 @@ func (g *grammar) toArgs(path string) (*Args, error) { a.Cmd, a.Sub = "recipe", "new" a.VM, a.OS, a.Backend = n.Name, n.OS, n.Backend + case "guest ls": + a.Cmd, a.Sub = "guest", "ls" + + case "guest show": + a.Cmd, a.Sub = "guest", "show" + a.VM = g.Guest.Show.Name + case "logs": l := g.Logs a.VM, a.N, a.Which = l.VM, l.N, core.Which(l.Which) diff --git a/internal/cli/json_test.go b/internal/cli/json_test.go index 7047c245..93e93e35 100644 --- a/internal/cli/json_test.go +++ b/internal/cli/json_test.go @@ -73,6 +73,9 @@ func TestJSONEnvelopeEveryCommand(t *testing.T) { {name: "prune", argv: []string{"prune"}, ok: true, exit: ExitOK}, {name: "logs", argv: []string{"logs"}, ok: true, exit: ExitOK}, {name: "recipe list", argv: []string{"recipe", "list"}, ok: true, exit: ExitOK}, + {name: "guest ls", argv: []string{"guest", "ls"}, ok: true, exit: ExitOK}, + {name: "guest show", argv: []string{"guest", "show", "alpine"}, ok: true, exit: ExitOK}, + {name: "guest show unknown", argv: []string{"guest", "show", "plan9"}, code: wire.CodeNotFound, exit: ExitFail}, {name: "forward show", argv: []string{"forward", "work"}, ok: true, exit: ExitOK}, // The fixture is a live VM, which has no qcow2 to snapshot; the point // here is the envelope, and no_disk is the honest answer for it. diff --git a/internal/cli/kong_test.go b/internal/cli/kong_test.go index ffa7e84b..5eae4924 100644 --- a/internal/cli/kong_test.go +++ b/internal/cli/kong_test.go @@ -279,7 +279,8 @@ func TestHelpListsEverySubcommand(t *testing.T) { "ls", "get", "create", "update", "up", "down", "wait", "rm", "clone", "exec", "ssh", "ssh-command", "cp", "forward", "images", "pull", "snapshot", "prune", "apply", "recipes", "check-recipes", - "recipe list", "recipe new", "logs", "doctor", "version", "help", + "recipe list", "recipe new", "guest ls", "guest show", + "logs", "doctor", "version", "help", } for _, args := range [][]string{{"help"}, {"--help"}, {"-h"}} { a, err := Parse(args) @@ -294,6 +295,19 @@ func TestHelpListsEverySubcommand(t *testing.T) { } } +// TestProvisionIsAnAliasOfApply pins that "provision" is a kong alias of +// "apply", not a second command: a.Cmd must read "apply" so the JSON +// envelope's cmd field and every downstream dispatch see one command. +func TestProvisionIsAnAliasOfApply(t *testing.T) { + a, err := Parse([]string{"provision", "work", "--dry-run"}) + if err != nil { + t.Fatal(err) + } + if a.Cmd != "apply" || a.VM != "work" || !a.DryRun { + t.Errorf("got %+v", a) + } +} + // TestProvisionIsHiddenFromHelp pins that the "provision" alias still parses // (TestParse's "provision" cases) while staying out of the command list a // user reads with --help, so the help surface names only the one spelling diff --git a/internal/cli/run_guest.go b/internal/cli/run_guest.go new file mode 100644 index 00000000..2eab513b --- /dev/null +++ b/internal/cli/run_guest.go @@ -0,0 +1,60 @@ +package cli + +import ( + "fmt" + "io" + + "github.com/novusedge/stoat/internal/cli/wire" + "github.com/novusedge/stoat/internal/core" +) + +// runGuest implements "guest ls" and "guest show". +func runGuest(a *Args, stdout, stderr io.Writer) int { + switch a.Sub { + case "ls": + gs := core.Guests() + if a.JSON { + return a.ok(stdout, wire.GuestList{Guests: wire.FromGuests(gs)}) + } + fmt.Fprintf(stdout, "%-10s %-8s %-8s %-10s %s\n", "NAME", "INIT", "PKG", "BACKEND", "SOURCE") + for _, g := range gs { + pkg := "" + if len(g.Pkg.Install) > 0 { + pkg = g.Pkg.Install[0] + } + fmt.Fprintf(stdout, "%-10s %-8s %-8s %-10s %s\n", g.Name, g.Init, pkg, g.DefaultBackend, g.Source) + } + return ExitOK + + case "show": + g, err := core.Guest(a.VM) + if err != nil { + return a.fail(stdout, stderr, err) + } + if a.JSON { + return a.ok(stdout, wire.GuestShow{Guest: wire.FromGuest(g)}) + } + w := wire.FromGuest(g) + fmt.Fprintf(stdout, "name: %s (%s)\n", w.Name, w.Source) + fmt.Fprintf(stdout, "init: %s\n", w.Init) + fmt.Fprintf(stdout, "shell: %s\n", w.Shell) + fmt.Fprintf(stdout, "default backend: %s\n", w.DefaultBackend) + fmt.Fprintf(stdout, "default ssh user: %s\n", w.DefaultSSHUser) + fmt.Fprintf(stdout, "escalate: %v\n", w.Escalate) + fmt.Fprintf(stdout, "capabilities: %v\n", w.Capabilities) + fmt.Fprintf(stdout, "aliases: %v\n", w.Aliases) + fmt.Fprintf(stdout, "pkg setup: %s\n", w.Pkg.Setup) + fmt.Fprintf(stdout, "pkg install: %v\n", w.Pkg.Install) + for _, k := range []string{"enable", "start", "stop", "restart", "status"} { + fmt.Fprintf(stdout, "svc %-8s %s\n", k+":", w.Svc[k]) + } + return ExitOK + } + // Unreachable: Parse rejects any action but ls/show. + if a.JSON { + _ = wire.NewEmitter(stdout).ResultErr(a.Cmd, wire.UsageError("guest: unknown action "+a.Sub)) + return ExitUsage + } + fmt.Fprintln(stderr, "stoat: guest: unknown action", a.Sub) + return ExitUsage +} diff --git a/internal/cli/run_recipes.go b/internal/cli/run_recipes.go index 3da2d1f1..4a919656 100644 --- a/internal/cli/run_recipes.go +++ b/internal/cli/run_recipes.go @@ -36,7 +36,7 @@ func recipesFor(osName, backend string) ([]core.Recipe, error) { case osName != "": b := backend if g, ok := guest.Lookup(osName); ok { - b = g.Backend + b = g.DefaultBackend } pairs = []pair{{osName, b}} case backend != "": @@ -45,7 +45,7 @@ func recipesFor(osName, backend string) ([]core.Recipe, error) { } default: for _, g := range guest.All() { - pairs = append(pairs, pair{g.Name, g.Backend}) + pairs = append(pairs, pair{g.Name, g.DefaultBackend}) } } diff --git a/internal/cli/wire/dto.go b/internal/cli/wire/dto.go index d7c0f5a3..71e85601 100644 --- a/internal/cli/wire/dto.go +++ b/internal/cli/wire/dto.go @@ -5,6 +5,7 @@ import ( "unicode/utf8" "github.com/novusedge/stoat/internal/core" + "github.com/novusedge/stoat/internal/guest" ) // DTOs, not json tags on core types (§3.1). Reasons, in order of weight: @@ -362,3 +363,71 @@ func FromExecResult(r core.ExecResult) ExecResult { } return out } + +// Guest is guest.OS for the wire. +type Guest struct { + Name string `json:"name"` + Init string `json:"init"` + Shell string `json:"shell"` + Installer string `json:"installer"` + DefaultBackend string `json:"default_backend"` + DefaultSSHUser string `json:"default_ssh_user"` + Escalate []string `json:"escalate"` + Capabilities []string `json:"capabilities"` + Aliases []string `json:"aliases"` + FilenameHints []string `json:"filename_hints"` + SeedPackages []string `json:"seed_packages"` + Pkg GuestPkg `json:"pkg"` + Svc map[string]string `json:"svc"` + Cmd map[string]string `json:"cmd"` + Backend map[string]map[string]any `json:"backend"` + Source string `json:"source"` +} + +type GuestPkg struct { + Setup string `json:"setup"` + Install []string `json:"install"` + Env map[string]string `json:"env"` + RuntimePackages map[string]string `json:"runtime_packages"` +} + +func FromGuest(o guest.OS) Guest { + nonNilMap := func(m map[string]string) map[string]string { + if m == nil { + return map[string]string{} + } + return m + } + backend := o.Backends + if backend == nil { + backend = map[string]map[string]any{} + } + return Guest{ + Name: o.Name, Init: string(o.Init), Shell: o.Shell, Installer: o.Installer, + DefaultBackend: o.DefaultBackend, DefaultSSHUser: o.DefaultSSHUser, + Escalate: nonNil(o.Escalate), Capabilities: nonNil(o.Capabilities), + Aliases: nonNil(o.Aliases), FilenameHints: nonNil(o.FilenameHints), + SeedPackages: nonNil(o.SeedPackages), + Pkg: GuestPkg{Setup: o.Pkg.Setup, Install: nonNil(o.Pkg.Install), Env: nonNilMap(o.Pkg.Env), RuntimePackages: nonNilMap(o.Pkg.RuntimePackages)}, + Svc: map[string]string{"enable": o.Svc.Enable, "start": o.Svc.Start, "stop": o.Svc.Stop, "restart": o.Svc.Restart, "status": o.Svc.Status}, + Cmd: nonNilMap(o.Cmd), Backend: backend, Source: o.Source, + } +} + +func FromGuests(os []guest.OS) []Guest { + out := make([]Guest, len(os)) + for i, o := range os { + out[i] = FromGuest(o) + } + return nonNil(out) +} + +// GuestList is `guest ls` data. +type GuestList struct { + Guests []Guest `json:"guests"` +} + +// GuestShow is `guest show` data. +type GuestShow struct { + Guest Guest `json:"guest"` +} diff --git a/internal/cli/wire/dto_test.go b/internal/cli/wire/dto_test.go index c78ab82b..e9244634 100644 --- a/internal/cli/wire/dto_test.go +++ b/internal/cli/wire/dto_test.go @@ -6,6 +6,7 @@ import ( "testing" "github.com/novusedge/stoat/internal/core" + "github.com/novusedge/stoat/internal/guest" ) // --- golden-file style: marshal each DTO and pin the exact JSON shape. --- @@ -250,6 +251,22 @@ func TestExecResultInvalidUTF8UsesBase64(t *testing.T) { } } +func TestGuestGolden(t *testing.T) { + o, ok := guest.Lookup("fedora") + if !ok { + t.Fatal("no bundled fedora guest") + } + got := marshal(t, FromGuest(o)) + for _, want := range []string{`"name":"fedora"`, `"init":"systemd"`, `"default_backend":"cloudinit"`, `"escalate":["sudo","-n"]`, `"capabilities":["dnf","systemd"]`, `"aliases":["rpm-family"]`, `"pkg":{"setup":"","install":["dnf","install","-y"]`, `"source":"bundled"`} { + if !strings.Contains(got, want) { + t.Errorf("missing %s in %s", want, got) + } + } + if strings.Contains(got, "null") { + t.Errorf("null in %s", got) + } +} + func marshal(t *testing.T, v any) string { t.Helper() b, err := json.Marshal(v) diff --git a/internal/cloudinit/cloudinit.go b/internal/cloudinit/cloudinit.go index 5a0bd36f..c9b09ad1 100644 --- a/internal/cloudinit/cloudinit.go +++ b/internal/cloudinit/cloudinit.go @@ -115,22 +115,32 @@ func userData(v *config.VM, pubkey string, recipeBodies []string) (string, error return buildArchive(docs) } +// SkipShares reports whether osName's guest.toml sets backend.cloudinit's +// skip_9p. A kernel without the 9p module makes cloud-init's mounts module +// fail the whole seed on every boot, and nofail does not cover an unknown +// filesystem type. +func SkipShares(osName string) bool { + o, ok := guest.Lookup(osName) + if !ok { + return false + } + skip, _ := o.Backends["cloudinit"]["skip_9p"].(bool) + return skip +} + // mountsDoc mounts the 9p exports. Cloud VMs used to get the exports on the // QEMU command line with nothing to mount them, so the share silently did // nothing. // // debian's cloud kernel (deb13-cloud) ships no 9p module, so the mount can -// never succeed and cloud-init's mounts module marks the whole seed as -// errored on every boot. Skip the mounts for debian; the 9p share does not -// work there. -// ponytail: keyed on the one bundled image without 9p. Add another OS here if -// the catalog gains a second 9p-less image. +// never succeed. Its guest.toml sets backend.cloudinit's skip_9p, and this +// document is skipped there. // // nofail keeps a share that drops out at runtime from holding up boot. The // host mount is ro, matching what QEMU enforces, so a write fails immediately // instead of after a remount that appears to succeed. func mountsDoc(v *config.VM) string { - if v.OS == "debian" { + if SkipShares(v.OS) { return "" } const opts = "trans=virtio,version=9p2000.L,%s,_netdev,nofail" diff --git a/internal/cloudinit/cloudinit_test.go b/internal/cloudinit/cloudinit_test.go index 11503855..0223a5de 100644 --- a/internal/cloudinit/cloudinit_test.go +++ b/internal/cloudinit/cloudinit_test.go @@ -544,3 +544,18 @@ func TestSeedFallsBackToBashForAnUnknownOS(t *testing.T) { } } } + +// SkipShares must read the flag from the guest file, not recognise debian by +// name: any future 9p-less guest sets backend.cloudinit.skip_9p and needs no +// Go change. +func TestSkipSharesReadsGuestFlag(t *testing.T) { + if !SkipShares("debian") { + t.Error("debian's guest.toml sets skip_9p; SkipShares must report true") + } + if SkipShares("fedora") { + t.Error("fedora keeps the 9p mounts") + } + if SkipShares("plan9") { + t.Error("an unknown guest must not skip shares") + } +} diff --git a/internal/cloudinit/scripts.go b/internal/cloudinit/scripts.go index 7615e725..10a4f151 100644 --- a/internal/cloudinit/scripts.go +++ b/internal/cloudinit/scripts.go @@ -3,6 +3,8 @@ package cloudinit import ( "fmt" "strings" + + "github.com/novusedge/stoat/internal/guest" ) // scriptDir is where WrapScripts places each script inside the guest, @@ -33,7 +35,10 @@ type Script struct { // meant to be handed to Seed alongside other recipe bodies, like any other // cloud recipe fragment (see userData in cloudinit.go). It goes through // withMergeHow and buildArchive the same way every other document does. -func WrapScripts(scripts []Script) string { +// prelude is prepended to each script's content and, when non-empty, run +// once via stoat_pkg_setup as the first runcmd entry, so the package index +// is refreshed before any script that installs a package runs. +func WrapScripts(scripts []Script, prelude string) string { if len(scripts) == 0 { return "" } @@ -41,12 +46,15 @@ func WrapScripts(scripts []Script) string { var wf, rc strings.Builder wf.WriteString("write_files:\n") rc.WriteString("runcmd:\n") + if prelude != "" { + rc.WriteString(fmt.Sprintf(" - sh -c %s\n", guest.ShQuote(prelude+"stoat_pkg_setup"))) + } for _, s := range scripts { path := fmt.Sprintf("%s/%s.sh", scriptDir, s.Name) wf.WriteString(fmt.Sprintf(" - path: %s\n", path)) wf.WriteString(" permissions: '0755'\n") wf.WriteString(" content: |\n") - wf.WriteString(indentBlock(s.Content)) + wf.WriteString(indentBlock(guest.WithPrelude(s.Content, prelude))) // The script runs, then drops a marker on success. core.Apply reads // MarkerDir over ssh to rebuild v.Applied post-boot. The && chain // leaves no marker for a script that failed, so a failed recipe stays diff --git a/internal/cloudinit/scripts_test.go b/internal/cloudinit/scripts_test.go index 85987ec6..e9963768 100644 --- a/internal/cloudinit/scripts_test.go +++ b/internal/cloudinit/scripts_test.go @@ -31,10 +31,10 @@ func parseWrapped(t *testing.T, body string) wrappedFragment { } func TestWrapScriptsEmpty(t *testing.T) { - if got := WrapScripts(nil); got != "" { + if got := WrapScripts(nil, ""); got != "" { t.Errorf("WrapScripts(nil) = %q, want empty string", got) } - if got := WrapScripts([]Script{}); got != "" { + if got := WrapScripts([]Script{}, ""); got != "" { t.Errorf("WrapScripts([]Script{}) = %q, want empty string", got) } } @@ -42,7 +42,7 @@ func TestWrapScriptsEmpty(t *testing.T) { func TestWrapScriptsSingleScript(t *testing.T) { body := WrapScripts([]Script{ {Name: "xfce", Content: "#!/bin/sh\napt-get install -y xfce4\n"}, - }) + }, "") f := parseWrapped(t, body) if len(f.WriteFiles) != 1 { @@ -80,7 +80,7 @@ func TestWrapScriptsPreservesOrder(t *testing.T) { {Name: "base", Content: "echo base\n"}, {Name: "docker", Content: "echo docker\n"}, {Name: "xfce", Content: "echo xfce\n"}, - }) + }, "") f := parseWrapped(t, body) wantPaths := []string{ @@ -118,7 +118,7 @@ func TestWrapScriptsPreservesOrder(t *testing.T) { // no useful error. func TestWrapScriptsMultilineContentSurvives(t *testing.T) { script := "#!/bin/sh\nset -e\n\nif true; then\n echo indented\nfi" - body := WrapScripts([]Script{{Name: "multi", Content: script}}) + body := WrapScripts([]Script{{Name: "multi", Content: script}}, "") f := parseWrapped(t, body) if len(f.WriteFiles) != 1 { @@ -136,7 +136,7 @@ func TestWrapScriptsMultilineContentSurvives(t *testing.T) { // way (see TestArchiveWriteFilesSurvives for the same contract on a // hand-written fragment). func TestWrapScriptsFragmentMergesIntoSeed(t *testing.T) { - fragment := WrapScripts([]Script{{Name: "xfce", Content: "echo hi\n"}}) + fragment := WrapScripts([]Script{{Name: "xfce", Content: "echo hi\n"}}, "") got := withMergeHow(fragment) if !strings.HasPrefix(got, "#cloud-config\n") { @@ -149,3 +149,17 @@ func TestWrapScriptsFragmentMergesIntoSeed(t *testing.T) { t.Errorf("merged fragment missing merge_how directive:\n%s", got) } } + +// A non-empty prelude runs once as the first runcmd entry, via +// stoat_pkg_setup, and is prepended to every script's own content. This is +// how the cloudinit path keeps the package index refresh and the recipe +// verbs behaving the same as the ssh path. +func TestWrapScriptsRunsSetupFirst(t *testing.T) { + got := WrapScripts([]Script{{Name: "x", Content: "#!/bin/sh\necho hi\n"}}, "P\n") + if !strings.Contains(got, "runcmd:\n - sh -c 'P\nstoat_pkg_setup'\n") { + t.Errorf("setup not first in runcmd:\n%s", got) + } + if !strings.Contains(got, " #!/bin/sh\n P\n echo hi\n") { + t.Errorf("prelude not after the shebang:\n%s", got) + } +} diff --git a/internal/config/config.go b/internal/config/config.go index 53272870..9d852ed3 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -3,6 +3,7 @@ package config import ( "fmt" + "io" "net" "os" "path/filepath" @@ -13,8 +14,13 @@ import ( "time" "github.com/BurntSushi/toml" + "github.com/novusedge/stoat/internal/tomlx" ) +// UnknownKeyWriter receives one line per unknown vm.toml key that Load +// tolerates. A test overrides it to capture the warning. +var UnknownKeyWriter io.Writer = os.Stderr + // PortForward is one user-declared host->guest TCP forward, additional to // the SSHPort forward every VM already gets. Validation (range, collisions // with other VMs' ports, privileged ports) lives in core.Forward, not here. @@ -201,19 +207,12 @@ func (v *VM) Save() error { // Load reads one VM by name. func Load(name string) (*VM, error) { dir := filepath.Join(Root(), name) - v := &VM{} - meta, err := toml.DecodeFile(filepath.Join(dir, "vm.toml"), v) - if err != nil { + // Absent allow_exec means true; the seed survives the decode, a written + // false overrides it. + v := &VM{AllowExec: true} + if err := tomlx.Decode(filepath.Join(dir, "vm.toml"), v, tomlx.Warn(UnknownKeyWriter)); err != nil { return nil, err } - // A vm.toml written before AllowExec existed has no "allow_exec" key. - // toml.Decode leaves that as Go's bool zero value: false. That would - // silently disable exec on every VM that predates the field, the - // opposite of the documented default. An absent key is corrected to - // true here instead of trusted as a real false. - if !meta.IsDefined("allow_exec") { - v.AllowExec = true - } v.Dir = dir v.Share = Expand(v.Share) return v, nil diff --git a/internal/config/config_test.go b/internal/config/config_test.go index 20f134fc..7252d20e 100644 --- a/internal/config/config_test.go +++ b/internal/config/config_test.go @@ -1,15 +1,35 @@ package config import ( + "bytes" "fmt" "net" "os" "path/filepath" "reflect" + "strings" "testing" "time" ) +func TestLoadWarnsUnknownKeyToStderr(t *testing.T) { + root := t.TempDir() + t.Setenv("STOAT_HOME", root) + dir := filepath.Join(root, "w") + os.MkdirAll(dir, 0o755) + os.WriteFile(filepath.Join(dir, "vm.toml"), []byte("name = \"w\"\nmode = \"live\"\ncpus_ = 2\n"), 0o644) + var w bytes.Buffer + UnknownKeyWriter = &w + t.Cleanup(func() { UnknownKeyWriter = os.Stderr }) + v, err := Load("w") + if err != nil { + t.Fatal(err) + } + if v.Name != "w" || !strings.Contains(w.String(), `unknown key "cpus_"`) { + t.Errorf("v=%+v warn=%q", v, w.String()) + } +} + func TestSaveLoadRoundtrip(t *testing.T) { t.Setenv("STOAT_HOME", t.TempDir()) if err := EnsureRoot(); err != nil { diff --git a/internal/core/clone.go b/internal/core/clone.go index 0114d12b..d1192d06 100644 --- a/internal/core/clone.go +++ b/internal/core/clone.go @@ -9,6 +9,7 @@ import ( "github.com/novusedge/stoat/internal/cloudinit" "github.com/novusedge/stoat/internal/config" + "github.com/novusedge/stoat/internal/guest" "github.com/novusedge/stoat/internal/keys" "github.com/novusedge/stoat/internal/qemu" "github.com/novusedge/stoat/internal/recipes" @@ -215,8 +216,12 @@ func cloneCloud(src, clone *config.VM) error { } scripts = append(scripts, cloudinit.Script{Name: name, Content: body}) } + var prelude string + if o, ok := guest.Lookup(clone.OS); ok { + prelude = guest.Prelude(o, "sh") + } var bodies []string - if frag := cloudinit.WrapScripts(scripts); frag != "" { + if frag := cloudinit.WrapScripts(scripts, prelude); frag != "" { bodies = []string{frag} } _, err = cloudinit.Seed(clone, pub, bodies) diff --git a/internal/core/guests.go b/internal/core/guests.go new file mode 100644 index 00000000..991de0d5 --- /dev/null +++ b/internal/core/guests.go @@ -0,0 +1,24 @@ +package core + +import ( + "errors" + "fmt" + + "github.com/novusedge/stoat/internal/guest" +) + +// ErrUnknownGuest marks a vm.toml whose os names no loaded guest. It is +// wrapped by ErrBroken so List and Get show the VM instead of hiding it. +var ErrUnknownGuest = errors.New("unknown guest") + +// Guests returns every loaded guest, sorted by name. +func Guests() []guest.OS { return guest.All() } + +// Guest returns one guest by name. +func Guest(name string) (guest.OS, error) { + o, ok := guest.Lookup(name) + if !ok { + return guest.OS{}, fmt.Errorf("%w: guest %s", ErrNotFound, name) + } + return o, nil +} diff --git a/internal/core/guests_test.go b/internal/core/guests_test.go new file mode 100644 index 00000000..1434d907 --- /dev/null +++ b/internal/core/guests_test.go @@ -0,0 +1,19 @@ +package core + +import ( + "errors" + "testing" +) + +func TestGuestsSorted(t *testing.T) { + gs := Guests() + if len(gs) < 5 || gs[0].Name != "alpine" || gs[1].Name != "arch" { + t.Errorf("got %d guests, first two %q %q", len(gs), gs[0].Name, gs[1].Name) + } +} + +func TestGuestUnknownIsNotFound(t *testing.T) { + if _, err := Guest("plan9"); !errors.Is(err, ErrNotFound) { + t.Errorf("err = %v", err) + } +} diff --git a/internal/core/vm.go b/internal/core/vm.go index e0da1d43..a747f050 100644 --- a/internal/core/vm.go +++ b/internal/core/vm.go @@ -9,6 +9,7 @@ import ( "time" "github.com/novusedge/stoat/internal/config" + "github.com/novusedge/stoat/internal/guest" "github.com/novusedge/stoat/internal/iso" "github.com/novusedge/stoat/internal/qemu" ) @@ -188,9 +189,26 @@ func load(name string) (*config.VM, error) { if err != nil { return nil, fmt.Errorf("%w: %s: %v", ErrBroken, name, err) } + if err := checkGuest(v); err != nil { + return nil, err + } return v, nil } +// checkGuest reports ErrUnknownGuest, wrapped in ErrBroken, when v.OS names +// no loaded guest. An empty OS (a live/disk VM predating the OS field, or one +// that never set it) is not checked: emptiness has its own fallbacks +// elsewhere and is not a broken state. +func checkGuest(v *config.VM) error { + if v.OS == "" { + return nil + } + if _, ok := guest.Lookup(v.OS); !ok { + return fmt.Errorf("%w: %s: %w %q; run stoat guest ls", ErrBroken, filepath.Base(v.Dir), ErrUnknownGuest, v.OS) + } + return nil +} + // fromConfig builds the point-in-time view for a VM that parsed cleanly. // State and Paths are the two things config.VM cannot answer for itself. func fromConfig(v *config.VM) VM { @@ -307,6 +325,10 @@ func List() ([]VM, error) { } var out []VM for _, cv := range cvms { + if err := checkGuest(cv); err != nil { + out = append(out, VM{Name: filepath.Base(cv.Dir), State: StateBroken, Error: err.Error()}) + continue + } out = append(out, fromConfig(cv)) } diff --git a/internal/core/vm_test.go b/internal/core/vm_test.go index 82571b5a..e7ca65aa 100644 --- a/internal/core/vm_test.go +++ b/internal/core/vm_test.go @@ -4,6 +4,7 @@ import ( "errors" "os" "path/filepath" + "strings" "testing" "time" @@ -88,6 +89,33 @@ func TestGetUnknownVM(t *testing.T) { } } +func TestUnknownGuestIsBroken(t *testing.T) { + root(t) + v := &config.VM{Name: "x", Mode: "cloud", OS: "plan9", RAM: 1024, CPUs: 1, SSHPort: 2200} + if err := v.Save(); err != nil { + t.Fatal(err) + } + got, err := Get("x") + if err != nil { + t.Fatal(err) + } + if got.State != StateBroken || !strings.Contains(got.Error, `unknown guest "plan9"`) { + t.Errorf("state %q error %q", got.State, got.Error) + } +} + +func TestEmptyOSIsNotBroken(t *testing.T) { + root(t) + v := &config.VM{Name: "y", Mode: "live", RAM: 1024, CPUs: 1, SSHPort: 2201} + if err := v.Save(); err != nil { + t.Fatal(err) + } + got, err := Get("y") + if err != nil || got.State == StateBroken { + t.Errorf("empty os must keep its fallbacks: %v %v", got.State, err) + } +} + // A broken vm.toml must come back as a VM the caller can see (StateBroken), // not as a bare error indistinguishable from "no such VM", otherwise there // is no way to ever offer deleting it. diff --git a/internal/guest/bundled/alpine.toml b/internal/guest/bundled/alpine.toml new file mode 100644 index 00000000..25906212 --- /dev/null +++ b/internal/guest/bundled/alpine.toml @@ -0,0 +1,36 @@ +# This file is one guest OS. Spec 1 covers sh-shaped guests with an +# imperative package manager; NixOS, OpenBSD, and Windows need another model. +schema = 1 +name = "alpine" +init = "openrc" +shell = "/bin/ash" +installer = "setup-alpine" +default_backend = "apkovl" +default_ssh_user = "root" +escalate = ["sudo", "-n"] +capabilities = ["apk"] +aliases = [] +filename_hints = ["alpine"] +seed_packages = ["sudo"] + +[pkg] +setup = "apk update" +install = ["apk", "--wait", "60", "add"] +scaffold_setup = """ +# -c enables the community repository (docker, tailscale and most of what +# you would want live there, not in main); -1 picks a mirror and refreshes +# the indexes, so a separate `apk update` is redundant. +setup-apkrepos -c -1 +""" +scaffold_install = "apk add " +runtime_packages = { python3 = "python3" } + +[svc] +enable = "rc-update add {name} default" +start = "rc-service {name} start" +stop = "rc-service {name} stop" +restart = "rc-service {name} restart" +status = "rc-service {name} status" + +[backend.cloudinit] +skip_9p = false diff --git a/internal/guest/bundled/arch.toml b/internal/guest/bundled/arch.toml new file mode 100644 index 00000000..9146f588 --- /dev/null +++ b/internal/guest/bundled/arch.toml @@ -0,0 +1,30 @@ +schema = 1 +name = "arch" +init = "systemd" +shell = "/bin/bash" +installer = "" +default_backend = "cloudinit" +default_ssh_user = "stoat" +escalate = ["sudo", "-n"] +capabilities = ["pacman"] +aliases = [] +filename_hints = ["arch"] +seed_packages = [] + +[pkg] +setup = "pacman -Sy --noconfirm" +install = ["pacman", "-S", "--noconfirm"] +scaffold_setup = "pacman -Sy --noconfirm\n" +scaffold_install = "pacman -S --noconfirm " +# Arch names its Python 3 package "python". +runtime_packages = { python3 = "python" } + +[svc] +enable = "systemctl enable {name}" +start = "systemctl start {name}" +stop = "systemctl stop {name}" +restart = "systemctl restart {name}" +status = "systemctl status {name}" + +[backend.cloudinit] +skip_9p = false diff --git a/internal/guest/bundled/debian.toml b/internal/guest/bundled/debian.toml new file mode 100644 index 00000000..3ec1b9fe --- /dev/null +++ b/internal/guest/bundled/debian.toml @@ -0,0 +1,35 @@ +schema = 1 +name = "debian" +init = "systemd" +shell = "/bin/bash" +installer = "" +default_backend = "cloudinit" +default_ssh_user = "stoat" +escalate = ["sudo", "-n"] +capabilities = ["apt"] +aliases = ["debian-family"] +filename_hints = ["debian"] +seed_packages = [] + +[pkg] +setup = "apt-get update" +install = ["apt-get", "install", "-y"] +env = { DEBIAN_FRONTEND = "noninteractive" } +scaffold_setup = """ +export DEBIAN_FRONTEND=noninteractive +apt-get update +""" +scaffold_install = "apt-get install -y " +runtime_packages = { python3 = "python3" } + +[svc] +enable = "systemctl enable {name}" +start = "systemctl start {name}" +stop = "systemctl stop {name}" +restart = "systemctl restart {name}" +status = "systemctl status {name}" + +# Debian's cloud kernel ships no 9p module; the cloudinit backend must not +# rely on a 9p share mount for this guest. +[backend.cloudinit] +skip_9p = true diff --git a/internal/guest/bundled/fedora.toml b/internal/guest/bundled/fedora.toml new file mode 100644 index 00000000..7ad5c097 --- /dev/null +++ b/internal/guest/bundled/fedora.toml @@ -0,0 +1,29 @@ +schema = 1 +name = "fedora" +init = "systemd" +shell = "/bin/bash" +installer = "" +default_backend = "cloudinit" +default_ssh_user = "stoat" +escalate = ["sudo", "-n"] +capabilities = ["dnf"] +aliases = ["rpm-family"] +filename_hints = ["fedora"] +seed_packages = [] + +[pkg] +setup = "" +install = ["dnf", "install", "-y"] +scaffold_setup = "" +scaffold_install = "dnf install -y " +runtime_packages = { python3 = "python3" } + +[svc] +enable = "systemctl enable {name}" +start = "systemctl start {name}" +stop = "systemctl stop {name}" +restart = "systemctl restart {name}" +status = "systemctl status {name}" + +[backend.cloudinit] +skip_9p = false diff --git a/internal/guest/bundled/ubuntu.toml b/internal/guest/bundled/ubuntu.toml new file mode 100644 index 00000000..cb9e4237 --- /dev/null +++ b/internal/guest/bundled/ubuntu.toml @@ -0,0 +1,33 @@ +schema = 1 +name = "ubuntu" +init = "systemd" +shell = "/bin/bash" +installer = "" +default_backend = "cloudinit" +default_ssh_user = "stoat" +escalate = ["sudo", "-n"] +capabilities = ["apt"] +aliases = ["debian-family"] +filename_hints = ["ubuntu"] +seed_packages = [] + +[pkg] +setup = "apt-get update" +install = ["apt-get", "install", "-y"] +env = { DEBIAN_FRONTEND = "noninteractive" } +scaffold_setup = """ +export DEBIAN_FRONTEND=noninteractive +apt-get update +""" +scaffold_install = "apt-get install -y " +runtime_packages = { python3 = "python3" } + +[svc] +enable = "systemctl enable {name}" +start = "systemctl start {name}" +stop = "systemctl stop {name}" +restart = "systemctl restart {name}" +status = "systemctl status {name}" + +[backend.cloudinit] +skip_9p = false diff --git a/internal/guest/guest.go b/internal/guest/guest.go index 111ac8e5..70107b81 100644 --- a/internal/guest/guest.go +++ b/internal/guest/guest.go @@ -5,8 +5,9 @@ // scaffolded recipe uses, and the filename hints that recognise the OS in a // bring-your-own image. // -// Adding an OS means adding one entry to registry below. Every field must -// be filled. A missing field fails silently: an unselectable catalog entry, +// Adding an OS means adding one guest.toml, bundled or dropped in +// ~/.stoat/guests/. validate rejects a file missing a required field: a +// missing field otherwise fails silently, as an unselectable catalog entry, // an OS that hands cloud-init a shell the image lacks, or a VM offered zero // recipes. See docs/design/guest-subsystem.md for the incident that made // this the rule. @@ -21,168 +22,115 @@ type InitSystem string const ( InitSystemd InitSystem = "systemd" InitOpenRC InitSystem = "openrc" + InitRC InitSystem = "rc" ) -// OS is one guest operating system's facts, gathered from the sites that -// used to decide them ad hoc (see docs/design/guest-subsystem.md §4 for the -// full inventory of call sites this replaces). +// OS is one guest operating system's facts, loaded from a guest.toml. See +// docs/reference/guest.md for the file. type OS struct { - Name string // canonical identity, matches vm.toml's os field + Schema int `toml:"schema"` + Name string `toml:"name"` // Shell is the login shell for a seeded account. It must exist in the // image. cloud-init's user module fails outright on a missing shell, // leaving no account and no authorized_keys. The only symptom is // "Permission denied (publickey)". - Shell string - - // SeedPackages are packages the base seed assumes are present but the - // image does not ship. Alpine needs sudo: the users: sudo key writes a - // sudoers fragment, but Alpine's cloud-init aport ships doas, not sudo. - SeedPackages []string - - // Backend names how this OS is provisioned: "apkovl", "cloudinit", or - // "ssh". It stays a string here, not the Backend interface itself. - // Package guest holds OS data only and stays a zero-import leaf; the - // interface lives in internal/backend, one level up. See - // docs/design/guest-subsystem.md §3.2. - Backend string - - // Init is the guest's init system. It is a separate field, not derived - // from Backend. Today apkovl and OpenRC both mean Alpine, but the two - // facts are independent: a BYO cloudinit image could run OpenRC, or a - // future apkovl-based distro could run systemd. A recipe's "requires - // systemd" needs the real answer, not a Backend guess. - Init InitSystem + Shell string `toml:"shell"` + + // Init is the guest's init system, a fact independent of the backend: a + // BYO cloudinit image can run OpenRC. A recipe's "requires systemd" + // needs the real answer. + Init InitSystem `toml:"init"` // Installer is the interactive install command named in UI hints. - // Empty means "the installer" generically. - Installer string - - // DefaultSSHUser is who to connect as when nothing overrides it. - DefaultSSHUser string - - // PkgSetup is the package-manager preamble a scaffolded recipe needs - // before it can install anything: enabling a repository, refreshing - // indexes, or nothing if the OS needs neither. - PkgSetup string - - // PkgInstall is the install verb a scaffolded recipe appends package - // names to, e.g. "apk add ". - PkgInstall string - - // FilenameHints recognise this OS in a BYO image filename. An empty OS - // on a BYO image is dangerous: it silently selects every default, which - // is how an Alpine image ends up asked for /bin/bash. - FilenameHints []string - - // CloudRecipes reports whether an OS is offered the shared - // "*.cloud.yaml" fragment set. True does not mean every shared fragment - // applies. cloud-init's packages: list has no per-distro syntax, so - // devtools.cloud.yaml's names work for Alpine's apk too, but - // xfce.cloud.yaml's systemd runcmd does not, since Alpine runs OpenRC. - // Alpine gets devtools.cloud.yaml from the shared set plus its own - // xfce.alpine.cloud.yaml, the same way Fedora gets its own - // xfce.fedora.cloud.yaml. See recipes/recipes.go's List doc comment for - // how a per-OS fragment and the shared set combine. - CloudRecipes bool + // Empty means "the installer". + Installer string `toml:"installer"` + + // DefaultBackend and DefaultSSHUser seed the VM fields at create time. + // A catalog entry overrides both (iso.Entry), and every code path reads + // the VM field, never these. + DefaultBackend string `toml:"default_backend"` + DefaultSSHUser string `toml:"default_ssh_user"` + + // Escalate is the argv that runs a command as root for a non-root ssh + // user. sshx applies it only when the VM's ssh user is not root. + Escalate []string `toml:"escalate"` + + // Capabilities feed recipe.toml's `requires`. The loader appends Init. + Capabilities []string `toml:"capabilities"` + + // Aliases are extra keys a recipe's [scripts] map may use for this OS, + // tried after Name. + Aliases []string `toml:"aliases"` + + // FilenameHints recognise this OS in a BYO image filename. + FilenameHints []string `toml:"filename_hints"` + + // SeedPackages are packages the cloud-init seed assumes but the image + // does not ship. Alpine needs sudo: its cloud-init aport ships doas. + SeedPackages []string `toml:"seed_packages"` + + Pkg Pkg `toml:"pkg"` + Svc Svc `toml:"svc"` + Cmd map[string]string `toml:"cmd"` + + // Backends holds one opaque table per backend name. The backend package + // that owns the name decodes it; this package never reads inside. + Backends map[string]map[string]any `toml:"backend"` + + // Source is where the definition came from: "bundled", "user", or + // "bundled+user" for a user file merged over a bundled one. + Source string `toml:"-"` } -// registry is the single home for every guest-OS fact. See the package doc -// comment: adding an OS means adding one complete entry here. -var registry = []OS{ - { - Name: "alpine", - Shell: "/bin/ash", - SeedPackages: []string{"sudo"}, - Backend: "apkovl", - Init: InitOpenRC, // Alpine has shipped OpenRC as its init since its first release; it has never carried systemd - Installer: "setup-alpine", - DefaultSSHUser: "root", - PkgSetup: "# -c enables the community repository (docker, tailscale and most of what\n" + - "# you would want live there, not in main); -1 picks a mirror and refreshes\n" + - "# the indexes, so a separate `apk update` is redundant.\nsetup-apkrepos -c -1\n", - PkgInstall: "apk add ", - FilenameHints: []string{"alpine"}, - CloudRecipes: true, - }, - { - Name: "ubuntu", - Shell: "/bin/bash", - SeedPackages: nil, - Backend: "cloudinit", - Init: InitSystemd, // Ubuntu switched from Upstart to systemd in 15.04 (2015); every supported release since is systemd - Installer: "", - DefaultSSHUser: "stoat", - PkgSetup: "export DEBIAN_FRONTEND=noninteractive\napt-get update\n", - PkgInstall: "apt-get install -y ", - FilenameHints: []string{"ubuntu"}, - CloudRecipes: true, - }, - { - Name: "debian", - Shell: "/bin/bash", - SeedPackages: nil, - Backend: "cloudinit", - Init: InitSystemd, // Debian made systemd the default init in Debian 8 "jessie" (2015) - Installer: "", - DefaultSSHUser: "stoat", - PkgSetup: "export DEBIAN_FRONTEND=noninteractive\napt-get update\n", - PkgInstall: "apt-get install -y ", - FilenameHints: []string{"debian"}, - CloudRecipes: true, - }, - { - Name: "fedora", - Shell: "/bin/bash", - SeedPackages: nil, - Backend: "cloudinit", - Init: InitSystemd, // Fedora was systemd's original adopter, default since Fedora 15 (2011) - Installer: "", - DefaultSSHUser: "stoat", - PkgSetup: "", - PkgInstall: "dnf install -y ", - FilenameHints: []string{"fedora"}, - CloudRecipes: false, - }, - { - Name: "arch", - Shell: "/bin/bash", - SeedPackages: nil, - Backend: "cloudinit", - Init: InitSystemd, // Arch replaced its own initscripts with systemd as the default in 2012 - Installer: "", - DefaultSSHUser: "stoat", - PkgSetup: "pacman -Sy --noconfirm\n", - PkgInstall: "pacman -S --noconfirm ", - FilenameHints: []string{"arch"}, - CloudRecipes: true, - }, +// Pkg is the package-manager surface. +type Pkg struct { + // Setup refreshes the index. Provision runs it once before the first + // recipe. Empty means the manager needs no refresh (dnf). + Setup string `toml:"setup"` + // Install is argv; apk carries "--wait 60" for the lock race. + Install []string `toml:"install"` + // Env is exported in the prelude (DEBIAN_FRONTEND). + Env map[string]string `toml:"env"` + // ScaffoldSetup and ScaffoldInstall are display text for `recipe new`. + ScaffoldSetup string `toml:"scaffold_setup"` + ScaffoldInstall string `toml:"scaffold_install"` + // RuntimePackages maps a recipe runtime to the package that provides it. + RuntimePackages map[string]string `toml:"runtime_packages"` } -// Lookup returns the OS declared for name, or false if name is not in the -// registry. It never guesses: an unknown OS is reported as unknown, and it -// is the caller's job to decide what that means. -func Lookup(name string) (OS, bool) { - for _, o := range registry { - if o.Name == name { - return o, true - } - } - return OS{}, false +// Svc is the service surface. Each value is a template: {name} renders to +// the first argument, and a template without {name} gets "$@" appended. +type Svc struct { + Enable string `toml:"enable"` + Start string `toml:"start"` + Stop string `toml:"stop"` + Restart string `toml:"restart"` + Status string `toml:"status"` } -// All returns every declared OS. -func All() []OS { - out := make([]OS, len(registry)) - copy(out, registry) - return out +// loaded is the active set: bundled at init, then Load merges user files +// over it. Lookup before Load sees bundled guests only, so a broken user +// file cannot take the bundled set down. +var loaded = loadBundled() + +// Lookup returns the OS declared for name. It never guesses: an unknown OS +// is reported as unknown, and it is the caller's job to decide what that +// means. +func Lookup(name string) (OS, bool) { + o, ok := loaded[name] + return o, ok } -// Names returns the canonical name of every declared OS, in registry order. -func Names() []string { - out := make([]string, len(registry)) - for i, o := range registry { - out[i] = o.Name +// All returns every loaded OS, sorted by name. +func All() []OS { + names := sortedNames(loaded) + out := make([]OS, len(names)) + for i, n := range names { + out[i] = loaded[n] } return out } + +// Names returns every loaded name, sorted. +func Names() []string { return sortedNames(loaded) } diff --git a/internal/guest/guest_test.go b/internal/guest/guest_test.go index 0dbc8ae0..230fb609 100644 --- a/internal/guest/guest_test.go +++ b/internal/guest/guest_test.go @@ -80,8 +80,8 @@ func TestEveryDeclarationIsComplete(t *testing.T) { if o.Shell == "" { t.Errorf("%s has no Shell", o.Name) } - if o.Backend == "" { - t.Errorf("%s has no Backend", o.Name) + if o.DefaultBackend == "" { + t.Errorf("%s has no DefaultBackend", o.Name) } if o.Init == "" { t.Errorf("%s has no Init", o.Name) @@ -89,8 +89,8 @@ func TestEveryDeclarationIsComplete(t *testing.T) { if o.DefaultSSHUser == "" { t.Errorf("%s has no DefaultSSHUser", o.Name) } - if o.PkgInstall == "" { - t.Errorf("%s has no PkgInstall", o.Name) + if o.Pkg.ScaffoldInstall == "" { + t.Errorf("%s has no Pkg.ScaffoldInstall", o.Name) } if len(o.FilenameHints) == 0 { t.Errorf("%s has no FilenameHints, so a BYO image of it can never be recognised", o.Name) diff --git a/internal/guest/load.go b/internal/guest/load.go new file mode 100644 index 00000000..71468e15 --- /dev/null +++ b/internal/guest/load.go @@ -0,0 +1,233 @@ +package guest + +import ( + "embed" + "fmt" + "io" + "io/fs" + "os" + "path/filepath" + "sort" + "strings" + + "github.com/novusedge/stoat/internal/tomlx" +) + +//go:embed bundled/*.toml +var bundledFS embed.FS + +// schemaVersion is the guest.toml schema this stoat reads. +const schemaVersion = 1 + +var initNames = map[InitSystem]bool{InitSystemd: true, InitOpenRC: true, InitRC: true} + +// parseFile reads one guest.toml. Unknown keys are an error: the file is +// hand-written and a typo must not become a silent default. +func parseFile(path string) (OS, error) { + var o OS + if err := tomlx.Decode(path, &o, tomlx.Reject, tomlx.Schema(schemaVersion)); err != nil { + return OS{}, err + } + if o.Schema == 0 { + return OS{}, fmt.Errorf("%s: missing schema", path) + } + return o, nil +} + +// loadBundled parses every embedded file. A bundled file that fails is a +// build error, so it panics. +func loadBundled() map[string]OS { + out := map[string]OS{} + entries, err := fs.ReadDir(bundledFS, "bundled") + if err != nil { + panic(err) + } + for _, e := range entries { + tmp, err := os.CreateTemp("", "stoat-guest-*.toml") + if err != nil { + panic(err) + } + b, _ := bundledFS.ReadFile("bundled/" + e.Name()) + if _, err := tmp.Write(b); err != nil { + panic(err) + } + tmp.Close() + o, err := parseFile(tmp.Name()) + os.Remove(tmp.Name()) + if err != nil { + panic(fmt.Sprintf("bundled guest %s: %v", e.Name(), err)) + } + if err := validate(o); err != nil { + panic(fmt.Sprintf("bundled guest %s: %v", e.Name(), err)) + } + o.Source = "bundled" + out[o.Name] = normalize(o) + } + return out +} + +// validate reports the first missing required field or inconsistency. +func validate(o OS) error { + req := []struct { + name string + ok bool + }{ + {"name", o.Name != ""}, + {"init", o.Init != ""}, + {"shell", o.Shell != ""}, + {"default_backend", o.DefaultBackend != ""}, + {"default_ssh_user", o.DefaultSSHUser != ""}, + {"escalate", o.Escalate != nil}, + {"capabilities", o.Capabilities != nil}, + {"seed_packages", o.SeedPackages != nil}, + {"pkg.install", len(o.Pkg.Install) > 0}, + {"pkg.scaffold_install", o.Pkg.ScaffoldInstall != ""}, + {"pkg.runtime_packages", o.Pkg.RuntimePackages != nil}, + {"svc.enable", o.Svc.Enable != ""}, + {"svc.start", o.Svc.Start != ""}, + {"svc.stop", o.Svc.Stop != ""}, + {"svc.restart", o.Svc.Restart != ""}, + {"svc.status", o.Svc.Status != ""}, + } + for _, r := range req { + if !r.ok { + return fmt.Errorf("guest.toml: %s: missing %s", o.Name, r.name) + } + } + if !initNames[o.Init] { + return fmt.Errorf("guest.toml: %s: init %q is not systemd, openrc, or rc", o.Name, o.Init) + } + for _, c := range o.Capabilities { + if initNames[InitSystem(c)] && InitSystem(c) != o.Init { + return fmt.Errorf("guest.toml: %s: capabilities lists init %q but init is %q", o.Name, c, o.Init) + } + } + // preludePython embeds each template in a single-quoted Python literal. + // A quote inside the template closes that literal and the recipe fails + // at run time with a SyntaxError, so reject it at load. + tmpls := map[string]string{ + "svc.enable": o.Svc.Enable, "svc.start": o.Svc.Start, "svc.stop": o.Svc.Stop, + "svc.restart": o.Svc.Restart, "svc.status": o.Svc.Status, + } + for name, t := range o.Cmd { + tmpls["cmd."+name] = t + } + for _, name := range sortedKeys(tmpls) { + if strings.Contains(tmpls[name], "'") { + return fmt.Errorf("guest.toml: %s: %s contains a single quote, which the python prelude cannot render", o.Name, name) + } + } + return nil +} + +// normalize appends init to capabilities once and fills nil maps, so a +// consumer never branches on nil. +func normalize(o OS) OS { + has := false + for _, c := range o.Capabilities { + if c == string(o.Init) { + has = true + } + } + if !has { + o.Capabilities = append(append([]string{}, o.Capabilities...), string(o.Init)) + } + if o.Cmd == nil { + o.Cmd = map[string]string{} + } + if o.Backends == nil { + o.Backends = map[string]map[string]any{} + } + if o.Pkg.Env == nil { + o.Pkg.Env = map[string]string{} + } + return o +} + +// sortedNames returns the keys of m in order, for a stable All(). +func sortedNames(m map[string]OS) []string { + names := make([]string, 0, len(m)) + for n := range m { + names = append(names, n) + } + sort.Strings(names) + return names +} + +// Load reads every /*.toml over the bundled set. A user file whose +// name matches a bundled guest merges per field: a field the file sets +// wins, an absent one keeps the bundled value. A new name needs every +// required field. The first bad file is the error, and the loaded set is +// left as it was. +func Load(dir string) error { + files, err := userFiles(dir) + if err != nil { + return err + } + next := map[string]OS{} + for k, v := range loaded { + next[k] = v + } + for _, path := range files { + var o OS + var src string + var probe struct { + Name string `toml:"name"` + } + if err := tomlx.Decode(path, &probe, tomlx.Warn(io.Discard)); err != nil { + return err + } + if base, ok := next[probe.Name]; ok && base.Source == "bundled" { + // Decoding over a copy is the merge: a scalar or list the file + // sets replaces, an absent field keeps the bundled value, and + // an inner [backend.x] table replaces whole. + o = base + src = "bundled+user" + } else { + src = "user" + } + if err := tomlx.Decode(path, &o, tomlx.Reject, tomlx.Schema(schemaVersion)); err != nil { + return err + } + if o.Schema == 0 { + return fmt.Errorf("%s: missing schema", path) + } + if err := validate(o); err != nil { + return fmt.Errorf("%s: %w", path, err) + } + o.Source = src + next[o.Name] = normalize(o) + } + loaded = next + return nil +} + +// Capabilities maps each capability to the guests that declare it. It +// replaces the table recipes/manifest.go used to hold. +func Capabilities() map[string][]string { + out := map[string][]string{} + for _, o := range All() { + for _, c := range o.Capabilities { + out[c] = append(out[c], o.Name) + } + } + return out +} + +// userFiles lists /*.toml, or nothing when dir is absent. +func userFiles(dir string) ([]string, error) { + entries, err := os.ReadDir(dir) + if err != nil { + if os.IsNotExist(err) { + return nil, nil + } + return nil, err + } + var out []string + for _, e := range entries { + if !e.IsDir() && strings.HasSuffix(e.Name(), ".toml") { + out = append(out, filepath.Join(dir, e.Name())) + } + } + return out, nil +} diff --git a/internal/guest/load_test.go b/internal/guest/load_test.go new file mode 100644 index 00000000..cf2384ed --- /dev/null +++ b/internal/guest/load_test.go @@ -0,0 +1,154 @@ +package guest + +import ( + "os" + "path/filepath" + "strings" + "testing" +) + +// The five bundled files must parse, pass validation, and (via normalize) +// carry their own init system in capabilities. +func TestBundledFilesParseAndValidate(t *testing.T) { + got := loadBundled() + for _, name := range []string{"alpine", "ubuntu", "debian", "fedora", "arch"} { + o, ok := got[name] + if !ok { + t.Fatalf("no bundled %s", name) + } + if o.Source != "bundled" { + t.Errorf("%s source = %q, want bundled", name, o.Source) + } + if err := validate(o); err != nil { + t.Errorf("%s: %v", name, err) + } + } + alpine := got["alpine"] + last := alpine.Capabilities[len(alpine.Capabilities)-1] + if last != "openrc" { + t.Errorf("alpine capabilities = %v, want openrc last", alpine.Capabilities) + } +} + +func TestValidateMissingField(t *testing.T) { + o := loadBundled()["fedora"] + o.Pkg.Install = nil + err := validate(o) + if err == nil || !strings.Contains(err.Error(), "fedora: missing pkg.install") { + t.Errorf("err = %v", err) + } +} + +func TestValidateRejectsForeignInit(t *testing.T) { + o := loadBundled()["alpine"] + o.Capabilities = append(o.Capabilities, "systemd") + err := validate(o) + if err == nil || !strings.Contains(err.Error(), `capabilities lists init "systemd" but init is "openrc"`) { + t.Errorf("err = %v", err) + } +} + +// A quote in a template survives the sh prelude but breaks the python one, +// where the template goes inside a single-quoted literal. Rejecting it at +// load turns a SyntaxError mid-recipe into a message about the file. +func TestValidateRejectsQuoteInTemplate(t *testing.T) { + o := loadBundled()["fedora"] + o.Svc.Enable = "systemctl enable 'x' {name}" + err := validate(o) + if err == nil || !strings.Contains(err.Error(), "svc.enable contains a single quote") { + t.Errorf("err = %v", err) + } + o = loadBundled()["fedora"] + o.Cmd = map[string]string{"reboot": "shutdown -r 'now'"} + if err := validate(o); err == nil || !strings.Contains(err.Error(), "cmd.reboot contains a single quote") { + t.Errorf("err = %v", err) + } +} + +func TestParseFileRejectsUnknownKey(t *testing.T) { + p := filepath.Join(t.TempDir(), "x.toml") + body := "schema = 1\nname = \"x\"\n[pkg]\ninstal = [\"apk\"]\n" + if err := os.WriteFile(p, []byte(body), 0o644); err != nil { + t.Fatal(err) + } + _, err := parseFile(p) + if err == nil || !strings.Contains(err.Error(), `unknown key "pkg.instal"`) { + t.Errorf("err = %v", err) + } +} + +func TestParseFileRequiresSchema(t *testing.T) { + p := filepath.Join(t.TempDir(), "x.toml") + if err := os.WriteFile(p, []byte("name = \"x\"\n"), 0o644); err != nil { + t.Fatal(err) + } + _, err := parseFile(p) + if err == nil || !strings.Contains(err.Error(), "missing schema") { + t.Errorf("err = %v", err) + } +} + +// A user file naming a bundled guest merges per field: a scalar it sets +// wins, a table it omits (pkg) keeps the bundled value, and a backend +// table it does set replaces whole rather than merging key by key. +func TestLoadMergesUserOverBundled(t *testing.T) { + dir := t.TempDir() + body := "schema = 1\nname = \"fedora\"\nshell = \"/bin/zsh\"\n[backend.cloudinit]\nskip_9p = true\n" + if err := os.WriteFile(filepath.Join(dir, "fedora.toml"), []byte(body), 0o644); err != nil { + t.Fatal(err) + } + t.Cleanup(func() { loaded = loadBundled() }) + if err := Load(dir); err != nil { + t.Fatal(err) + } + o, ok := Lookup("fedora") + if !ok { + t.Fatal("fedora missing after Load") + } + if o.Shell != "/bin/zsh" || o.Source != "bundled+user" { + t.Errorf("shell = %q source = %q", o.Shell, o.Source) + } + if len(o.Pkg.Install) == 0 || o.Pkg.Install[0] != "dnf" { + t.Errorf("merge lost pkg.install: %v", o.Pkg.Install) + } + if o.Backends["cloudinit"]["skip_9p"] != true { + t.Errorf("backend table not replaced: %v", o.Backends) + } +} + +// A file naming a guest not already loaded is a new guest, not a merge: it +// must carry every required field on its own. +func TestLoadNewGuestNeedsEveryField(t *testing.T) { + dir := t.TempDir() + if err := os.WriteFile(filepath.Join(dir, "freebsd.toml"), []byte("schema = 1\nname = \"freebsd\"\ninit = \"rc\"\n"), 0o644); err != nil { + t.Fatal(err) + } + t.Cleanup(func() { loaded = loadBundled() }) + err := Load(dir) + if err == nil || !strings.Contains(err.Error(), "freebsd: missing shell") { + t.Errorf("err = %v", err) + } + if _, ok := Lookup("freebsd"); ok { + t.Error("a failed Load changed the loaded set") + } +} + +// Load runs at startup with a directory the user may never have created. +func TestLoadMissingDirIsFine(t *testing.T) { + if err := Load(filepath.Join(t.TempDir(), "nope")); err != nil { + t.Fatal(err) + } +} + +// Capabilities replaces recipes/manifest.go's capabilityOSes table: each +// capability, including the init name the loader appends, must map back +// to the guests that declare it. +func TestCapabilitiesIncludeInit(t *testing.T) { + c := Capabilities() + if got := c["openrc"]; len(got) != 1 || got[0] != "alpine" { + t.Errorf("openrc = %v", got) + } + if got := c["apt"]; len(got) != 2 { + t.Errorf("apt = %v", got) + } +} diff --git a/internal/guest/prelude.go b/internal/guest/prelude.go new file mode 100644 index 00000000..743f1950 --- /dev/null +++ b/internal/guest/prelude.go @@ -0,0 +1,135 @@ +package guest + +import ( + "fmt" + "path" + "sort" + "strings" +) + +// ShQuote wraps s in single quotes for POSIX sh, escaping embedded quotes. +func ShQuote(s string) string { + return "'" + strings.ReplaceAll(s, "'", `'\''`) + "'" +} + +func quoteArgv(argv []string) string { + q := make([]string, len(argv)) + for i, a := range argv { + q[i] = ShQuote(a) + } + return strings.Join(q, " ") +} + +// shTemplate renders a [svc] or [cmd] template into a function body: +// {name} becomes "$1"; a template without it gets "$@" appended. +func shTemplate(tmpl string) string { + if strings.Contains(tmpl, "{name}") { + return strings.ReplaceAll(tmpl, "{name}", `"$1"`) + } + return tmpl + ` "$@"` +} + +// pkgMgr is the basename of pkg.install's first word. +func pkgMgr(o OS) string { + if len(o.Pkg.Install) == 0 { + return "" + } + return path.Base(o.Pkg.Install[0]) +} + +// verbs lists every function the prelude defines, in a fixed order, so the +// sh and python renderings stay in step and the golden files are stable. +func verbs(o OS) []struct{ name, tmpl string } { + v := []struct{ name, tmpl string }{ + {"stoat_svc_enable", o.Svc.Enable}, + {"stoat_svc_start", o.Svc.Start}, + {"stoat_svc_stop", o.Svc.Stop}, + {"stoat_svc_restart", o.Svc.Restart}, + {"stoat_svc_status", o.Svc.Status}, + } + names := make([]string, 0, len(o.Cmd)) + for n := range o.Cmd { + names = append(names, n) + } + sort.Strings(names) + for _, n := range names { + v = append(v, struct{ name, tmpl string }{"stoat_" + n, o.Cmd[n]}) + } + return v +} + +// Prelude renders the verb definitions and STOAT_* variables for runtime +// ("sh" or "python3"). sshx and cloudinit put it in front of every recipe +// body, so a recipe behaves the same over ssh and in runcmd. +func Prelude(o OS, runtime string) string { + if runtime == "python3" { + return preludePython(o) + } + var b strings.Builder + setup := o.Pkg.Setup + if strings.TrimSpace(setup) == "" { + setup = ":" + } + fmt.Fprintf(&b, "stoat_pkg_setup() { %s; }\n", setup) + fmt.Fprintf(&b, "stoat_pkg_install() { %s \"$@\"; }\n", quoteArgv(o.Pkg.Install)) + for _, v := range verbs(o) { + fmt.Fprintf(&b, "%s() { %s; }\n", v.name, shTemplate(v.tmpl)) + } + for _, k := range sortedKeys(o.Pkg.Env) { + fmt.Fprintf(&b, "export %s=%s\n", k, ShQuote(o.Pkg.Env[k])) + } + fmt.Fprintf(&b, "STOAT_OS=%s; STOAT_INIT=%s; STOAT_PKGMGR=%s\n", o.Name, o.Init, pkgMgr(o)) + b.WriteString("export STOAT_OS STOAT_INIT STOAT_PKGMGR\n") + return b.String() +} + +func preludePython(o OS) string { + var b strings.Builder + b.WriteString("import os, subprocess\n") + fmt.Fprintf(&b, "os.environ[\"STOAT_OS\"] = %q\nos.environ[\"STOAT_INIT\"] = %q\nos.environ[\"STOAT_PKGMGR\"] = %q\n", o.Name, string(o.Init), pkgMgr(o)) + for _, k := range sortedKeys(o.Pkg.Env) { + fmt.Fprintf(&b, "os.environ[%q] = %q\n", k, o.Pkg.Env[k]) + } + b.WriteString("def _run(*argv): subprocess.run(list(argv), check=True)\n") + setup := o.Pkg.Setup + if strings.TrimSpace(setup) == "" { + setup = ":" + } + fmt.Fprintf(&b, "def stoat_pkg_setup(): _run(\"sh\", \"-c\", %q)\n", setup) + fmt.Fprintf(&b, "def stoat_pkg_install(*pkgs): _run(%s, *pkgs)\n", pyArgv(o.Pkg.Install)) + for _, v := range verbs(o) { + fmt.Fprintf(&b, "def %s(name): _run(\"sh\", \"-c\", '%s', \"stoat\", name)\n", v.name, shTemplate(v.tmpl)) + } + return b.String() +} + +func pyArgv(argv []string) string { + q := make([]string, len(argv)) + for i, a := range argv { + q[i] = fmt.Sprintf("%q", a) + } + return strings.Join(q, ", ") +} + +func sortedKeys(m map[string]string) []string { + keys := make([]string, 0, len(m)) + for k := range m { + keys = append(keys, k) + } + sort.Strings(keys) + return keys +} + +// WithPrelude inserts prelude after a leading "#!" line, or in front of the +// body when there is none. cloud-init's write_files keeps the shebang +// first; a prelude before it would make the file run under /bin/sh with the +// shebang as a comment, which changes nothing on Linux but reads wrong. +func WithPrelude(body, prelude string) string { + if strings.HasPrefix(body, "#!") { + if i := strings.IndexByte(body, '\n'); i >= 0 { + return body[:i+1] + prelude + body[i+1:] + } + return body + "\n" + prelude + } + return prelude + body +} diff --git a/internal/guest/prelude_test.go b/internal/guest/prelude_test.go new file mode 100644 index 00000000..aa97c91c --- /dev/null +++ b/internal/guest/prelude_test.go @@ -0,0 +1,74 @@ +package guest + +import ( + "os" + "os/exec" + "path/filepath" + "strings" + "testing" +) + +// Every bundled guest's rendered sh prelude is a golden file and must parse +// as POSIX sh. UPDATE_GOLDEN=1 writes the golden from the current output; +// run it once when Prelude is implemented, then commit the files. +func TestPreludeGolden(t *testing.T) { + for _, o := range All() { + got := Prelude(o, "sh") + path := filepath.Join("testdata", "prelude", o.Name+".sh") + if os.Getenv("UPDATE_GOLDEN") != "" { + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(path, []byte(got), 0o644); err != nil { + t.Fatal(err) + } + } + want, err := os.ReadFile(path) + if err != nil { + t.Fatalf("%s: %v (run with UPDATE_GOLDEN=1 once)", o.Name, err) + } + if string(want) != got { + t.Errorf("%s prelude drifted from testdata; diff and update on purpose", o.Name) + } + if _, err := exec.LookPath("sh"); err == nil { + cmd := exec.Command("sh", "-n") + cmd.Stdin = strings.NewReader(got) + if out, err := cmd.CombinedOutput(); err != nil { + t.Errorf("%s prelude is not valid sh: %s", o.Name, out) + } + } + } +} + +// The python3 runtime gets a parallel prelude over subprocess.run instead of +// sh functions, per docs/specs/2026-09-04-guest-definitions-design.md's +// "Verbs in recipes" section. No module ships to the guest. +func TestPreludePython(t *testing.T) { + o, ok := Lookup("arch") + if !ok { + t.Fatal("bundled arch missing") + } + got := Prelude(o, "python3") + for _, want := range []string{ + "import os, subprocess", + `os.environ["STOAT_OS"] = "arch"`, + "def stoat_pkg_install(", + "def stoat_svc_enable(", + } { + if !strings.Contains(got, want) { + t.Errorf("missing %q in:\n%s", want, got) + } + } +} + +// WithPrelude inserts after a leading shebang line so the interpreter line +// stays first; a body with no shebang gets the prelude in front. +func TestWithPreludeKeepsShebangFirst(t *testing.T) { + got := WithPrelude("#!/bin/sh\nset -e\necho hi\n", "P\n") + if !strings.HasPrefix(got, "#!/bin/sh\nP\nset -e\n") { + t.Errorf("got:\n%s", got) + } + if got := WithPrelude("echo hi\n", "P\n"); got != "P\necho hi\n" { + t.Errorf("no-shebang case: %q", got) + } +} diff --git a/internal/guest/testdata/prelude/alpine.sh b/internal/guest/testdata/prelude/alpine.sh new file mode 100644 index 00000000..0670c7e1 --- /dev/null +++ b/internal/guest/testdata/prelude/alpine.sh @@ -0,0 +1,9 @@ +stoat_pkg_setup() { apk update; } +stoat_pkg_install() { 'apk' '--wait' '60' 'add' "$@"; } +stoat_svc_enable() { rc-update add "$1" default; } +stoat_svc_start() { rc-service "$1" start; } +stoat_svc_stop() { rc-service "$1" stop; } +stoat_svc_restart() { rc-service "$1" restart; } +stoat_svc_status() { rc-service "$1" status; } +STOAT_OS=alpine; STOAT_INIT=openrc; STOAT_PKGMGR=apk +export STOAT_OS STOAT_INIT STOAT_PKGMGR diff --git a/internal/guest/testdata/prelude/arch.sh b/internal/guest/testdata/prelude/arch.sh new file mode 100644 index 00000000..6f76ba3c --- /dev/null +++ b/internal/guest/testdata/prelude/arch.sh @@ -0,0 +1,9 @@ +stoat_pkg_setup() { pacman -Sy --noconfirm; } +stoat_pkg_install() { 'pacman' '-S' '--noconfirm' "$@"; } +stoat_svc_enable() { systemctl enable "$1"; } +stoat_svc_start() { systemctl start "$1"; } +stoat_svc_stop() { systemctl stop "$1"; } +stoat_svc_restart() { systemctl restart "$1"; } +stoat_svc_status() { systemctl status "$1"; } +STOAT_OS=arch; STOAT_INIT=systemd; STOAT_PKGMGR=pacman +export STOAT_OS STOAT_INIT STOAT_PKGMGR diff --git a/internal/guest/testdata/prelude/debian.sh b/internal/guest/testdata/prelude/debian.sh new file mode 100644 index 00000000..236336d4 --- /dev/null +++ b/internal/guest/testdata/prelude/debian.sh @@ -0,0 +1,10 @@ +stoat_pkg_setup() { apt-get update; } +stoat_pkg_install() { 'apt-get' 'install' '-y' "$@"; } +stoat_svc_enable() { systemctl enable "$1"; } +stoat_svc_start() { systemctl start "$1"; } +stoat_svc_stop() { systemctl stop "$1"; } +stoat_svc_restart() { systemctl restart "$1"; } +stoat_svc_status() { systemctl status "$1"; } +export DEBIAN_FRONTEND='noninteractive' +STOAT_OS=debian; STOAT_INIT=systemd; STOAT_PKGMGR=apt-get +export STOAT_OS STOAT_INIT STOAT_PKGMGR diff --git a/internal/guest/testdata/prelude/fedora.sh b/internal/guest/testdata/prelude/fedora.sh new file mode 100644 index 00000000..71a5b2c0 --- /dev/null +++ b/internal/guest/testdata/prelude/fedora.sh @@ -0,0 +1,9 @@ +stoat_pkg_setup() { :; } +stoat_pkg_install() { 'dnf' 'install' '-y' "$@"; } +stoat_svc_enable() { systemctl enable "$1"; } +stoat_svc_start() { systemctl start "$1"; } +stoat_svc_stop() { systemctl stop "$1"; } +stoat_svc_restart() { systemctl restart "$1"; } +stoat_svc_status() { systemctl status "$1"; } +STOAT_OS=fedora; STOAT_INIT=systemd; STOAT_PKGMGR=dnf +export STOAT_OS STOAT_INIT STOAT_PKGMGR diff --git a/internal/guest/testdata/prelude/ubuntu.sh b/internal/guest/testdata/prelude/ubuntu.sh new file mode 100644 index 00000000..524c4a92 --- /dev/null +++ b/internal/guest/testdata/prelude/ubuntu.sh @@ -0,0 +1,10 @@ +stoat_pkg_setup() { apt-get update; } +stoat_pkg_install() { 'apt-get' 'install' '-y' "$@"; } +stoat_svc_enable() { systemctl enable "$1"; } +stoat_svc_start() { systemctl start "$1"; } +stoat_svc_stop() { systemctl stop "$1"; } +stoat_svc_restart() { systemctl restart "$1"; } +stoat_svc_status() { systemctl status "$1"; } +export DEBIAN_FRONTEND='noninteractive' +STOAT_OS=ubuntu; STOAT_INIT=systemd; STOAT_PKGMGR=apt-get +export STOAT_OS STOAT_INIT STOAT_PKGMGR diff --git a/internal/recipes/bootstrap.go b/internal/recipes/bootstrap.go new file mode 100644 index 00000000..52075269 --- /dev/null +++ b/internal/recipes/bootstrap.go @@ -0,0 +1,46 @@ +package recipes + +import ( + "fmt" + + "github.com/novusedge/stoat/internal/guest" +) + +// interpreterCommand maps a runtime to the guest command that reads a script +// from stdin. "sh -s" and "python3 -" both read a script body piped to +// them, matching how sshx.Provision pipes ScriptBody as the recipe's stdin. +var interpreterCommand = map[string][]string{ + "sh": {"sh", "-s"}, + "python3": {"python3", "-"}, +} + +// InterpreterArgs returns the guest command that runs a recipe body under +// runtime, for use as the ssh command's trailing argv. +func InterpreterArgs(runtime string) []string { + if args, ok := interpreterCommand[runtime]; ok { + return args + } + return []string{runtime} +} + +// BootstrapScript returns a sh snippet that installs runtime on osName if +// missing, or "" when runtime needs no install step ("sh" is always present +// in a POSIX guest) or osName is not a loaded guest. sshx.Provision pipes +// this to `sh -s` over ssh before running a recipe under a non-sh runtime. +// It calls the guest prelude's verbs rather than a package manager by name, +// so the same snippet works on any guest that defines runtime_packages for +// the runtime. +func BootstrapScript(runtime, osName string) string { + if runtime == "sh" { + return "" + } + o, ok := guest.Lookup(osName) + if !ok { + return "" + } + pkg := o.Pkg.RuntimePackages[runtime] + if pkg == "" { + return "" + } + return fmt.Sprintf("set -e\nif ! command -v %s >/dev/null 2>&1; then\nstoat_pkg_setup\nstoat_pkg_install %s\nfi\n", runtime, pkg) +} diff --git a/internal/recipes/bootstrap_test.go b/internal/recipes/bootstrap_test.go new file mode 100644 index 00000000..39ada768 --- /dev/null +++ b/internal/recipes/bootstrap_test.go @@ -0,0 +1,41 @@ +package recipes + +import ( + "strings" + "testing" +) + +func TestBootstrapScriptShIsEmpty(t *testing.T) { + if got := BootstrapScript("sh", "alpine"); got != "" { + t.Errorf("BootstrapScript(sh, alpine) = %q, want empty", got) + } +} + +// The installed package name comes from the guest's runtime_packages table, +// not a name hardcoded per OS: arch calls its python3 package "python". +func TestBootstrapScriptInstallsGuestPackage(t *testing.T) { + cases := []struct { + os, wantInstall string + }{ + {"alpine", "stoat_pkg_install python3"}, + {"arch", "stoat_pkg_install python"}, + } + for _, c := range cases { + got := BootstrapScript("python3", c.os) + if !strings.Contains(got, "command -v python3") { + t.Errorf("%s: missing the command -v check, got:\n%s", c.os, got) + } + if !strings.Contains(got, "stoat_pkg_setup") { + t.Errorf("%s: missing stoat_pkg_setup, got:\n%s", c.os, got) + } + if !strings.Contains(got, c.wantInstall) { + t.Errorf("%s: missing %q, got:\n%s", c.os, c.wantInstall, got) + } + } +} + +func TestBootstrapScriptUnknownOSIsEmpty(t *testing.T) { + if got := BootstrapScript("python3", "plan9"); got != "" { + t.Errorf("BootstrapScript(python3, plan9) = %q, want empty for an unknown guest", got) + } +} diff --git a/internal/recipes/bundled/xfce/install-alpine.sh b/internal/recipes/bundled/xfce/install-alpine.sh deleted file mode 100644 index 00b3d614..00000000 --- a/internal/recipes/bundled/xfce/install-alpine.sh +++ /dev/null @@ -1,110 +0,0 @@ -#!/bin/sh -# Installs XFCE and starts it. Runs as root over ssh, on a booted Alpine live VM. -set -e - -# setup-apkrepos -c -1 refreshes the indexes, so no separate `apk update` runs -# here. It runs apk update with no lock-wait, so another apk that holds the -# database lock fails it with exit 99. Retry until the lock frees, up to ~60s. -n=0 -until setup-apkrepos -c -1; do - n=$((n + 1)) - [ "$n" -ge 30 ] && { echo "apk database stayed locked; giving up" >&2; exit 1; } - sleep 2 -done - -setup-xorg-base -# --wait 60 makes apk wait up to 60s for the lock instead of failing with -# exit 99 when another apk run holds it. -apk --wait 60 add xfce4 xfce4-terminal dbus-x11 - -# Xorg reads its input devices from udev. Alpine boots mdev by default, which -# never feeds Xorg, so libinput finds no mouse or keyboard and the desktop -# ignores every click. setup-devd switches the device manager to eudev and -# adds udev, udev-trigger, and udev-settle to sysinit for the next boot. -# Ubuntu, Debian, and Arch already run udev under systemd, so only Alpine -# needs this. -setup-devd udev -# Cold-plug the devices that already exist so Xorg finds the tablet in this -# boot, not only after a reboot. The live case below starts X now with a HUP -# to init, before udev-trigger would run at the next sysinit. -udevadm trigger 2>/dev/null || true -udevadm settle 2>/dev/null || true - -rc-update add dbus -rc-service dbus start || true - -mkdir -p /mnt/host -mount -a || true - -echo 'exec startxfce4' > /root/.xinitrc - -# Autologin root on tty1 and start X from the profile: a display manager is -# more machinery than a disposable single-user VM needs. -# ponytail: no DM. If you ever want multi-user or a login screen, that is when to add one. -# -# busybox getty (this is Alpine, not util-linux) has NO -a/autologin flag, -# confirmed against the actual busybox 1.37.0-r31 shipped on the Alpine 3.24.1 -# ISO, whose `getty --help` lists only -h -L -m -n -w -i -f -l -t -I -H. The -# busybox-native trick is `getty -n -l LOGIN`: -n skips the username prompt -# and LOGIN is invoked directly in place of /bin/login. LOGIN then runs -# `login -f root` (busybox login's "-f: don't authenticate, user already -# authenticated" flag) so a real login session starts: MOTD, utmp entry, -# and a proper login shell that sources /root/.profile. -cat > /sbin/autologin <<'EOF' -#!/bin/sh -exec /bin/login -f root -EOF -chmod +x /sbin/autologin - -sed -i 's|^tty1::respawn:.*|tty1::respawn:/sbin/getty -n -l /sbin/autologin 38400 tty1|' /etc/inittab -grep -q '^tty1::respawn:/sbin/getty -n -l /sbin/autologin' /etc/inittab || { - echo "failed to configure tty1 autologin in /etc/inittab" >&2 - exit 1 -} - -# Guarded so re-running the recipe doesn't stack multiple "exec startx" -# blocks in .profile. exec matters: without it, quitting X drops to a shell -# that immediately re-runs the profile and restarts X. The DISPLAY check -# stops a recursive startx if .profile is sourced from inside the X session. -if ! grep -q 'exec startx' /root/.profile 2>/dev/null; then - cat >> /root/.profile <<'EOF' -if [ "$(tty)" = "/dev/tty1" ] && [ -z "$DISPLAY" ]; then - exec startx -fi -EOF -fi - -# Live vs disk: a live VM boots Alpine's diskless mode, whose initramfs -# mounts the root filesystem as tmpfs (and, on the -o overlaytmpfs boot -# option, overlay-on-tmpfs), confirmed against -# /usr/share/mkinitfs/initramfs-init from mkinitfs-3.14.0-r0 (the package -# apk installs from the Alpine 3.24 repos): the diskless path reads -# `mount -t tmpfs -o "$rootflags" tmpfs "$sysroot"` and the overlaytmpfs -# path reads `mount -t overlay -o ... overlayfs "$sysroot"`. Everything -# written there (/etc/inittab, /root/.profile, the installed packages) -# lives in RAM and is gone on reboot; only the apkovl (which cannot carry -# installed packages) survives. A disk install (setup-disk) instead mounts -# a real block-device filesystem (ext4 by default) as root, which persists. -# So: root mounted tmpfs/overlay == live/diskless, anything else == disk. -root_fstype=$(awk '$2 == "/" { print $3 }' /proc/mounts) - -case "$root_fstype" in -tmpfs | overlay) - # Live VM: the autologin/.profile edits above are still worth having in - # case tty1 gets re-inited some other way, but they will NOT survive a - # reboot, so telling the user to reboot for a desktop would be a lie. - # Instead, get XFCE onto the VM's own console *right now*, in this - # boot: busybox init (this is Alpine, not sysvinit) documents `HUP: - # reload /etc/inittab` in its own --help text, and reloading inittab is - # the standard way to make init respawn a tty entry whose command - # changed without a reboot. Sending it makes init respawn tty1 through - # /sbin/autologin -> login -f root -> .profile -> exec startx, so XFCE - # comes up on the VM's console (the one qemu displays) without the user - # touching the console at all. - kill -HUP 1 - echo "xfce installed and starting NOW on the vm's console (this session only). A live VM keeps nothing across reboots (root is tmpfs/overlay, wiped on restart), so rebooting will NOT bring xfce back. For a desktop that survives reboots, provision a disk VM instead." - ;; -*) - echo "xfce installed. Reboot the vm to land in xfce automatically (tty1 now autologins root and starts X on next boot)" - ;; -esac diff --git a/internal/recipes/bundled/xfce/install-arch.sh b/internal/recipes/bundled/xfce/install-arch.sh deleted file mode 100644 index ca25f491..00000000 --- a/internal/recipes/bundled/xfce/install-arch.sh +++ /dev/null @@ -1,61 +0,0 @@ -#!/bin/sh -# Installs XFCE and starts it. Runs as root over ssh on a booted Arch VM. -set -e - -pacman -Syu --noconfirm xfce4 xfce4-terminal dbus xorg-xinit - -systemctl enable dbus -systemctl start dbus || true - -mkdir -p /mnt/host -mount -a || true - -echo 'exec startxfce4' > /root/.xinitrc - -# Autologin root on tty1 and start X from the profile: a display manager is -# more machinery than a disposable single-user VM needs. -# ponytail: no DM. If you ever want multi-user or a login screen, that is when to add one. -# -# Arch's tty1 is managed by systemd's getty@.service, so autologin is a -# drop-in override rather than an /etc/inittab edit (that's the alpine/ -# busybox mechanism, not systemd's). -mkdir -p /etc/systemd/system/getty@tty1.service.d -cat > /etc/systemd/system/getty@tty1.service.d/autologin.conf <<'EOF' -[Service] -ExecStart= -ExecStart=-/sbin/agetty --autologin root --noclear %I $TERM -EOF -systemctl daemon-reload -systemctl enable getty@tty1.service - -# Guarded so re-running the recipe doesn't stack multiple "exec startx" -# blocks in .profile. exec matters: without it, quitting X drops to a shell -# that immediately re-runs the profile and restarts X. The DISPLAY check -# stops a recursive startx if .profile is sourced from inside the X session. -if ! grep -q 'exec startx' /root/.profile 2>/dev/null; then - cat >> /root/.profile <<'EOF' -if [ "$(tty)" = "/dev/tty1" ] && [ -z "$DISPLAY" ]; then - exec startx -fi -EOF -fi - -# Live vs disk: an ssh-reachable Arch VM is not always a normal disk -# install: a live/installer session also mounts its root as tmpfs/overlay, -# and everything written above (the getty override, /root/.profile, the -# installed packages) lives only in that RAM-backed root and is gone on -# reboot. Same detection as the alpine recipe: inspect the mounted root -# filesystem type via /proc/mounts. -root_fstype=$(awk '$2 == "/" { print $3 }' /proc/mounts) - -case "$root_fstype" in -tmpfs | overlay) - # Live root: apply the getty change to the running session right now - # instead of promising a reboot that would wipe it. - systemctl restart getty@tty1.service || true - echo "xfce installed and starting NOW on the vm's console (this session only). A live root keeps nothing across reboots (root is tmpfs/overlay, wiped on restart), so rebooting will NOT bring xfce back. For a desktop that survives reboots, provision a disk-installed VM instead." - ;; -*) - echo "xfce installed. Reboot the vm to land in xfce automatically (tty1 now autologins root and starts X on next boot)" - ;; -esac diff --git a/internal/recipes/bundled/xfce/install-debian.sh b/internal/recipes/bundled/xfce/install-debian.sh deleted file mode 100644 index 169c2909..00000000 --- a/internal/recipes/bundled/xfce/install-debian.sh +++ /dev/null @@ -1,63 +0,0 @@ -#!/bin/sh -# Installs XFCE and starts it. Runs as root over ssh on a booted Debian VM. -set -e - -apt-get update -apt-get install -y xfce4 xfce4-terminal dbus-x11 xinit xserver-xorg - -systemctl enable dbus -systemctl start dbus || true - -mkdir -p /mnt/host -mount -a || true - -echo 'exec startxfce4' > /root/.xinitrc - -# Autologin root on tty1 and start X from the profile: a display manager is -# more machinery than a disposable single-user VM needs. -# ponytail: no DM. If you ever want multi-user or a login screen, that is when to add one. -# -# Debian's tty1 is managed by systemd's getty@.service, so autologin is a -# drop-in override rather than an /etc/inittab edit (that's the alpine/ -# busybox mechanism, not systemd's). agetty's --autologin does the same job -# busybox getty's `-n -l LOGIN` trick does on Alpine. -mkdir -p /etc/systemd/system/getty@tty1.service.d -cat > /etc/systemd/system/getty@tty1.service.d/autologin.conf <<'EOF' -[Service] -ExecStart= -ExecStart=-/sbin/agetty --autologin root --noclear %I $TERM -EOF -systemctl daemon-reload -systemctl enable getty@tty1.service - -# Guarded so re-running the recipe doesn't stack multiple "exec startx" -# blocks in .profile. exec matters: without it, quitting X drops to a shell -# that immediately re-runs the profile and restarts X. The DISPLAY check -# stops a recursive startx if .profile is sourced from inside the X session. -if ! grep -q 'exec startx' /root/.profile 2>/dev/null; then - cat >> /root/.profile <<'EOF' -if [ "$(tty)" = "/dev/tty1" ] && [ -z "$DISPLAY" ]; then - exec startx -fi -EOF -fi - -# Live vs disk: an ssh-reachable Debian VM is not always a normal disk -# install: a live/installer session also mounts its root as tmpfs/overlay, -# and everything written above (the getty override, /root/.profile, the -# installed packages) lives only in that RAM-backed root and is gone on -# reboot. Same detection as the alpine recipe: inspect the mounted root -# filesystem type via /proc/mounts. -root_fstype=$(awk '$2 == "/" { print $3 }' /proc/mounts) - -case "$root_fstype" in -tmpfs | overlay) - # Live root: apply the getty change to the running session right now - # instead of promising a reboot that would wipe it. - systemctl restart getty@tty1.service || true - echo "xfce installed and starting NOW on the vm's console (this session only). A live root keeps nothing across reboots (root is tmpfs/overlay, wiped on restart), so rebooting will NOT bring xfce back. For a desktop that survives reboots, provision a disk-installed VM instead." - ;; -*) - echo "xfce installed. Reboot the vm to land in xfce automatically (tty1 now autologins root and starts X on next boot)" - ;; -esac diff --git a/internal/recipes/bundled/xfce/install.sh b/internal/recipes/bundled/xfce/install.sh index bc26dfa9..db3265c6 100644 --- a/internal/recipes/bundled/xfce/install.sh +++ b/internal/recipes/bundled/xfce/install.sh @@ -1,12 +1,31 @@ #!/bin/sh -# Installs XFCE and starts it. Runs as root over ssh on a booted Ubuntu VM. +# Installs XFCE and starts it. Runs as root over ssh on a booted VM. set -e -apt-get update -apt-get install -y xfce4 xfce4-terminal dbus-x11 xinit xserver-xorg +stoat_pkg_setup -systemctl enable dbus -systemctl start dbus || true +case "$STOAT_PKGMGR" in +apk) + # Xorg reads its input devices from udev. Alpine boots mdev by default, + # which never feeds Xorg, so libinput finds no mouse or keyboard and the + # desktop ignores every click. setup-devd switches to eudev and adds it + # to sysinit for the next boot; udevadm trigger/settle cold-plugs the + # devices that already exist so this boot sees them too. + setup-devd udev + udevadm trigger 2>/dev/null || true + udevadm settle 2>/dev/null || true + stoat_pkg_install xfce4 xfce4-terminal dbus-x11 + ;; +apt-get) + stoat_pkg_install xfce4 xfce4-terminal dbus-x11 xinit xserver-xorg + ;; +pacman) + stoat_pkg_install xfce4 xfce4-terminal dbus xorg-xinit + ;; +esac + +stoat_svc_enable dbus +stoat_svc_start dbus || true mkdir -p /mnt/host mount -a || true @@ -16,19 +35,37 @@ echo 'exec startxfce4' > /root/.xinitrc # Autologin root on tty1 and start X from the profile: a display manager is # more machinery than a disposable single-user VM needs. # ponytail: no DM. If you ever want multi-user or a login screen, that is when to add one. -# -# Ubuntu's tty1 is managed by systemd's getty@.service, so autologin is a -# drop-in override rather than an /etc/inittab edit (that's the alpine/ -# busybox mechanism, not systemd's). agetty's --autologin does the same job -# busybox getty's `-n -l LOGIN` trick does on Alpine. -mkdir -p /etc/systemd/system/getty@tty1.service.d -cat > /etc/systemd/system/getty@tty1.service.d/autologin.conf <<'EOF' +case "$STOAT_INIT" in +openrc) + # busybox getty (Alpine, not util-linux) has no -a/autologin flag: the + # busybox-native trick is `getty -n -l LOGIN`, invoking LOGIN directly in + # place of /bin/login. LOGIN runs `login -f root` (busybox login's + # "-f: don't authenticate, user already authenticated" flag), so a real + # login session starts and sources /root/.profile. + cat > /sbin/autologin <<'EOF' +#!/bin/sh +exec /bin/login -f root +EOF + chmod +x /sbin/autologin + sed -i 's|^tty1::respawn:.*|tty1::respawn:/sbin/getty -n -l /sbin/autologin 38400 tty1|' /etc/inittab + grep -q '^tty1::respawn:/sbin/getty -n -l /sbin/autologin' /etc/inittab || { + echo "failed to configure tty1 autologin in /etc/inittab" >&2 + exit 1 + } + ;; +systemd) + # tty1 is managed by systemd's getty@.service, so autologin is a + # drop-in override rather than an inittab edit. + mkdir -p /etc/systemd/system/getty@tty1.service.d + cat > /etc/systemd/system/getty@tty1.service.d/autologin.conf <<'EOF' [Service] ExecStart= ExecStart=-/sbin/agetty --autologin root --noclear %I $TERM EOF -systemctl daemon-reload -systemctl enable getty@tty1.service + systemctl daemon-reload + systemctl enable getty@tty1.service + ;; +esac # Guarded so re-running the recipe doesn't stack multiple "exec startx" # blocks in .profile. exec matters: without it, quitting X drops to a shell @@ -42,19 +79,26 @@ fi EOF fi -# Live vs disk: an ssh-reachable Ubuntu VM is not always a normal disk -# install: a live/installer session also mounts its root as tmpfs/overlay, -# and everything written above (the getty override, /root/.profile, the -# installed packages) lives only in that RAM-backed root and is gone on -# reboot. Same detection as the alpine recipe: inspect the mounted root -# filesystem type via /proc/mounts. +# Live vs disk: an ssh-reachable VM is not always a normal disk install: a +# live/installer session also mounts its root as tmpfs/overlay, and +# everything written above (the getty config, /root/.profile, the installed +# packages) lives only in that RAM-backed root and is gone on reboot. root_fstype=$(awk '$2 == "/" { print $3 }' /proc/mounts) case "$root_fstype" in tmpfs | overlay) - # Live root: apply the getty change to the running session right now - # instead of promising a reboot that would wipe it. - systemctl restart getty@tty1.service || true + case "$STOAT_INIT" in + openrc) + # busybox init documents `HUP: reload /etc/inittab` in its own + # --help text; reloading inittab makes it respawn tty1 through the + # new command without a reboot, so XFCE comes up on the VM's + # console this boot. + kill -HUP 1 + ;; + systemd) + systemctl restart getty@tty1.service || true + ;; + esac echo "xfce installed and starting NOW on the vm's console (this session only). A live root keeps nothing across reboots (root is tmpfs/overlay, wiped on restart), so rebooting will NOT bring xfce back. For a desktop that survives reboots, provision a disk-installed VM instead." ;; *) diff --git a/internal/recipes/bundled/xfce/recipe.toml b/internal/recipes/bundled/xfce/recipe.toml index 8c96412c..4637d267 100644 --- a/internal/recipes/bundled/xfce/recipe.toml +++ b/internal/recipes/bundled/xfce/recipe.toml @@ -4,8 +4,3 @@ os = ["alpine", "ubuntu", "debian", "arch"] stage = "provision" script = "install.sh" reboot = true - -[scripts] -alpine = "install-alpine.sh" -debian = "install-debian.sh" -arch = "install-arch.sh" diff --git a/internal/recipes/manifest.go b/internal/recipes/manifest.go index 080e20bf..4facaeef 100644 --- a/internal/recipes/manifest.go +++ b/internal/recipes/manifest.go @@ -6,7 +6,8 @@ import ( "path/filepath" "strings" - "github.com/BurntSushi/toml" + "github.com/novusedge/stoat/internal/guest" + "github.com/novusedge/stoat/internal/tomlx" ) // Manifest is a recipe.toml, the v2 recipe format @@ -41,8 +42,8 @@ var validRuntimes = map[string]bool{"sh": true, "python3": true} // case, docs/recipe-spec-v2.md's Stages section), Run defaults to "once". func ParseManifest(path string) (Manifest, error) { var m Manifest - if _, err := toml.DecodeFile(path, &m); err != nil { - return Manifest{}, fmt.Errorf("parse %s: %w", path, err) + if err := tomlx.Decode(path, &m, tomlx.Reject); err != nil { + return Manifest{}, err } m.dir = filepath.Dir(path) @@ -96,11 +97,17 @@ func ManifestFor(name string) (m Manifest, ok bool, err error) { } // ScriptFor returns the absolute path to the script osName runs: the -// Scripts override for osName if one is declared, otherwise the manifest's -// default Script. +// Scripts override for osName, then for each of the guest's aliases in +// order, then the manifest's default Script. func (m Manifest) ScriptFor(osName string) string { - if s, ok := m.Scripts[osName]; ok { - return filepath.Join(m.dir, s) + keys := []string{osName} + if o, ok := guest.Lookup(osName); ok { + keys = append(keys, o.Aliases...) + } + for _, k := range keys { + if s, ok := m.Scripts[k]; ok { + return filepath.Join(m.dir, s) + } } return filepath.Join(m.dir, m.Script) } @@ -114,22 +121,11 @@ func (m Manifest) ScriptContent(osName string) (string, error) { return string(b), nil } -// capabilityOSes maps each recipe.toml "requires" capability to the guest -// OSes that have it (docs/recipe-spec-v2.md's Capabilities table). An -// unrecognised capability satisfies no OS, so MatchesVM rejects it rather -// than silently letting the recipe through. -var capabilityOSes = map[string][]string{ - "systemd": {"ubuntu", "debian", "arch", "fedora"}, - "openrc": {"alpine"}, - "apt": {"ubuntu", "debian"}, - "apk": {"alpine"}, - "dnf": {"fedora"}, - "pacman": {"arch"}, -} - -// hasCapability reports whether cap resolves against vmOS. +// hasCapability reports whether cap resolves against vmOS. The table comes +// from the loaded guests, so a new guest file adds capabilities without a +// Go edit. func hasCapability(cap, vmOS string) bool { - for _, o := range capabilityOSes[cap] { + for _, o := range guest.Capabilities()[cap] { if o == vmOS { return true } diff --git a/internal/recipes/manifest_test.go b/internal/recipes/manifest_test.go index c564bffb..68281510 100644 --- a/internal/recipes/manifest_test.go +++ b/internal/recipes/manifest_test.go @@ -8,6 +8,17 @@ import ( "testing" ) +func TestParseManifestRejectsUnknownKey(t *testing.T) { + p := filepath.Join(t.TempDir(), "recipe.toml") + if err := os.WriteFile(p, []byte("name = \"x\"\nscript = \"i.sh\"\nrunn = \"once\"\n"), 0o644); err != nil { + t.Fatal(err) + } + _, err := ParseManifest(p) + if err == nil || !strings.Contains(err.Error(), `unknown key "runn"`) { + t.Errorf("err = %v", err) + } +} + // writeManifestFile writes contents to /recipe.toml and returns its path. func writeManifestFile(t *testing.T, dir, contents string) string { t.Helper() @@ -254,6 +265,29 @@ alpine = "install-alpine.sh" } } +// ubuntu's guest.toml declares the alias "debian-family". ScriptFor must +// try it before falling back to the manifest's default script. +func TestManifestScriptForTriesGuestAlias(t *testing.T) { + dir := t.TempDir() + path := writeManifestFile(t, dir, ` +name = "install" +script = "install.sh" + +[scripts] +debian-family = "install-deb.sh" +`) + m, err := ParseManifest(path) + if err != nil { + t.Fatal(err) + } + if want := filepath.Join(dir, "install-deb.sh"); m.ScriptFor("ubuntu") != want { + t.Errorf("ScriptFor(ubuntu) = %q, want %q via the debian-family alias", m.ScriptFor("ubuntu"), want) + } + if want := filepath.Join(dir, "install.sh"); m.ScriptFor("fedora") != want { + t.Errorf("ScriptFor(fedora) = %q, want the default, fedora has no matching alias", m.ScriptFor("fedora")) + } +} + func TestManifestScriptContent(t *testing.T) { dir := t.TempDir() path := writeManifestFile(t, dir, ` @@ -356,6 +390,29 @@ func TestMatchesVMOSAndCapabilityCombined(t *testing.T) { } } +// hasCapability now resolves against guest.Capabilities() instead of a +// hardcoded map. Pin the per-package-manager capabilities the old map held, +// since MatchesVM's tests above only exercise systemd/openrc/apt. +func TestMatchesVMPackageManagerCapabilities(t *testing.T) { + cases := []struct { + cap, vmOS string + want bool + }{ + {"apk", "alpine", true}, + {"apk", "ubuntu", false}, + {"dnf", "fedora", true}, + {"dnf", "arch", false}, + {"pacman", "arch", true}, + {"pacman", "fedora", false}, + } + for _, c := range cases { + m := &Manifest{Name: "svc", Requires: []string{c.cap}} + if got := MatchesVM(m, c.vmOS); got != c.want { + t.Errorf("MatchesVM with requires=%q on %s = %v, want %v", c.cap, c.vmOS, got, c.want) + } + } +} + func TestManifestScriptContentMissingFile(t *testing.T) { dir := t.TempDir() path := writeManifestFile(t, dir, ` diff --git a/internal/recipes/runtime.go b/internal/recipes/runtime.go deleted file mode 100644 index 56990cf4..00000000 --- a/internal/recipes/runtime.go +++ /dev/null @@ -1,71 +0,0 @@ -package recipes - -import "fmt" - -// installCommand maps a guest OS to the shell command that installs a -// package by name, one word per argument the way apk/apt/pacman/dnf all -// accept. BootstrapScript appends the package name itself. -var installCommand = map[string][]string{ - "alpine": {"apk", "--wait", "60", "add"}, - "ubuntu": {"apt-get", "install", "-y"}, - "debian": {"apt-get", "install", "-y"}, - "arch": {"pacman", "-S", "--noconfirm"}, - "fedora": {"dnf", "install", "-y"}, -} - -// runtimePackage maps a non-sh runtime to the package name that provides it -// per guest OS. Arch names its Python 2 successor package "python", not -// "python3"; every other OS here uses "python3". -var runtimePackage = map[string]map[string]string{ - "python3": { - "alpine": "python3", - "ubuntu": "python3", - "debian": "python3", - "fedora": "python3", - "arch": "python", - }, -} - -// interpreterCommand maps a runtime to the guest command that reads a script -// from stdin. "sh -s" and "python3 -" both read a script body piped to -// them, matching how sshx.Provision pipes ScriptBody as the recipe's stdin. -var interpreterCommand = map[string][]string{ - "sh": {"sh", "-s"}, - "python3": {"python3", "-"}, -} - -// InterpreterArgs returns the guest command that runs a recipe body under -// runtime, for use as the ssh command's trailing argv. -func InterpreterArgs(runtime string) []string { - if args, ok := interpreterCommand[runtime]; ok { - return args - } - return []string{runtime} -} - -// BootstrapScript returns a sh snippet that installs runtime on osName if -// missing, or "" when runtime needs no install step ("sh" is always present -// in a POSIX guest). sshx.Provision pipes this to `sh -s` over ssh before -// running a recipe under a non-sh runtime. -// -// The check-then-install shape, and apk's --wait 60, mirror the bundled -// recipes' own idempotent-install pattern (internal/recipes/bundled), so a -// recipe re-applied on a VM that already has the runtime does nothing. -func BootstrapScript(runtime, osName string) string { - if runtime == "sh" { - return "" - } - pkg := runtimePackage[runtime][osName] - if pkg == "" { - return "" - } - install := installCommand[osName] - if install == nil { - return "" - } - cmd := "" - for _, w := range install { - cmd += w + " " - } - return fmt.Sprintf("set -e\nif ! command -v %s >/dev/null 2>&1; then\n%s%s\nfi\n", runtime, cmd, pkg) -} diff --git a/internal/recipes/runtime_test.go b/internal/recipes/runtime_test.go deleted file mode 100644 index c9f92fba..00000000 --- a/internal/recipes/runtime_test.go +++ /dev/null @@ -1,42 +0,0 @@ -package recipes - -import ( - "strings" - "testing" -) - -func TestBootstrapScriptShIsEmpty(t *testing.T) { - if got := BootstrapScript("sh", "alpine"); got != "" { - t.Errorf("BootstrapScript(sh, alpine) = %q, want empty", got) - } -} - -func TestBootstrapScriptPython3Alpine(t *testing.T) { - got := BootstrapScript("python3", "alpine") - for _, want := range []string{"command -v python3", "apk --wait 60 add python3"} { - if !strings.Contains(got, want) { - t.Errorf("BootstrapScript(python3, alpine) missing %q, got:\n%s", want, got) - } - } -} - -func TestBootstrapScriptPython3Arch(t *testing.T) { - got := BootstrapScript("python3", "arch") - for _, want := range []string{"command -v python3", "pacman -S --noconfirm python"} { - if !strings.Contains(got, want) { - t.Errorf("BootstrapScript(python3, arch) missing %q, got:\n%s", want, got) - } - } - if strings.Contains(got, "add python3") { - t.Errorf("BootstrapScript(python3, arch) installed python3, want the arch package name python") - } -} - -func TestInterpreterArgs(t *testing.T) { - if got := InterpreterArgs("sh"); len(got) != 2 || got[0] != "sh" || got[1] != "-s" { - t.Errorf("InterpreterArgs(sh) = %v, want [sh -s]", got) - } - if got := InterpreterArgs("python3"); len(got) != 2 || got[0] != "python3" || got[1] != "-" { - t.Errorf("InterpreterArgs(python3) = %v, want [python3 -]", got) - } -} diff --git a/internal/recipes/scaffold.go b/internal/recipes/scaffold.go index 74b05e28..f8a21938 100644 --- a/internal/recipes/scaffold.go +++ b/internal/recipes/scaffold.go @@ -58,7 +58,7 @@ esac // osSetup is the package-manager preamble per OS, or "" where none is needed. func osSetup(osName string) (setup, install string) { if os, ok := guest.Lookup(osName); ok { - return os.PkgSetup, os.PkgInstall + return os.Pkg.ScaffoldSetup, os.Pkg.ScaffoldInstall } return "", "# install: " } diff --git a/internal/sshx/sharemount.go b/internal/sshx/sharemount.go index 6c783048..fe5bb438 100644 --- a/internal/sshx/sharemount.go +++ b/internal/sshx/sharemount.go @@ -106,7 +106,7 @@ func mountShares(ctx context.Context, v *config.VM, log io.Writer) { return } fmt.Fprintln(log, "\n=== mounting 9p shares ===") - cmd := exec.CommandContext(ctx, "ssh", Args(v, sudoWrap(v, []string{"sh", "-s"})...)...) + cmd := exec.CommandContext(ctx, "ssh", Args(v, escalate(v, []string{"sh", "-s"})...)...) cmd.Cancel = func() error { return cmd.Process.Signal(syscall.SIGTERM) } cmd.WaitDelay = recipeShutdownGrace cmd.Stdin = strings.NewReader(shareMountScript(tags, guestFstabPath)) diff --git a/internal/sshx/sshx.go b/internal/sshx/sshx.go index d890cf9c..90e74630 100644 --- a/internal/sshx/sshx.go +++ b/internal/sshx/sshx.go @@ -13,6 +13,7 @@ import ( "time" "github.com/novusedge/stoat/internal/config" + "github.com/novusedge/stoat/internal/guest" "github.com/novusedge/stoat/internal/keys" "github.com/novusedge/stoat/internal/logx" "github.com/novusedge/stoat/internal/recipes" @@ -72,16 +73,30 @@ func Args(v *config.VM, extra ...string) []string { return append(a, extra...) } -// sudoWrap prefixes remote with "sudo" for a non-root ssh user. Recipe bodies -// install packages, so they need root. Cloud images log in as an unprivileged -// account; cloudinit grants it passwordless sudo ("ALL=(ALL) NOPASSWD:ALL"). -// An apkovl or BYO-root image connects as root and gets no prefix. sudo reads -// the piped script body on stdin the same way the bare interpreter does. -func sudoWrap(v *config.VM, remote []string) []string { +// escalate prefixes remote with the guest's escalate argv for a non-root ssh +// user. Cloud images log in as an unprivileged account with passwordless +// sudo from the seed; apkovl and BYO-root images connect as root and get no +// prefix. An unknown guest falls back to sudo, the only answer that has ever +// been right for a cloud image. +func escalate(v *config.VM, remote []string) []string { if User(v) == "root" { return remote } - return append([]string{"sudo"}, remote...) + prefix := []string{"sudo", "-n"} + if o, ok := guest.Lookup(v.OS); ok && len(o.Escalate) > 0 { + prefix = o.Escalate + } + return append(append([]string{}, prefix...), remote...) +} + +// preludeFor renders the guest prelude for v, or "" for a guest stoat does +// not know, in which case a recipe runs bare as it always did. +func preludeFor(v *config.VM, runtime string) string { + o, ok := guest.Lookup(v.OS) + if !ok { + return "" + } + return guest.Prelude(o, runtime) } // CopyArgs returns the argv (excluding argv[0]) for scp between the host and @@ -252,7 +267,7 @@ func Provision(ctx context.Context, v *config.VM) (err error) { // images ship no cloud-init and skip this. if v.Backend == "cloudinit" { fmt.Fprintln(log, "waiting for cloud-init to finish…") - ci := exec.CommandContext(ctx, "ssh", Args(v, sudoWrap(v, []string{"cloud-init", "status", "--wait"})...)...) + ci := exec.CommandContext(ctx, "ssh", Args(v, escalate(v, []string{"cloud-init", "status", "--wait"})...)...) ci.Cancel = func() error { return ci.Process.Signal(syscall.SIGTERM) } ci.WaitDelay = recipeShutdownGrace ci.Stdout = io.Discard @@ -262,6 +277,24 @@ func Provision(ctx context.Context, v *config.VM) (err error) { mountShares(ctx, v, log) + if o, ok := guest.Lookup(v.OS); ok && strings.TrimSpace(o.Pkg.Setup) != "" { + fmt.Fprintln(log, "refreshing the package index...") + st := exec.CommandContext(ctx, "ssh", Args(v, escalate(v, []string{"sh", "-s"})...)...) + st.Cancel = func() error { return st.Process.Signal(syscall.SIGTERM) } + st.WaitDelay = recipeShutdownGrace + st.Stdin = strings.NewReader(guest.Prelude(o, "sh") + "stoat_pkg_setup\n") + st.Stdout = log + st.Stderr = log + if err := st.Run(); err != nil { + if ctxErr := ctx.Err(); ctxErr != nil { + fmt.Fprintf(log, "CANCELLED: %v\n", ctxErr) + return ctxErr + } + fmt.Fprintf(log, "FAILED: package index refresh: %v\n", err) + return fmt.Errorf("package index refresh: %w", err) + } + } + for _, name := range v.Recipes { // Checked before each recipe, not left to cmd.Run below alone: a ctx // cancelled between recipes must stop here rather than start one more @@ -285,10 +318,10 @@ func Provision(ctx context.Context, v *config.VM) (err error) { if bootstrap := recipes.BootstrapScript(runtime, v.OS); bootstrap != "" { fmt.Fprintf(log, "ensuring %s is installed...\n", runtime) - bs := exec.CommandContext(ctx, "ssh", Args(v, sudoWrap(v, []string{"sh", "-s"})...)...) + bs := exec.CommandContext(ctx, "ssh", Args(v, escalate(v, []string{"sh", "-s"})...)...) bs.Cancel = func() error { return bs.Process.Signal(syscall.SIGTERM) } bs.WaitDelay = recipeShutdownGrace - bs.Stdin = strings.NewReader(bootstrap) + bs.Stdin = strings.NewReader(guest.WithPrelude(bootstrap, preludeFor(v, "sh"))) bs.Stdout = log bs.Stderr = log if err := bs.Run(); err != nil { @@ -301,10 +334,10 @@ func Provision(ctx context.Context, v *config.VM) (err error) { } } - cmd := exec.CommandContext(ctx, "ssh", Args(v, sudoWrap(v, recipes.InterpreterArgs(runtime))...)...) + cmd := exec.CommandContext(ctx, "ssh", Args(v, escalate(v, recipes.InterpreterArgs(runtime))...)...) cmd.Cancel = func() error { return cmd.Process.Signal(syscall.SIGTERM) } cmd.WaitDelay = recipeShutdownGrace - cmd.Stdin = strings.NewReader(body) + cmd.Stdin = strings.NewReader(guest.WithPrelude(body, preludeFor(v, runtime))) cmd.Stdout = log cmd.Stderr = log if err := cmd.Run(); err != nil { diff --git a/internal/sshx/sshx_test.go b/internal/sshx/sshx_test.go index 390317af..77dae4e5 100644 --- a/internal/sshx/sshx_test.go +++ b/internal/sshx/sshx_test.go @@ -545,3 +545,26 @@ func TestProvisionCancelKillsTheSSHProcess(t *testing.T) { t.Errorf("provision log reports success for a cancelled run:\n%s", log) } } + +// escalate must use the guest's own escalate argv, not a hardcoded "sudo": +// a BYO image with a different sudo path, or a doas-only image, needs its +// own prefix. Root gets no prefix regardless of the guest. +func TestEscalateUsesGuestArgv(t *testing.T) { + v := &config.VM{OS: "ubuntu", SSHUser: "stoat"} + got := escalate(v, []string{"sh", "-s"}) + want := []string{"sudo", "-n", "sh", "-s"} + if strings.Join(got, " ") != strings.Join(want, " ") { + t.Errorf("got %v want %v", got, want) + } + if got := escalate(&config.VM{OS: "alpine"}, []string{"sh", "-s"}); len(got) != 2 { + t.Errorf("root must get no prefix: %v", got) + } +} + +// A guest stoat does not know renders no prelude: the recipe runs bare, as +// it always did before this feature existed. +func TestPreludeForUnknownOSIsEmpty(t *testing.T) { + if p := preludeFor(&config.VM{OS: "plan9"}, "sh"); p != "" { + t.Errorf("got %q", p) + } +} diff --git a/internal/tomlx/tomlx.go b/internal/tomlx/tomlx.go new file mode 100644 index 00000000..9c4ff8f7 --- /dev/null +++ b/internal/tomlx/tomlx.go @@ -0,0 +1,63 @@ +// Package tomlx is the one TOML decoder for vm.toml, recipe.toml, and +// guest.toml. It wraps the file path into every error, reports keys the +// target type does not declare, and bounds a file's schema version. +package tomlx + +import ( + "fmt" + "io" + "strings" + + "github.com/BurntSushi/toml" +) + +type options struct { + warn io.Writer // nil means unknown keys are an error + schemaMax int // 0 means no schema check +} + +// Option adjusts one Decode call. +type Option func(*options) + +// Reject makes an unknown key an error. It is the default and exists so a +// call site can say so. +var Reject Option = func(*options) {} + +// Warn writes one line per unknown key to w and keeps going. vm.toml uses +// it: a key an older stoat wrote must not mark the VM broken. +func Warn(w io.Writer) Option { return func(o *options) { o.warn = w } } + +// Schema errors when the file's top-level `schema` is greater than max. An +// absent schema passes; the caller decides what absence means. +func Schema(max int) Option { return func(o *options) { o.schemaMax = max } } + +// Decode reads path into v. +func Decode(path string, v any, opts ...Option) error { + var o options + for _, opt := range opts { + opt(&o) + } + md, err := toml.DecodeFile(path, v) + if err != nil { + return fmt.Errorf("%s: %w", path, err) + } + if o.schemaMax > 0 && md.IsDefined("schema") { + var s struct { + Schema int `toml:"schema"` + } + if _, err := toml.DecodeFile(path, &s); err != nil { + return fmt.Errorf("%s: %w", path, err) + } + if s.Schema > o.schemaMax { + return fmt.Errorf("%s: schema %d is newer than this stoat (%d)", path, s.Schema, o.schemaMax) + } + } + for _, k := range md.Undecoded() { + key := strings.Join(k, ".") + if o.warn == nil { + return fmt.Errorf("%s: unknown key %q", path, key) + } + fmt.Fprintf(o.warn, "%s: unknown key %q\n", path, key) + } + return nil +} diff --git a/internal/tomlx/tomlx_test.go b/internal/tomlx/tomlx_test.go new file mode 100644 index 00000000..2b47c352 --- /dev/null +++ b/internal/tomlx/tomlx_test.go @@ -0,0 +1,84 @@ +package tomlx + +import ( + "bytes" + "os" + "path/filepath" + "strings" + "testing" +) + +type doc struct { + Schema int `toml:"schema"` + Name string `toml:"name"` + Nested struct { + A string `toml:"a"` + } `toml:"nested"` +} + +func write(t *testing.T, body string) string { + t.Helper() + p := filepath.Join(t.TempDir(), "x.toml") + if err := os.WriteFile(p, []byte(body), 0o644); err != nil { + t.Fatal(err) + } + return p +} + +func TestDecodeRejectsUnknownKey(t *testing.T) { + p := write(t, "name = \"x\"\n[nested]\nb = 1\n") + var d doc + err := Decode(p, &d, Reject) + if err == nil { + t.Fatal("unknown key accepted") + } + for _, want := range []string{p, `unknown key "nested.b"`} { + if !strings.Contains(err.Error(), want) { + t.Errorf("error %q lacks %q", err, want) + } + } +} + +func TestDecodeWarnsUnknownKey(t *testing.T) { + p := write(t, "name = \"x\"\ncpus_ = 2\n") + var d doc + var w bytes.Buffer + if err := Decode(p, &d, Warn(&w)); err != nil { + t.Fatal(err) + } + if d.Name != "x" { + t.Errorf("name = %q", d.Name) + } + if !strings.Contains(w.String(), `unknown key "cpus_"`) { + t.Errorf("no warning written: %q", w.String()) + } +} + +func TestDecodeWrapsPath(t *testing.T) { + p := write(t, "name = \n") + var d doc + err := Decode(p, &d) + if err == nil || !strings.Contains(err.Error(), p) { + t.Errorf("error %v lacks the path", err) + } +} + +func TestDecodeSchemaTooNew(t *testing.T) { + p := write(t, "schema = 2\nname = \"x\"\n") + var d doc + err := Decode(p, &d, Schema(1)) + if err == nil || !strings.Contains(err.Error(), "schema 2 is newer than this stoat (1)") { + t.Errorf("err = %v", err) + } +} + +func TestDecodeSchemaAbsentIsFine(t *testing.T) { + p := write(t, "name = \"x\"\n") + var d doc + if err := Decode(p, &d, Schema(1)); err != nil { + t.Fatal(err) + } + if d.Schema != 0 { + t.Errorf("schema = %d, want 0 (absent)", d.Schema) + } +} diff --git a/internal/tui/app.go b/internal/tui/app.go index a29bcec6..e33847ce 100644 --- a/internal/tui/app.go +++ b/internal/tui/app.go @@ -13,6 +13,7 @@ import ( "github.com/novusedge/stoat/internal/config" "github.com/novusedge/stoat/internal/core" + "github.com/novusedge/stoat/internal/guest" "github.com/novusedge/stoat/internal/keys" "github.com/novusedge/stoat/internal/logx" "github.com/novusedge/stoat/internal/recipes" @@ -134,6 +135,7 @@ func Run() error { if err := keys.Ensure(); err != nil { return err } + guestErr := guest.Load(filepath.Join(config.Root(), "guests")) // ponytail: a log we can't open is not worth refusing to start over. // logx.L() falls back to io.Discard, so the TUI just runs without a log. _ = logx.Init() @@ -146,6 +148,10 @@ func Run() error { spin: newSpinner(), } m.preflight = preflightReport(core.Doctor()) + if guestErr != nil { + // A toast would expire; the preflight block stays until the user acts. + m.preflight += "\n" + guestErr.Error() + } _, err := tea.NewProgram(m).Run() return err }