diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md index 873d66da..9a261643 100644 --- a/docs/SUMMARY.md +++ b/docs/SUMMARY.md @@ -20,6 +20,9 @@ * [CLI](reference/cli.md) * [JSON output](reference/json.md) * [Guest definitions](reference/guest.md) +* [Recipe sample](reference/samples/recipe.toml) +* [VM sample](reference/samples/vm.toml) +* [Guest sample](reference/samples/guest.toml) ## Recipes diff --git a/docs/concepts/modes-and-backends.md b/docs/concepts/modes-and-backends.md index f827fed0..6f3bfb2b 100644 --- a/docs/concepts/modes-and-backends.md +++ b/docs/concepts/modes-and-backends.md @@ -95,6 +95,11 @@ recipes already ran at first boot, and that changing them means recreating the VM (the seed isn't rebuilt on later starts, since by then the overlay holds real guest state you don't want thrown away). +The host seed artifacts remain in the VM directory for inspection and later +diagnosis. Stoat creates seed directories with mode `0700` and seed files with +mode `0600` before writing their bytes; it does not promise to delete or +detach those artifacts after boot. + ## Comparison | | `live` | `disk` | `cloud` | diff --git a/docs/reference/cli.md b/docs/reference/cli.md index 2e6db57a..aaf7a49a 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -41,6 +41,7 @@ usage: stoat [flags] | [`recipes`](#stoat-recipes) | List recipes, optionally only applicable ones | 0, 1 | | [`check-recipes`](#stoat-check-recipes-names---osos) | Report why a recipe would not apply | 0, 1, 2 | | [`recipe list`](#stoat-recipe-list) | List installed recipes and where they live | 0, 1 | +| [`recipe show`](#stoat-recipe-show-name) | Show one recipe's parameter and output contract | 0, 1 | | [`recipe new`](#stoat-recipe-new-name) | Scaffold a recipe in the recipes directory | 0, 1 | | [`guest ls`](#stoat-guest-ls) | List loaded guest OS definitions | 0 | | [`guest show`](#stoat-guest-show-name) | Print one guest's merged definition | 0, 1 | @@ -104,7 +105,7 @@ created work (alpine, live, ssh port 2222) start it with: stoat up work ``` -Flags: `--image` (required; catalog id or a path to your own image), `--os`, `--backend` (override what a bring-your-own image's filename would otherwise infer), `--mode` (`live` or `disk`; only meaningful for the alpine iso, every other image has one mode), `--ram` (MB), `--cpus`, `--disk` (absolute size, e.g. `8G`), `--share` (host directory to expose), `--console-password` (`random` generates one), `--recipes` (comma-separated or repeated), `--allow-exec` (default true; `--allow-exec=false` opts this VM out of `exec`/`copy_to`/`copy_from`, enforced by the MCP server rather than stoat itself). +Flags: `--image` (required; catalog id or a path to your own image), `--os`, `--backend` (override what a bring-your-own image's filename would otherwise infer), `--mode` (`live` or `disk`; only meaningful for the alpine iso, every other image has one mode), `--ram` (MB), `--cpus`, `--disk` (absolute size, e.g. `8G`), `--share` (host directory to expose), `--console-password` (`random` generates one), `--recipes` (comma-separated or repeated), `--set recipe.param=value` (set a non-secret recipe parameter), `--secret recipe.param` (read a secret from the environment or prompt), `--allow-exec` (default true; `--allow-exec=false` opts this VM out of `exec`/`copy_to`/`copy_from`, enforced by the MCP server rather than stoat itself). **Exit codes:** 0 on success; 1 if creation fails (e.g. the image isn't downloaded yet: run `stoat pull` or download it from the TUI's image picker first); 2 if `--image` is missing. @@ -126,7 +127,7 @@ updated work: [share] `work`'s share is now unset. Compare to `stoat update work` with no flags at all, which is a usage error (there is nothing to change), not a no-op. -Flags: `--ram`, `--cpus`, `--ssh-port`, `--disk` (grow-only), `--share` (empty clears it), `--recipes` (empty clears it; replaces the whole list, it does not add to it). +Flags: `--ram`, `--cpus`, `--ssh-port`, `--disk` (grow-only), `--share` (empty clears it), `--recipes` (empty clears it; replaces the whole list, it does not add to it), `--set recipe.param=value`, `--unset recipe.param` (remove a non-secret override and restore its manifest default), and `--secret recipe.param` (set a secret without writing its value to `vm.toml`). Use `recipe show` to inspect declared types, defaults, enum values, and required parameters. Most fields are read by qemu only at start, so a change to a *running* VM is saved to `vm.toml` but doesn't take effect until the VM is next started; `update` says so: @@ -211,7 +212,7 @@ $ stoat wait work --until reachable work reached reachable (1240ms) ``` -`--until` is one of `reachable` (sshd answering on the VM's forwarded port, default), `applied` (the most recent recipe run finished), or `stopped` (qemu no longer running). `--timeout` (default `2m`) is a Go duration (`30s`, `5m`). +`--until` is one of `reachable` (sshd answering on the VM's forwarded port, default), `applied` (the most recent recipe run finished), or `stopped` (qemu no longer running). `--healthy` waits for reachability and then every applied recipe's declared health check; it cannot be combined with an explicit `--until`. `--timeout` (default `2m`) is a Go duration (`30s`, `5m`). A request that cannot ever be satisfied fails immediately rather than waiting out the timeout: `--until applied` on a VM with no recipes configured, or `--until reachable` on a VM that isn't running. @@ -471,17 +472,49 @@ $ stoat recipe list **Exit codes:** 0 on success; 1 if the directory can't be read. +## `stoat recipe show ` + +Prints the recipe's schema, sorted named parameters and outputs, and its +declared health check without requiring a VM: + +``` +$ stoat recipe show docker +docker: Docker engine and the compose plugin +schema: 3 +runtime: sh + +params: + user string, default dev account to add to the docker group + +outputs: + socket path of the docker socket + +health: docker info (timeout 30s) +``` + +Under `--json`, the result is `data.recipe` with the `RecipeSchema` documented +in [json.md](json.md). Secret parameter values are never part of this output; +the schema only says that a parameter has type `secret`. + ## `stoat recipe new ` -Scaffolds a new recipe file in the recipes directory and prints its path. +Scaffolds a new recipe directory (manifest plus scripts) and prints its path. ``` $ stoat recipe new mytool --os alpine -/home/user/.stoat/recipes/mytool.alpine.sh -edit it, then pick it in the new-vm form for a matching vm +/home/user/.stoat/recipes/mytool +edit its recipe.toml and scripts, then pick it in the new-vm form for a matching vm ``` -`--backend cloudinit` scaffolds a cloud-init fragment instead of a shell script. `-q` suppresses the trailing hint line. +`--backend` is accepted for CLI compatibility but does not change the scaffold: +all recipes are directories with a manifest and shell scripts. `-q` suppresses +the trailing hint line. + +`recipe new` copies the annotated [recipe sample](samples/recipe.toml), with +`name` and `os` filled for the new recipe. It creates the default script and +every script path declared by the sample's `[scripts]` overrides. The strict +VM and guest samples are [here](samples/vm.toml) and +[here](samples/guest.toml). **Exit codes:** 0 on success; 1 if the recipe can't be created (e.g. the name is already taken). diff --git a/docs/reference/json.md b/docs/reference/json.md index 3cf0d8b1..392a0204 100644 --- a/docs/reference/json.md +++ b/docs/reference/json.md @@ -190,6 +190,10 @@ VM {"name":"work","os":"alpine","mode":"cloud","backend":"cloudinit", "allow_exec":true,"display":"vnc", "error":"only on a broken VM"} +VMStatus {"name":"work",...VM fields...,"health":"ok","recipes_detail":[ + {"name":"xfce","applied":true,"version":"1.2","at":"...", + "health":"unknown","params":{},"outputs":{}}]} + Image {"id":"alpine-virt","os":"alpine","variant":"virt", "backend":"apkovl","file":"alpine-virt-3.24.1-x86_64.iso", "downloaded":true,"bytes":62914560,"bytes_exact":true,"byo":false} @@ -203,7 +207,17 @@ Check {"name":"qemu-img","ok":false,"detail":"not found", PruneItem {"class":"orphaned_image","path":"/home/u/.stoat/isos/old.iso"} Recipe {"name":"xfce","description":"XFCE desktop over SSH or at boot", - "reboot":false,"depends":[],"runtime":"sh"} + "schema":2,"runtime":"sh","reboot":false,"depends":[], + "params":[],"outputs":[],"health":null} + +RecipeSchema {"name":"docker","description":"Docker engine and the compose plugin", + "schema":3,"runtime":"sh","reboot":false,"depends":[], + "params":[RecipeParam,...],"outputs":[RecipeOutput,...], + "health":{"check":"docker info","timeout":"30s"}} +RecipeParam {"name":"channel","type":"enum","required":false, + "default":"stable","values":["stable","test"],"help":"..."} +RecipeOutput {"name":"socket","help":"path of the socket"} +RecipeHealth {"check":"docker info","timeout":"30s"} RecipeIssue {"name":"docker","reason":"docker is not offered to debian/cloudinit"} @@ -312,7 +326,7 @@ so a leak fails the build rather than shipping. | `cmd` | `data` | |---|---| | `ls` | `{"vms":[VM,...]}` | -| `get` | `{"vm":VM}` | +| `get` | `{"vm":VMStatus}` | | `create` | `{"vm":VM}` | | `update` | `{"vm":VM,"changed":["ram"],"applies_at":"now"}` | | `up` | `{"vm":VM}` (re-read after start, so `state` is authoritative) | @@ -337,8 +351,15 @@ so a leak fails the build rather than shipping. | `guest ls` | `{"guests":[Guest,...]}` | | `guest show` | `{"guest":Guest}` | | `recipe list` | `{"dir":"...","recipes":["xfce"]}`, see note below | -| `recipe new` | `{"path":"/home/u/.stoat/recipes/foo.alpine.sh"}` | +| `recipe show` | `{"recipe":RecipeSchema}` | +| `recipe new` | `{"path":"/home/u/.stoat/recipes/foo"}` | | `screenshot` | `{"vm":"work","path":"/home/u/.stoat/work/screenshots/2026-09-05T140302Z.png","bytes":48213,"width":1280,"height":800}` | +| `logs` (no VM) | `{"lines":[...]}` (stoat's own log) | +| `logs ` | `{"vm":"work","which":"console","lines":[...]}` | +| `doctor` | `{"healthy":false,"checks":[Check,...]}` | +| `version` | `{"version":"1.2.3","contract":2}` | +| `help` | `{"usage":"..."}` | +| `ssh` | **refused**, see below | Both `recipe` subcommands report `"cmd":"recipe"`, not `"cmd":"recipe list"`, and both `guest` subcommands report `"cmd":"guest"`. Distinguish them by which @@ -348,13 +369,18 @@ fields `data` carries. as "every recipe you can use": it currently includes the `.bak` files the one-time manifest upgrade left behind, and those are not applicable to any VM. Use `recipes` (which filters by OS and backend) to find something a VM can -actually run; use `recipe list` only to find a file to edit. -| `logs` (no VM) | `{"lines":[...]}` (stoat's own log) | -| `logs ` | `{"vm":"work","which":"console","lines":[...]}` | -| `doctor` | `{"healthy":false,"checks":[Check,...]}` | -| `version` | `{"version":"1.2.3","contract":2}` | -| `help` | `{"usage":"..."}` | -| `ssh` | **refused**, see below | +actually run; use `recipe list` only to find a recipe directory to inspect. + +`get` returns `{"vm":VMStatus}`: `VMStatus` embeds the VM fields directly; +only the outer get result has the `vm` member. `recipes` remains the compatible +string list, while `recipes_detail` adds stored per-recipe state. `health` is the stored aggregate +(`ok`, `failed`, or `unknown`); it is not a live SSH check. Every detail's +`params` and `outputs` is an object, even when empty. Secret parameters are +`` or `` and are never emitted as their value. + +`recipe show` and `recipes` use the same `RecipeSchema` projection. Parameters +and outputs are named arrays sorted by name. A recipe without a health check +has `health:null`; all list fields are `[]`, never `null`. Fields worth knowing about: diff --git a/docs/reference/samples/guest.toml b/docs/reference/samples/guest.toml new file mode 100644 index 00000000..8259d7bc --- /dev/null +++ b/docs/reference/samples/guest.toml @@ -0,0 +1,36 @@ +# Every field in a guest definition. A guest file describes the image and its +# package/service surface; recipes consume these facts through the prelude. +schema = 1 # int; required fixed value 1; guest author writes. +name = "alpine" # string; required; guest author writes. +shell = "/bin/ash" # string; required; guest author writes the login shell. +init = "openrc" # string; required; guest author writes: systemd, openrc, or rc. +installer = "setup-alpine" # string; default empty means "the installer"; guest author writes. +default_backend = "apkovl" # string; default none; guest author writes the create-time backend. +default_ssh_user = "root" # string; default none; guest author writes the create-time SSH user. +escalate = ["sudo", "-n"] # string[]; default []; guest author writes root escalation argv. +capabilities = ["apk"] # string[]; default []; guest author writes recipe capabilities; init is appended. +aliases = [] # string[]; default []; guest author writes alternate script keys. +filename_hints = ["alpine"] # string[]; default []; guest author writes BYO-image filename hints. +seed_packages = ["sudo"] # string[]; default []; guest author writes cloud-init seed packages. + +[pkg] +setup = "apk update" # string; default empty; guest author writes the package-index prelude. +install = ["apk", "--wait", "60", "add"] # string[]; default []; guest author writes install argv. +env = {} # map[string]string; default {}; guest author writes prelude environment. +scaffold_setup = "setup-apkrepos -c -1" # string; default empty; guest author writes scaffold comment text. +scaffold_install = "apk add " # string; default empty; guest author writes scaffold install text. +runtime_packages = { python3 = "python3" } # map[string]string; default {}; guest author writes runtime packages. + +[svc] +enable = "rc-update add {name} default" # string; required; guest author writes service-enable template. +start = "rc-service {name} start" # string; required; guest author writes service-start template. +stop = "rc-service {name} stop" # string; required; guest author writes service-stop template. +restart = "rc-service {name} restart" # string; required; guest author writes service-restart template. +status = "rc-service {name} status" # string; required; guest author writes service-status template. + +[cmd] +download = "wget -O" # string; default empty; guest author writes the image download command. +useradd = "adduser -D {name}" # string; default empty; guest author writes the account command. + +[backend.cloudinit] +skip_9p = false # bool; default false; cloud-init backend owner writes this opaque setting. diff --git a/docs/reference/samples/recipe.toml b/docs/reference/samples/recipe.toml new file mode 120000 index 00000000..a6d519d6 --- /dev/null +++ b/docs/reference/samples/recipe.toml @@ -0,0 +1 @@ +../../../internal/recipes/samples/recipe.toml \ No newline at end of file diff --git a/docs/reference/samples/vm.toml b/docs/reference/samples/vm.toml new file mode 100644 index 00000000..688e789c --- /dev/null +++ b/docs/reference/samples/vm.toml @@ -0,0 +1,40 @@ +# Every field a vm.toml can carry. stoat writes this file; a human edits the +# resource fields and recipe params. `stoat update` is safer for automation. + +name = "work" # string; default none; user creates, stoat writes the directory identity. +mode = "disk" # string; default inferred from image/backend; stoat writes: live, disk, or cloud. +os = "alpine" # string; default inferred from image; stoat writes the guest definition name. +iso = "isos/x.iso" # string path; default none; stoat writes it relative to the data root. +ram = 2048 # int MB; default 4096; stoat writes at create/update. +cpus = 2 # int; default 4; stoat writes at create/update. +disk = "16G" # string; default 8G in disk mode; stoat writes, grow-only after creation. +installed = true # bool; default false; stoat writes for disk mode and flips boot order. +share = "~/src" # string path; default empty; user/TUI and stoat update write the host share. +sshport = 2200 # int; default an allocated free port; stoat writes the host forward to guest sshd. +recipes = ["docker"] # string[]; default []; user/TUI and stoat create/update write the selection. +display = "auto" # string; default "auto"; user/TUI writes: auto, window, or vnc. +backend = "cloudinit" # string; default inferred from image; stoat writes: apkovl, cloudinit, or ssh. +base = "" # string path; default empty; stoat writes the absolute shared base-image path. +sshuser = "stoat" # string; default guest-defined user (empty means root); stoat writes it. +console_password = "" # string; default "stoat" for cloud VMs, empty otherwise; stoat writes, never ssh. +allow_exec = true # bool; default true; user/TUI and stoat create write the MCP exec/copy opt-in. + +[[forwards]] # table[]; default []; user/TUI and `stoat forward` write extra forwards. +hostport = 8080 # int; default none; user writes the host port. +guestport = 80 # int; default none; user writes the guest port. + +[params.docker] # table; default {}; stoat writes parameter values; do not edit by hand. +user = "dev" # string; default recipe value "dev"; stoat writes the non-secret override. +channel = "stable" # string; default recipe value; stoat writes the non-secret override. + +# Written by stoat; do not edit. +[applied.docker] +version = "1.2.0" # string; default empty; stoat writes the applied recipe version. +hash = "recipe-and-params-hash" # string; default empty; stoat writes the recipe/params hash. +script_hash = "script-hash" # string; default empty; stoat writes the applied script hash. +at = 2026-09-04T10:00:00Z # datetime; default zero; stoat writes the apply time. +health = "ok" # string; default unknown; stoat writes the stored health result. + +# Written by stoat; do not edit. +[applied.docker.outputs] +socket = "/var/run/docker.sock" # string; default empty; stoat writes recipe output values. diff --git a/go.mod b/go.mod index 2aa00e77..744ca0a1 100644 --- a/go.mod +++ b/go.mod @@ -5,28 +5,35 @@ go 1.26 require ( charm.land/bubbles/v2 v2.1.1 charm.land/bubbletea/v2 v2.0.8 + charm.land/huh/v2 v2.0.3 charm.land/lipgloss/v2 v2.0.5 charm.land/log/v2 v2.0.0 github.com/BurntSushi/toml v1.6.0 github.com/alecthomas/kong v1.16.0 github.com/charmbracelet/x/ansi v0.11.7 + github.com/pelletier/go-toml/v2 v2.4.3 golang.org/x/sys v0.47.0 gopkg.in/yaml.v3 v3.0.1 ) require ( github.com/atotto/clipboard v0.1.4 // indirect + github.com/catppuccin/go v0.2.0 // indirect github.com/charmbracelet/colorprofile v0.4.3 // indirect github.com/charmbracelet/harmonica v0.2.0 // indirect github.com/charmbracelet/ultraviolet v0.0.0-20260703014108-f5a850f9c2b7 // indirect + github.com/charmbracelet/x/exp/ordered v0.1.0 // indirect + github.com/charmbracelet/x/exp/strings v0.0.0-20240722160745-212f7b056ed0 // indirect github.com/charmbracelet/x/term v0.2.2 // indirect github.com/charmbracelet/x/termios v0.1.1 // indirect github.com/charmbracelet/x/windows v0.2.2 // indirect github.com/clipperhouse/displaywidth v0.11.0 // indirect github.com/clipperhouse/uax29/v2 v2.7.0 // indirect + github.com/dustin/go-humanize v1.0.1 // indirect github.com/go-logfmt/logfmt v0.6.1 // indirect github.com/lucasb-eyer/go-colorful v1.4.0 // indirect github.com/mattn/go-runewidth v0.0.27 // indirect + github.com/mitchellh/hashstructure/v2 v2.0.2 // indirect github.com/muesli/cancelreader v0.2.2 // indirect github.com/rivo/uniseg v0.4.7 // indirect github.com/sahilm/fuzzy v0.1.3 // indirect diff --git a/go.sum b/go.sum index 80374f26..1f81cdf2 100644 --- a/go.sum +++ b/go.sum @@ -2,12 +2,16 @@ charm.land/bubbles/v2 v2.1.1 h1:7r55WzBxpo/R3z98hGmY7KKPd3ET6vsf0Fb9sDHOV60= charm.land/bubbles/v2 v2.1.1/go.mod h1:GE6M31gaWZVXzGw73OeuTTgy4lX+OtkH0E5ymnNsHxo= charm.land/bubbletea/v2 v2.0.8 h1:SxTJMhCAI3lbPmy4SgX5LWZ24AdINr4I6UEqzZvYJuY= charm.land/bubbletea/v2 v2.0.8/go.mod h1:2SkdgoTXluXJHOUwAoRlRXF/28vklb1rFl6GcgV1/ss= +charm.land/huh/v2 v2.0.3 h1:2cJsMqEPwSywGHvdlKsJyQKPtSJLVnFKyFbsYZTlLkU= +charm.land/huh/v2 v2.0.3/go.mod h1:93eEveeeqn47MwiC3tf+2atZ2l7Is88rAtmZNZ8x9Wc= charm.land/lipgloss/v2 v2.0.5 h1:kbNxgeeUOYv5J0YdpxFjfvf3dFvqH8Aci4zB6xqFtrY= charm.land/lipgloss/v2 v2.0.5/go.mod h1:9oqhxt4yxIMe6q5A4kHr44DremZk7J9UNh74GlWa5nc= charm.land/log/v2 v2.0.0 h1:SY3Cey7ipx86/MBXQHwsguOT6X1exT94mmJRdzTNs+s= charm.land/log/v2 v2.0.0/go.mod h1:c3cZSRqm20qUVVAR1WmS/7ab8bgha3C6G7DjPcaVZz0= github.com/BurntSushi/toml v1.6.0 h1:dRaEfpa2VI55EwlIW72hMRHdWouJeRF7TPYhI+AUQjk= github.com/BurntSushi/toml v1.6.0/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho= +github.com/MakeNowJust/heredoc v1.0.0 h1:cXCdzVdstXyiTqTvfqk9SDHpKNjxuom+DOlyEeQ4pzQ= +github.com/MakeNowJust/heredoc v1.0.0/go.mod h1:mG5amYoWBHf8vpLOuehzbGGw0EHxpZZ6lCpQ4fNJ8LE= github.com/alecthomas/assert/v2 v2.11.0 h1:2Q9r3ki8+JYXvGsDyBXwH3LcJ+WK5D0gc5E8vS6K3D0= github.com/alecthomas/assert/v2 v2.11.0/go.mod h1:Bze95FyfUr7x34QZrjL+XP+0qgp/zg8yS+TtBj1WA3k= github.com/alecthomas/kong v1.16.0 h1:g92/kUxBcdcTPOM79yE63viJgtcp5dNyrB3/O2cjYT4= @@ -18,6 +22,8 @@ github.com/atotto/clipboard v0.1.4 h1:EH0zSVneZPSuFR11BlR9YppQTVDbh5+16AmcJi4g1z github.com/atotto/clipboard v0.1.4/go.mod h1:ZY9tmq7sm5xIbd9bOK4onWV4S6X0u6GY7Vn0Yu86PYI= github.com/aymanbagabas/go-udiff v0.4.1 h1:OEIrQ8maEeDBXQDoGCbbTTXYJMYRCRO1fnodZ12Gv5o= github.com/aymanbagabas/go-udiff v0.4.1/go.mod h1:0L9PGwj20lrtmEMeyw4WKJ/TMyDtvAoK9bf2u/mNo3w= +github.com/catppuccin/go v0.2.0 h1:ktBeIrIP42b/8FGiScP9sgrWOss3lw0Z5SktRoithGA= +github.com/catppuccin/go v0.2.0/go.mod h1:8IHJuMGaUUjQM82qBrGNBv7LFq6JI3NnQCF6MOlZjpc= github.com/charmbracelet/colorprofile v0.4.3 h1:QPa1IWkYI+AOB+fE+mg/5/4HRMZcaXex9t5KX76i20Q= github.com/charmbracelet/colorprofile v0.4.3/go.mod h1:/zT4BhpD5aGFpqQQqw7a+VtHCzu+zrQtt1zhMt9mR4Q= github.com/charmbracelet/harmonica v0.2.0 h1:8NxJWRWg/bzKqqEaaeFNipOu77YR5t8aSwG4pgaUBiQ= @@ -26,20 +32,34 @@ github.com/charmbracelet/ultraviolet v0.0.0-20260703014108-f5a850f9c2b7 h1:3FmWo github.com/charmbracelet/ultraviolet v0.0.0-20260703014108-f5a850f9c2b7/go.mod h1:f/jRa757WUmaOZrbPspXymbg/GnbF+rwe4OLsG7aXYo= github.com/charmbracelet/x/ansi v0.11.7 h1:kzv1kJvjg2S3r9KHo8hDdHFQLEqn4RBCb39dAYC84jI= github.com/charmbracelet/x/ansi v0.11.7/go.mod h1:9qGpnAVYz+8ACONkZBUWPtL7lulP9No6p1epAihUZwQ= +github.com/charmbracelet/x/conpty v0.1.1 h1:s1bUxjoi7EpqiXysVtC+a8RrvPPNcNvAjfi4jxsAuEs= +github.com/charmbracelet/x/conpty v0.1.1/go.mod h1:OmtR77VODEFbiTzGE9G1XiRJAga6011PIm4u5fTNZpk= +github.com/charmbracelet/x/errors v0.0.0-20240508181413-e8d8b6e2de86 h1:JSt3B+U9iqk37QUU2Rvb6DSBYRLtWqFqfxf8l5hOZUA= +github.com/charmbracelet/x/errors v0.0.0-20240508181413-e8d8b6e2de86/go.mod h1:2P0UgXMEa6TsToMSuFqKFQR+fZTO9CNGUNokkPatT/0= github.com/charmbracelet/x/exp/golden v0.0.0-20250806222409-83e3a29d542f h1:pk6gmGpCE7F3FcjaOEKYriCvpmIN4+6OS/RD0vm4uIA= github.com/charmbracelet/x/exp/golden v0.0.0-20250806222409-83e3a29d542f/go.mod h1:IfZAMTHB6XkZSeXUqriemErjAWCCzT0LwjKFYCZyw0I= +github.com/charmbracelet/x/exp/ordered v0.1.0 h1:55/qLwjIh0gL0Vni+QAWk7T/qRVP6sBf+2agPBgnOFE= +github.com/charmbracelet/x/exp/ordered v0.1.0/go.mod h1:5UHwmG+is5THxMyCJHNPCn2/ecI07aKNrW+LcResjJ8= +github.com/charmbracelet/x/exp/strings v0.0.0-20240722160745-212f7b056ed0 h1:qko3AQ4gK1MTS/de7F5hPGx6/k1u0w4TeYmBFwzYVP4= +github.com/charmbracelet/x/exp/strings v0.0.0-20240722160745-212f7b056ed0/go.mod h1:pBhA0ybfXv6hDjQUZ7hk1lVxBiUbupdw5R31yPUViVQ= github.com/charmbracelet/x/term v0.2.2 h1:xVRT/S2ZcKdhhOuSP4t5cLi5o+JxklsoEObBSgfgZRk= github.com/charmbracelet/x/term v0.2.2/go.mod h1:kF8CY5RddLWrsgVwpw4kAa6TESp6EB5y3uxGLeCqzAI= github.com/charmbracelet/x/termios v0.1.1 h1:o3Q2bT8eqzGnGPOYheoYS8eEleT5ZVNYNy8JawjaNZY= github.com/charmbracelet/x/termios v0.1.1/go.mod h1:rB7fnv1TgOPOyyKRJ9o+AsTU/vK5WHJ2ivHeut/Pcwo= github.com/charmbracelet/x/windows v0.2.2 h1:IofanmuvaxnKHuV04sC0eBy/smG6kIKrWG2/jYn2GuM= github.com/charmbracelet/x/windows v0.2.2/go.mod h1:/8XtdKZzedat74NQFn0NGlGL4soHB0YQZrETF96h75k= +github.com/charmbracelet/x/xpty v0.1.3 h1:eGSitii4suhzrISYH50ZfufV3v085BXQwIytcOdFSsw= +github.com/charmbracelet/x/xpty v0.1.3/go.mod h1:poPYpWuLDBFCKmKLDnhBp51ATa0ooD8FhypRwEFtH3Y= github.com/clipperhouse/displaywidth v0.11.0 h1:lBc6kY44VFw+TDx4I8opi/EtL9m20WSEFgwIwO+UVM8= github.com/clipperhouse/displaywidth v0.11.0/go.mod h1:bkrFNkf81G8HyVqmKGxsPufD3JhNl3dSqnGhOoSD/o0= github.com/clipperhouse/uax29/v2 v2.7.0 h1:+gs4oBZ2gPfVrKPthwbMzWZDaAFPGYK72F0NJv2v7Vk= github.com/clipperhouse/uax29/v2 v2.7.0/go.mod h1:EFJ2TJMRUaplDxHKj1qAEhCtQPW2tJSwu5BF98AuoVM= +github.com/creack/pty v1.1.24 h1:bJrF4RRfyJnbTJqzRLHzcGaZK1NeM5kTC9jGgovnR1s= +github.com/creack/pty v1.1.24/go.mod h1:08sCNb52WyoAwi2QDyzUCTgcvVFhUzewun7wtTfvcwE= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY= +github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto= github.com/go-logfmt/logfmt v0.6.1 h1:4hvbpePJKnIzH1B+8OR/JPbTx37NktoI9LE2QZBBkvE= github.com/go-logfmt/logfmt v0.6.1/go.mod h1:EV2pOAQoZaT1ZXZbqDl5hrymndi4SY9ED9/z6CO0XAk= github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= @@ -52,8 +72,12 @@ github.com/lucasb-eyer/go-colorful v1.4.0 h1:UtrWVfLdarDgc44HcS7pYloGHJUjHV/4FwW github.com/lucasb-eyer/go-colorful v1.4.0/go.mod h1:R4dSotOR9KMtayYi1e77YzuveK+i7ruzyGqttikkLy0= github.com/mattn/go-runewidth v0.0.27 h1:Feg/Oou5zI/wnpgDF6omIU0OokC9GxLC/WRknhVlIR0= github.com/mattn/go-runewidth v0.0.27/go.mod h1:3qAiGCV4Koz/yuveO58qUefmUTRm8r0IGEXZ9jeHp/8= +github.com/mitchellh/hashstructure/v2 v2.0.2 h1:vGKWl0YJqUNxE8d+h8f6NJLcCJrgbhC4NcD46KavDd4= +github.com/mitchellh/hashstructure/v2 v2.0.2/go.mod h1:MG3aRVU/N29oo/V/IhBX8GR/zz4kQkprJgF2EVszyDE= github.com/muesli/cancelreader v0.2.2 h1:3I4Kt4BQjOR54NavqnDogx/MIoWBFa0StPA8ELUXHmA= github.com/muesli/cancelreader v0.2.2/go.mod h1:3XuTXfFS2VjM+HTLZY9Ak0l6eUKfijIfMUZ4EgX0QYo= +github.com/pelletier/go-toml/v2 v2.4.3 h1:GTRvJQutkOSftxIFD5xw9aepkYNuPWmVJpffdDPYVpY= +github.com/pelletier/go-toml/v2 v2.4.3/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ= diff --git a/internal/backend/cloudinit.go b/internal/backend/cloudinit.go index 4b056be5..639744c5 100644 --- a/internal/backend/cloudinit.go +++ b/internal/backend/cloudinit.go @@ -98,6 +98,10 @@ func (cloudinitBackend) Prepare(v *config.VM) error { // renders: each recipe's manifest, then the script body for v.OS. A recipe // with no recipe.toml went missing since create time and errors here. func recipeScripts(v *config.VM) ([]cloudinit.Script, error) { + stored, err := config.LoadSecrets(v.Dir) + if err != nil { + return nil, fmt.Errorf("reading recipe secrets: %w", err) + } var scripts []cloudinit.Script for _, name := range v.Recipes { m, ok, err := recipes.ManifestFor(name) @@ -111,7 +115,26 @@ func recipeScripts(v *config.VM) ([]cloudinit.Script, error) { if err != nil { return nil, fmt.Errorf("reading recipe %s: %w", name, err) } - scripts = append(scripts, cloudinit.Script{Name: name, Content: body}) + resolved, err := recipes.Resolve(m, v.Params[name], stored[name]) + if err != nil { + return nil, fmt.Errorf("resolving recipe %s: %w", name, err) + } + nonSecrets := make(map[string]string) + secrets := make(map[string]string) + for param, value := range resolved { + if m.Params[param].Type == "secret" { + if value != "" { + secrets[param] = value + } + continue + } + nonSecrets[param] = value + } + var env []string + if m.Schema >= 3 { + env = recipes.Env(name, nonSecrets) + } + scripts = append(scripts, cloudinit.Script{Name: name, Content: body, Env: env, Secrets: secrets}) } return scripts, nil } diff --git a/internal/cli/cli.go b/internal/cli/cli.go index d1d6748d..171d7a80 100644 --- a/internal/cli/cli.go +++ b/internal/cli/cli.go @@ -117,10 +117,11 @@ type Args struct { // Until and Timeout belong to "wait"; Which belongs to "logs"; Only // belongs to "apply" and carries the names for "check-recipes". - Until core.Until - Timeout time.Duration - Which core.Which - Only []string + Until core.Until + UntilExplicit bool + Timeout time.Duration + Which core.Which + Only []string // Patch belongs to "update", and Changed names the flags that were // actually GIVEN. core.Patch is all pointers so "not set" differs from @@ -129,6 +130,18 @@ type Args struct { // nil pointer, and `--share ""` is a pointer to the empty string. Patch core.Patch Changed []string + Params []ParamEdit +} + +// ParamEdit is one recipe parameter edit parsed from create or update flags. +// Secret values are resolved at the run boundary, not while Parse interprets +// argv, so parsing remains free of prompts and environment reads. +type ParamEdit struct { + Recipe string + Param string + Value string + Secret bool + Unset bool } // usageError marks a Parse failure as an exit-2 condition. Every Parse @@ -206,7 +219,22 @@ func Parse(args []string) (*Args, error) { if perr != nil { return nil, usageError(perr.Error()) } - return g.toArgs(commandPath(ctx)) + a, err := g.toArgs(commandPath(ctx)) + if err != nil { + return nil, err + } + if a.Cmd == "wait" { + for _, arg := range args { + if arg == "--until" || strings.HasPrefix(arg, "--until=") { + a.UntilExplicit = true + break + } + } + if a.Until == core.UntilHealthy && a.UntilExplicit { + return nil, usageError("wait: --healthy and --until are two different waits; pass one") + } + } + return a, nil } // parseExec handles `exec ...` without kong. Kong's passthrough is @@ -327,6 +355,13 @@ func Main(args []string, version string, stdin io.Reader, stdout, stderr io.Writ // lines: they are all already gated on it. a.Quiet = true } + if len(a.Params) > 0 { + resolved, err := resolveParamEdits(a.Params, stdin, stderr, !jsonMode && streamIsTTY(stdin)) + if err != nil { + return a.failMsg(stdout, stderr, core.ErrInvalidSpec, err.Error()) + } + a.Params = resolved + } switch a.Cmd { case "help": diff --git a/internal/cli/grammar.go b/internal/cli/grammar.go index 59a9db5c..99136382 100644 --- a/internal/cli/grammar.go +++ b/internal/cli/grammar.go @@ -111,6 +111,8 @@ type createCmd struct { Share string `help:"host directory to expose in the guest"` ConsolePassword string `help:"console password; \"random\" generates one"` Recipes []string `help:"recipe names to record on the VM"` + Set []string `help:"set a recipe param: .="` + Secret []string `help:"set a secret recipe param"` // default:"true" is load-bearing, not decoration: without it kong treats // an absent --allow-exec the same as an explicit --allow-exec=false, // since a bare bool flag's zero value is false. With it, the flag must @@ -129,6 +131,9 @@ type updateCmd struct { Disk *string `help:"disk size, absolute only and grow-only (16G)"` Share *string `help:"host directory to expose in the guest; empty clears it"` Recipes *[]string `help:"replace the recipe list; empty clears it"` + Set []string `help:"set a recipe param: .="` + Unset []string `help:"clear a recipe param back to its manifest default"` + Secret []string `help:"set a secret recipe param"` } type cloneCmd struct { @@ -195,6 +200,7 @@ type pruneCmd struct { type waitCmd struct { VM string `arg:"" help:"vm name"` Until string `enum:"reachable,applied,stopped" default:"reachable" help:"state to wait for"` + Healthy bool `help:"wait for every applied recipe's health check to pass"` Timeout time.Duration `default:"2m" help:"give up after this long"` } @@ -218,6 +224,7 @@ type checkRecipesCmd struct { type recipeCmd struct { List recipeListCmd `cmd:"" help:"list installed recipes and where they live"` New recipeNewCmd `cmd:"" help:"scaffold a recipe in the recipes directory"` + Show recipeShowCmd `cmd:"" help:"print one recipe's params, outputs and health check"` } type recipeListCmd struct{} @@ -228,6 +235,10 @@ type recipeNewCmd struct { Backend string `help:"\"cloudinit\" for a cloud-init fragment; shell otherwise"` } +type recipeShowCmd struct { + Name string `arg:"" help:"recipe name"` +} + type recipeGuestCmd struct { LS guestLsCmd `cmd:"" name:"ls" help:"one line per guest: name, init, package manager, backend, source"` Show guestShowCmd `cmd:"" help:"the merged definition of one guest"` @@ -286,6 +297,11 @@ func (g *grammar) toArgs(path string) (*Args, error) { ConsolePassword: c.ConsolePassword, Recipes: trimList(c.Recipes), AllowExec: &allowExec, } + edits, err := parseParamFlags(c.Set, nil, c.Secret) + if err != nil { + return nil, err + } + a.Params = edits case "update": u := g.Update @@ -315,8 +331,16 @@ func (g *grammar) toArgs(path string) (*Args, error) { a.Changed = append(a.Changed, f.name) } } + edits, err := parseParamFlags(u.Set, u.Unset, u.Secret) + if err != nil { + return nil, err + } + a.Params = edits + if len(edits) > 0 { + a.Changed = append(a.Changed, "params") + } if len(a.Changed) == 0 { - return nil, usageError("update: nothing to change; pass at least one of --ram --cpus --disk --share --ssh-port --recipes") + return nil, usageError("update: nothing to change; pass at least one of --ram --cpus --disk --share --ssh-port --recipes --set --unset --secret") } case "clone": @@ -398,7 +422,15 @@ func (g *grammar) toArgs(path string) (*Args, error) { if w.Timeout <= 0 { return nil, usageError("wait: --timeout must be positive") } - a.VM, a.Until, a.Timeout = w.VM, core.Until(w.Until), w.Timeout + if w.Healthy { + if w.Until != "reachable" { + return nil, usageError("wait: --healthy and --until are two different waits; pass one") + } + a.Until = core.UntilHealthy + } else { + a.Until = core.Until(w.Until) + } + a.VM, a.Timeout = w.VM, w.Timeout case "apply": a.VM, a.Only, a.DryRun = g.Apply.VM, trimList(g.Apply.Only), g.Apply.DryRun @@ -420,6 +452,10 @@ func (g *grammar) toArgs(path string) (*Args, error) { a.Cmd, a.Sub = "recipe", "new" a.VM, a.OS, a.Backend = n.Name, n.OS, n.Backend + case "recipe show": + a.Cmd, a.Sub = "recipe", "show" + a.VM = g.Recipe.Show.Name + case "guest ls": a.Cmd, a.Sub = "guest", "ls" diff --git a/internal/cli/json_test.go b/internal/cli/json_test.go index ddfed2a0..1550e278 100644 --- a/internal/cli/json_test.go +++ b/internal/cli/json_test.go @@ -3,12 +3,15 @@ package cli import ( "bytes" "encoding/json" + "os" + "path/filepath" "slices" "strings" "testing" "github.com/novusedge/stoat/internal/cli/wire" "github.com/novusedge/stoat/internal/config" + "github.com/novusedge/stoat/internal/recipes" ) // runJSON runs Main with --json and returns every stdout line decoded. It @@ -74,6 +77,9 @@ func TestJSONEnvelopeEveryCommand(t *testing.T) { {name: "prune", argv: []string{"prune"}, ok: true, exit: ExitOK}, {name: "logs", argv: []string{"logs"}, ok: true, exit: ExitOK}, {name: "recipe list", argv: []string{"recipe", "list"}, ok: true, exit: ExitOK}, + {name: "wait healthy and until", argv: []string{"wait", "work", "--healthy", "--until", "applied"}, code: wire.CodeUsage, exit: ExitUsage}, + {name: "recipe show", argv: []string{"recipe", "show", "docker"}, ok: true, exit: ExitOK}, + {name: "recipe show unknown", argv: []string{"recipe", "show", "nope"}, code: wire.CodeNotFound, exit: ExitFail}, {name: "guest ls", argv: []string{"guest", "ls"}, ok: true, exit: ExitOK}, {name: "guest show", argv: []string{"guest", "show", "alpine"}, ok: true, exit: ExitOK}, {name: "guest show unknown", argv: []string{"guest", "show", "plan9"}, code: wire.CodeNotFound, exit: ExitFail}, @@ -104,6 +110,11 @@ func TestJSONEnvelopeEveryCommand(t *testing.T) { for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { cliRoot(t) + if tt.name == "recipe show" { + if err := recipes.Install(); err != nil { + t.Fatal(err) + } + } if err := (&config.VM{Name: "work", Mode: "live", RAM: 1024, CPUs: 1, SSHPort: 2200}).Save(); err != nil { t.Fatal(err) } @@ -170,6 +181,38 @@ func TestJSONEmptyListsAreArraysNotNull(t *testing.T) { } } +// Recipe show is a caller boundary, so pin the named contract payload rather +// than accepting a generic map whose fields can drift from recipe list. +func TestJSONRecipeShowCarriesNamedContract(t *testing.T) { + cliRoot(t) + if err := recipes.Install(); err != nil { + t.Fatal(err) + } + code, objs := runJSON(t, "recipe", "show", "docker") + if code != ExitOK { + t.Fatalf("recipe show exit = %d, want %d: %v", code, ExitOK, objs) + } + data, ok := result(t, objs)["data"].(map[string]any) + if !ok { + t.Fatalf("recipe show data = %#v, want object", result(t, objs)["data"]) + } + show, ok := data["recipe"].(map[string]any) + if !ok { + t.Fatalf("recipe show data.recipe = %#v, want named object", data["recipe"]) + } + for _, field := range []string{"name", "schema", "params", "outputs", "health"} { + if _, exists := show[field]; !exists { + t.Errorf("recipe show omitted %q: %v", field, show) + } + } + if _, ok := show["params"].([]any); !ok { + t.Errorf("recipe show params = %#v, want array", show["params"]) + } + if _, ok := show["outputs"].([]any); !ok { + t.Errorf("recipe show outputs = %#v, want array", show["outputs"]) + } +} + // The three fields that must never reach the wire. Asserted here, outside // package wire, because this is the path a real consumer sees: the DTO could // be correct and a run body could still hand raw core types to the encoder. @@ -191,6 +234,96 @@ func TestJSONNeverLeaksHostPathsOrConsolePassword(t *testing.T) { } } +// Get is a real wire sink, not just a DTO unit test: a stored secret must be +// represented by the redacted marker in the status detail and never by the +// value loaded from secrets.toml. +func TestJSONGetRedactsStoredSecretValue(t *testing.T) { + dir := cliRoot(t) + recipeDir := filepath.Join(dir, "recipes", "redaction") + if err := os.MkdirAll(recipeDir, 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(recipeDir, "recipe.toml"), []byte(`schema = 3 +name = "redaction" +script = "install.sh" + +[params.token] +type = "secret" +required = true +`), 0o644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(recipeDir, "install.sh"), []byte("#!/bin/sh\n"), 0o755); err != nil { + t.Fatal(err) + } + v := &config.VM{Name: "work", Mode: "live", RAM: 1024, CPUs: 1, SSHPort: 2200, Recipes: []string{"redaction"}} + if err := v.Save(); err != nil { + t.Fatal(err) + } + const sentinel = "synthetic-secret-sentinel" + if err := config.SaveSecrets(v.Dir, config.Secrets{"redaction": {"token": sentinel}}); err != nil { + t.Fatal(err) + } + + code, objs := runJSON(t, "get", "work") + if code != ExitOK { + t.Fatalf("get exit = %d, want %d: %v", code, ExitOK, objs) + } + raw, err := json.Marshal(objs) + if err != nil { + t.Fatal(err) + } + if bytes.Contains(raw, []byte(sentinel)) { + t.Fatalf("get output leaked secret %q: %s", sentinel, raw) + } + data, ok := result(t, objs)["data"].(map[string]any) + if !ok { + t.Fatalf("get data = %#v, want object", result(t, objs)["data"]) + } + vm, ok := data["vm"].(map[string]any) + if !ok { + t.Fatalf("get data.vm = %#v, want object", data["vm"]) + } + detail, ok := vm["recipes_detail"].([]any) + if !ok || len(detail) != 1 { + t.Fatalf("recipes_detail = %#v, want one recipe detail", vm["recipes_detail"]) + } + state, ok := detail[0].(map[string]any) + if !ok || state["name"] != "redaction" { + t.Fatalf("recipe detail = %#v, want named redaction state", detail[0]) + } + params, ok := state["params"].(map[string]any) + if !ok { + t.Fatalf("recipe detail params = %#v, want object", state["params"]) + } + if params["token"] != "" { + t.Fatalf("recipe detail token = %#v, want ", params["token"]) + } +} + +// List must not silently omit a VM merely because its secret store is +// unreadable. The error remains tied to the VM so a caller can repair the +// right directory. +func TestJSONListPropagatesInsecureSecretErrorWithVMContext(t *testing.T) { + dir := cliRoot(t) + v := &config.VM{Name: "work", Mode: "live", RAM: 1024, CPUs: 1, SSHPort: 2200} + if err := v.Save(); err != nil { + t.Fatal(err) + } + path := filepath.Join(dir, "work", config.SecretsName) + if err := os.WriteFile(path, []byte("[redaction]\ntoken = \"x\"\n"), 0o644); err != nil { + t.Fatal(err) + } + code, objs := runJSON(t, "ls") + if code != ExitFail { + t.Fatalf("ls exit = %d, want %d: %v", code, ExitFail, objs) + } + raw, _ := json.Marshal(objs) + if !bytes.Contains(raw, []byte("work")) || !bytes.Contains(raw, []byte(config.SecretsName)) { + t.Fatalf("insecure secret error lacks VM context: %s", raw) + } +} + // --json implies non-interactive: rm must not read stdin looking for a "y", // or an MCP server that pipes a stray newline gets a VM deleted. func TestJSONRMNeverReadsStdin(t *testing.T) { diff --git a/internal/cli/paramflags.go b/internal/cli/paramflags.go new file mode 100644 index 00000000..88c6cac6 --- /dev/null +++ b/internal/cli/paramflags.go @@ -0,0 +1,117 @@ +package cli + +import ( + "bufio" + "fmt" + "io" + "os" + "strings" + + "github.com/novusedge/stoat/internal/config" +) + +// parseParamFlags turns parameter flag values into edits without consulting +// the environment or prompting. Secret resolution belongs to Main, where the +// caller supplies stdin and stderr. +func parseParamFlags(set, unset, secret []string) ([]ParamEdit, error) { + var edits []ParamEdit + for _, raw := range set { + target, value, ok := strings.Cut(raw, "=") + recipe, param, valid := splitParamTarget(target) + if !valid || !ok { + return nil, usageError(fmt.Sprintf("--set %s: want .=", raw)) + } + edits = append(edits, ParamEdit{Recipe: recipe, Param: param, Value: value}) + } + for _, raw := range unset { + recipe, param, valid := splitParamTarget(raw) + if !valid || strings.Contains(raw, "=") { + return nil, usageError(fmt.Sprintf("--unset %s: want .", raw)) + } + edits = append(edits, ParamEdit{Recipe: recipe, Param: param, Unset: true}) + } + for _, raw := range secret { + recipe, param, valid := splitParamTarget(raw) + if !valid || strings.Contains(raw, "=") { + return nil, usageError(fmt.Sprintf("--secret %s: want .", raw)) + } + edits = append(edits, ParamEdit{Recipe: recipe, Param: param, Secret: true}) + } + return edits, nil +} + +func splitParamTarget(target string) (recipe, param string, ok bool) { + recipe, param, ok = strings.Cut(target, ".") + return recipe, param, ok && recipe != "" && param != "" +} + +// resolveParamEdits fills secret values at the run boundary. A real terminal +// permits a prompt; JSON and non-terminal callers must provide an environment +// value so a command never blocks waiting for input. +func resolveParamEdits(edits []ParamEdit, stdin io.Reader, stderr io.Writer, tty bool) ([]ParamEdit, error) { + resolved := append([]ParamEdit(nil), edits...) + for i := range resolved { + if !resolved[i].Secret { + continue + } + e := &resolved[i] + envName := secretEnvName(e.Recipe, e.Param) + if value, ok := os.LookupEnv(envName); ok && value != "" { + e.Value = value + continue + } + if !tty { + return nil, fmt.Errorf("--secret %s.%s: set %s or run without --json", e.Recipe, e.Param, envName) + } + fmt.Fprintf(stderr, "%s.%s: ", e.Recipe, e.Param) + line, err := bufio.NewReader(stdin).ReadString('\n') + if err != nil && line == "" { + return nil, fmt.Errorf("--secret %s.%s: %w", e.Recipe, e.Param, err) + } + e.Value = strings.TrimRight(line, "\r\n") + if e.Value == "" { + return nil, fmt.Errorf("--secret %s.%s: no value given", e.Recipe, e.Param) + } + } + return resolved, nil +} + +func secretEnvName(recipe, param string) string { + return "STOAT_SECRET_" + strings.ToUpper(recipe) + "_" + strings.ToUpper(param) +} + +// streamIsTTY identifies the process terminal without requiring callers to +// pass an os.File. Test callers use an ordinary io.Reader and therefore take +// the non-interactive environment path. +func streamIsTTY(r io.Reader) bool { + f, ok := r.(*os.File) + if !ok { + return false + } + info, err := f.Stat() + return err == nil && info.Mode()&os.ModeCharDevice != 0 +} + +// paramMaps splits parsed edits into the storage shapes core accepts. +func paramMaps(edits []ParamEdit) (set map[string]map[string]string, unset map[string][]string, secrets config.Secrets) { + set = map[string]map[string]string{} + unset = map[string][]string{} + secrets = config.Secrets{} + for _, edit := range edits { + switch { + case edit.Unset: + unset[edit.Recipe] = append(unset[edit.Recipe], edit.Param) + case edit.Secret: + if secrets[edit.Recipe] == nil { + secrets[edit.Recipe] = map[string]string{} + } + secrets[edit.Recipe][edit.Param] = edit.Value + default: + if set[edit.Recipe] == nil { + set[edit.Recipe] = map[string]string{} + } + set[edit.Recipe][edit.Param] = edit.Value + } + } + return set, unset, secrets +} diff --git a/internal/cli/paramflags_test.go b/internal/cli/paramflags_test.go new file mode 100644 index 00000000..a841e137 --- /dev/null +++ b/internal/cli/paramflags_test.go @@ -0,0 +1,111 @@ +package cli + +import ( + "bytes" + "os" + "path/filepath" + "reflect" + "strings" + "testing" + + "github.com/novusedge/stoat/internal/config" +) + +func TestParseParamFlagsStaysPure(t *testing.T) { + t.Setenv("STOAT_SECRET_PARAMDEMO_AUTHKEY", "must-not-be-read-during-parse") + a, err := Parse([]string{ + "create", "work", "--image", "alpine", "--set", "paramdemo.user=dev", + "--secret", "paramdemo.authkey", + }) + if err != nil { + t.Fatal(err) + } + want := []ParamEdit{ + {Recipe: "paramdemo", Param: "user", Value: "dev"}, + {Recipe: "paramdemo", Param: "authkey", Secret: true}, + } + if !reflect.DeepEqual(a.Params, want) { + t.Errorf("Params = %+v, want %+v; parsing must not resolve the secret", a.Params, want) + } +} + +func TestParseUnsetParamFlag(t *testing.T) { + a, err := Parse([]string{"update", "work", "--unset", "paramdemo.user"}) + if err != nil { + t.Fatal(err) + } + want := []ParamEdit{{Recipe: "paramdemo", Param: "user", Unset: true}} + if !reflect.DeepEqual(a.Params, want) { + t.Errorf("Params = %+v, want %+v", a.Params, want) + } + if !containsString(a.Changed, "params") { + t.Errorf("Changed = %v, want params", a.Changed) + } +} + +func TestCLICreateAndUpdatePersistsParamEdits(t *testing.T) { + dir := cliRoot(t) + if err := os.MkdirAll(filepath.Join(dir, "isos"), 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(dir, "isos", "alpine-virt-3.24.1-x86_64.iso"), []byte("iso"), 0o644); err != nil { + t.Fatal(err) + } + recipeDir := filepath.Join(dir, "recipes", "paramdemo") + if err := os.MkdirAll(recipeDir, 0o755); err != nil { + t.Fatal(err) + } + manifest := "schema = 3\nname = \"paramdemo\"\nscript = \"install.sh\"\n" + + "[params.user]\ntype = \"string\"\ndefault = \"dev\"\n" + + "[params.authkey]\ntype = \"secret\"\nrequired = true\n" + if err := os.WriteFile(filepath.Join(recipeDir, "recipe.toml"), []byte(manifest), 0o644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(recipeDir, "install.sh"), []byte("#!/bin/sh\n"), 0o755); err != nil { + t.Fatal(err) + } + + const secret = "cli-secret-value" + t.Setenv("STOAT_SECRET_PARAMDEMO_AUTHKEY", secret) + var out, errOut bytes.Buffer + if code := Main([]string{ + "--json", "create", "work", "--image", "alpine-virt-3.24.1-x86_64.iso", + "--recipes", "paramdemo", "--set", "paramdemo.user=alice", "--secret", "paramdemo.authkey", + }, "test", strings.NewReader(""), &out, &errOut); code != ExitOK { + t.Fatalf("create exit %d: stdout=%s stderr=%s", code, out.String(), errOut.String()) + } + if strings.Contains(out.String()+errOut.String(), secret) { + t.Fatal("secret appeared in create output") + } + + out.Reset() + errOut.Reset() + if code := Main([]string{ + "--json", "update", "work", "--set", "paramdemo.user=bob", "--unset", "paramdemo.authkey", + }, "test", strings.NewReader(""), &out, &errOut); code != ExitOK { + t.Fatalf("update exit %d: stdout=%s stderr=%s", code, out.String(), errOut.String()) + } + v, err := config.Load("work") + if err != nil { + t.Fatal(err) + } + if got, ok := v.Param("paramdemo", "user"); !ok || got != "bob" { + t.Errorf("user = %q/%v, want bob/true", got, ok) + } + secrets, err := config.LoadSecrets(v.Dir) + if err != nil { + t.Fatal(err) + } + if _, ok := secrets["paramdemo"]["authkey"]; ok { + t.Error("--unset did not remove the secret value") + } +} + +func containsString(values []string, want string) bool { + for _, value := range values { + if value == want { + return true + } + } + return false +} diff --git a/internal/cli/run_access.go b/internal/cli/run_access.go index 3d587d06..0d610f09 100644 --- a/internal/cli/run_access.go +++ b/internal/cli/run_access.go @@ -7,6 +7,7 @@ import ( "io" "os" "os/exec" + "sort" "strings" "syscall" "time" @@ -158,6 +159,84 @@ func (w *jsonLogWriter) line(s string) { _ = w.em.Event(wire.TypeLog, w.cmd, map[string]any{"line": s}) } +// secretRedactor sits in front of every CLI apply-log reader. The log can be +// written in chunks that split a secret, so keeping a short suffix is needed +// in addition to replacing complete chunks. +type secretRedactor struct { + out io.Writer + values []string + pending string + keep int +} + +func newSecretRedactor(dir string, out io.Writer) (*secretRedactor, error) { + secrets, err := config.LoadSecrets(dir) + if err != nil { + return nil, err + } + values := make([]string, 0) + for _, recipe := range secrets { + for _, value := range recipe { + if value != "" { + values = append(values, value) + } + } + } + sort.Slice(values, func(i, j int) bool { return len(values[i]) > len(values[j]) }) + keep := 0 + for _, value := range values { + if len(value) > keep { + keep = len(value) + } + } + return &secretRedactor{out: out, values: values, keep: keep}, nil +} + +func (r *secretRedactor) Write(p []byte) (int, error) { + data := r.pending + string(p) + cut := len(data) - r.keep + if cut > 0 { + for _, secret := range r.values { + for start := strings.Index(data, secret); start >= 0; { + end := start + len(secret) + if start < cut && end > cut { + cut = start + } + next := strings.Index(data[start+1:], secret) + if next < 0 { + break + } + start += next + 1 + } + } + } + if cut < 0 { + cut = 0 + } + if err := r.writeRedacted(data[:cut]); err != nil { + return 0, err + } + r.pending = data[cut:] + return len(p), nil +} + +func (r *secretRedactor) Flush() error { + if r.pending == "" { + return nil + } + err := r.writeRedacted(r.pending) + r.pending = "" + return err +} + +func (r *secretRedactor) writeRedacted(value string) error { + for _, secret := range r.values { + value = strings.ReplaceAll(value, secret, "") + } + _, err := io.WriteString(r.out, value) + return err +} + // streamFile copies newly-appended bytes of path to out every tick until // done fires, then does one final copy so nothing written just before // completion is missed. diff --git a/internal/cli/run_apply.go b/internal/cli/run_apply.go index 26c66f89..a12d9287 100644 --- a/internal/cli/run_apply.go +++ b/internal/cli/run_apply.go @@ -37,9 +37,6 @@ func runApply(a *Args, stdout, stderr io.Writer) int { fmt.Fprintf(stdout, "applying recipes to %s...\n", a.VM) } - done := make(chan error, 1) - go func() { done <- core.Apply(context.Background(), a.VM, core.ApplyOpts{Only: a.Only}) }() - // Under --json, raw log bytes must not reach stdout: they would sit // inside the JSON Lines stream and break every consumer's parse. Each // appended line becomes a "log" event instead. @@ -49,7 +46,17 @@ func runApply(a *Args, stdout, stderr io.Writer) int { lw = &jsonLogWriter{em: wire.NewEmitter(stdout), cmd: a.Cmd} out = lw } - aerr := streamFile(v.Paths.ApplyLog, out, done) + redactor, err := newSecretRedactor(v.Paths.Dir, out) + if err != nil { + return a.fail(stdout, stderr, err) + } + + done := make(chan error, 1) + go func() { done <- core.Apply(context.Background(), a.VM, core.ApplyOpts{Only: a.Only}) }() + aerr := streamFile(v.Paths.ApplyLog, redactor, done) + if redactorErr := redactor.Flush(); aerr == nil && redactorErr != nil { + aerr = redactorErr + } if lw != nil { lw.Flush() } diff --git a/internal/cli/run_apply_test.go b/internal/cli/run_apply_test.go new file mode 100644 index 00000000..0682ca5b --- /dev/null +++ b/internal/cli/run_apply_test.go @@ -0,0 +1,227 @@ +package cli + +import ( + "encoding/json" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/novusedge/stoat/internal/config" + "github.com/novusedge/stoat/internal/core" + "github.com/novusedge/stoat/internal/recipes" + "github.com/novusedge/stoat/internal/testutil" +) + +// TestApplyJSONRedactsSecretsAcrossStreamChunks exercises the apply command's +// real log tail. The stored secret is split at the stream buffer boundary and +// the final line has no newline, so both the redactor and JSON line flusher +// must preserve the tail without exposing the secret. +func TestApplyJSONRedactsSecretsAcrossStreamChunks(t *testing.T) { + dir := cliRoot(t) + const ( + recipe = "stream-redaction-caller" + secret = "abc" + trailer = "last-line-without-newline" + ) + recipeDir := filepath.Join(dir, "recipes", recipe) + if err := os.MkdirAll(recipeDir, 0o755); err != nil { + t.Fatal(err) + } + manifest := `schema = 3 +name = "stream-redaction-caller" +version = "1.0.0" +os = ["alpine"] +script = "install.sh" +run = "once" + +[params.token] +type = "secret" +required = true +` + if err := os.WriteFile(filepath.Join(recipeDir, "recipe.toml"), []byte(manifest), 0o644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(recipeDir, "install.sh"), []byte("#!/bin/sh\n"), 0o755); err != nil { + t.Fatal(err) + } + + v := &config.VM{ + Name: "stream-redaction-vm", + OS: "alpine", + Mode: "live", + Backend: "apkovl", + RAM: 1024, + CPUs: 1, + SSHPort: 2200, + Recipes: []string{recipe}, + } + if err := v.Save(); err != nil { + t.Fatal(err) + } + if err := config.SaveSecrets(v.Dir, config.Secrets{recipe: {"token": secret}}); err != nil { + t.Fatal(err) + } + + hash, err := recipes.RecipeHash(recipe, v.OS, nil, []string{"token"}) + if err != nil { + t.Fatal(err) + } + v.Applied = map[string]config.AppliedRecipe{ + recipe: {Version: "1.0.0", Hash: hash}, + } + if err := v.Save(); err != nil { + t.Fatal(err) + } + plan, err := core.PlanApply(v.Name, core.ApplyOpts{}) + if err != nil { + t.Fatalf("PlanApply: %v", err) + } + if len(plan) != 1 || plan[0].Action != "skip" || plan[0].Reason != "already applied" { + t.Fatalf("plan = %+v, want one already-applied skip", plan) + } + + // 32 KiB is the io.Copy buffer used by the apply log tail. Ending the + // prefix four bytes before it puts the whole secret line at the end of + // the first write, where the redactor's retained suffix is exercised. + log := strings.Repeat("P", 32764) + secret + "\n" + trailer + if err := os.WriteFile(v.ProvisionLogPath(), []byte(log), 0o644); err != nil { + t.Fatal(err) + } + stop := testutil.FakeRunning(t, v.Dir) + defer stop() + + code, objs := runJSON(t, "apply", v.Name) + if code != ExitOK { + t.Fatalf("apply exit = %d, want %d: %v", code, ExitOK, objs) + } + raw, err := json.Marshal(objs) + if err != nil { + t.Fatal(err) + } + if strings.Contains(string(raw), secret) { + t.Fatalf("apply output leaked stored secret %q: %s", secret, raw) + } + + foundRedacted, foundTrailer := false, false + for _, obj := range objs { + if obj["type"] != "log" { + continue + } + data, _ := obj["data"].(map[string]any) + line, _ := data["line"].(string) + if strings.Contains(line, "") { + foundRedacted = true + } + if strings.Contains(line, trailer) { + foundTrailer = true + } + } + if !foundRedacted { + t.Errorf("apply log has no redaction marker: %v", objs) + } + if !foundTrailer { + t.Errorf("apply log dropped trailing unterminated line %q: %v", trailer, objs) + } +} + +// The static source contains one intact secret whose first eight bytes are in +// io.Copy's first 32 KiB write and whose remaining bytes are in the next. +// This exercises the actual Main --json apply caller boundary while retaining +// the final unterminated tail assertion. +func TestApplyStreamRedactsSecretAcrossSourceWrites(t *testing.T) { + dir := cliRoot(t) + const ( + recipe = "stream-cross-write-caller" + secret = "Zq7X9pL2-SECRET" + tail = "unterminated-tail-ALPHA" + ) + recipeDir := filepath.Join(dir, "recipes", recipe) + if err := os.MkdirAll(recipeDir, 0o755); err != nil { + t.Fatal(err) + } + manifest := `schema = 3 +name = "stream-cross-write-caller" +os = ["alpine"] +script = "install.sh" + +[params.token] +type = "secret" +required = true +` + if err := os.WriteFile(filepath.Join(recipeDir, "recipe.toml"), []byte(manifest), 0o644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(recipeDir, "install.sh"), []byte("#!/bin/sh\n"), 0o755); err != nil { + t.Fatal(err) + } + v := &config.VM{Name: "stream-cross-write-vm", OS: "alpine", Mode: "live", Backend: "apkovl", RAM: 1024, CPUs: 1, SSHPort: 2200, Recipes: []string{recipe}} + if err := v.Save(); err != nil { + t.Fatal(err) + } + if err := config.SaveSecrets(v.Dir, config.Secrets{recipe: {"token": secret}}); err != nil { + t.Fatal(err) + } + hash, err := recipes.RecipeHash(recipe, v.OS, nil, []string{"token"}) + if err != nil { + t.Fatal(err) + } + v.Applied = map[string]config.AppliedRecipe{recipe: {Version: "1.0.0", Hash: hash}} + if err := v.Save(); err != nil { + t.Fatal(err) + } + plan, err := core.PlanApply(v.Name, core.ApplyOpts{}) + if err != nil { + t.Fatalf("PlanApply: %v", err) + } + if len(plan) != 1 || plan[0].Action != "skip" || plan[0].Reason != "already applied" { + t.Fatalf("plan = %+v, want one already-applied skip", plan) + } + + logPath := v.ProvisionLogPath() + prefix := strings.Repeat("P", 32760) + log := prefix + secret + "\n" + tail + if !strings.HasPrefix(log[32760:], secret) || !strings.HasPrefix(log[32768:], secret[8:]) { + t.Fatalf("fixture secret does not cross the 32 KiB source boundary: offset=%d", strings.Index(log, secret)) + } + for _, fragment := range []string{secret[:8], secret[8:]} { + if strings.Contains(tail, fragment) { + t.Fatalf("fixture fragment %q overlaps preserved tail %q", fragment, tail) + } + } + if err := os.WriteFile(logPath, []byte(log), 0o644); err != nil { + t.Fatal(err) + } + stop := testutil.FakeRunning(t, v.Dir) + defer stop() + + code, objs := runJSON(t, "apply", v.Name) + if code != ExitOK { + t.Fatalf("apply exit = %d, want %d: %v", code, ExitOK, objs) + } + foundRedacted, foundTail := false, false + for _, obj := range objs { + if obj["type"] != "log" { + continue + } + data, _ := obj["data"].(map[string]any) + line, _ := data["line"].(string) + for _, fragment := range []string{secret, secret[:8], secret[8:]} { + if strings.Contains(line, fragment) { + t.Fatalf("apply log event leaked secret fragment %q: %q", fragment, line) + } + } + if strings.Contains(line, tail) { + foundTail = true + } + if strings.Contains(line, "") { + foundRedacted = true + } + } + if !foundRedacted { + t.Fatalf("apply output has no redaction marker: %v", objs) + } + if !foundTail { + t.Fatalf("apply output dropped unterminated tail %q: %v", tail, objs) + } +} diff --git a/internal/cli/run_get.go b/internal/cli/run_get.go index c325b260..17b02081 100644 --- a/internal/cli/run_get.go +++ b/internal/cli/run_get.go @@ -3,6 +3,7 @@ package cli import ( "fmt" "io" + "sort" "strings" "github.com/novusedge/stoat/internal/cli/wire" @@ -15,7 +16,7 @@ func runGet(a *Args, stdout, stderr io.Writer) int { return a.fail(stdout, stderr, err) } if a.JSON { - return a.ok(stdout, map[string]any{"vm": wire.FromVM(v, core.GraphicalSession())}) + return a.ok(stdout, wire.VMStatusResult{VM: wire.FromVMStatus(v, core.GraphicalSession())}) } fmt.Fprintf(stdout, "name: %s\n", v.Name) fmt.Fprintf(stdout, "os: %s\n", v.OS) @@ -29,6 +30,19 @@ func runGet(a *Args, stdout, stderr io.Writer) int { fmt.Fprintf(stdout, "ssh port: %d\n", v.SSHPort) fmt.Fprintf(stdout, "ssh user: %s\n", v.SSHUser) fmt.Fprintf(stdout, "recipes: %s\n", strings.Join(v.Recipes, ", ")) + for _, state := range v.RecipeStates { + status := "pending" + if state.Applied { + status = "applied " + state.Version + } + fmt.Fprintf(stdout, " %-14s %-18s health %s\n", state.Name, status, state.Health) + for _, name := range sortedKeys(state.Params) { + fmt.Fprintf(stdout, " param %-12s %s\n", name, state.Params[name]) + } + for _, name := range sortedKeys(state.Outputs) { + fmt.Fprintf(stdout, " out %-12s %s\n", name, state.Outputs[name]) + } + } forwards := make([]string, len(v.Forwards)) for i, f := range v.Forwards { forwards[i] = fmt.Sprintf("%d:%d", f.HostPort, f.GuestPort) @@ -44,6 +58,15 @@ func runGet(a *Args, stdout, stderr io.Writer) int { return ExitOK } +func sortedKeys(values map[string]string) []string { + keys := make([]string, 0, len(values)) + for key := range values { + keys = append(keys, key) + } + sort.Strings(keys) + return keys +} + func runSSHCommand(a *Args, stdout, stderr io.Writer) int { argv, err := core.SSHCommand(a.VM) if err != nil { diff --git a/internal/cli/run_misc.go b/internal/cli/run_misc.go index b351e9a7..829bf774 100644 --- a/internal/cli/run_misc.go +++ b/internal/cli/run_misc.go @@ -164,6 +164,9 @@ func runRecipe(a *Args, stdout, stderr io.Writer) int { fmt.Fprintln(stdout, "edit it, then pick it in the new-vm form for a matching vm") } return ExitOK + + case "show": + return runRecipeShow(a, stdout, stderr) } // Unreachable: Parse rejects any action but list/new. if a.JSON { diff --git a/internal/cli/run_recipe_show.go b/internal/cli/run_recipe_show.go new file mode 100644 index 00000000..c84f5259 --- /dev/null +++ b/internal/cli/run_recipe_show.go @@ -0,0 +1,50 @@ +package cli + +import ( + "fmt" + "io" + "strings" + + "github.com/novusedge/stoat/internal/cli/wire" + "github.com/novusedge/stoat/internal/core" +) + +// runRecipeShow prints one recipe's contract without filtering it by a VM's +// operating system. A caller reads this before choosing an apply target. +func runRecipeShow(a *Args, stdout, stderr io.Writer) int { + r, err := core.RecipeShow(a.VM) + if err != nil { + return a.fail(stdout, stderr, err) + } + if a.JSON { + return a.ok(stdout, wire.RecipeShowResult{Recipe: wire.FromRecipeSchema(r)}) + } + + fmt.Fprintf(stdout, "%s: %s\n", r.Name, r.Description) + fmt.Fprintf(stdout, "schema: %d\nruntime: %s\n", r.Schema, r.Runtime) + if len(r.Params) > 0 { + fmt.Fprintln(stdout, "\nparams:") + for _, p := range r.Params { + detail := p.Type + switch { + case p.Type == "enum": + detail = "enum(" + strings.Join(p.Values, ", ") + ")" + case p.Required: + detail += ", required" + case p.Default != "": + detail += ", default " + p.Default + } + fmt.Fprintf(stdout, " %-14s %-28s %s\n", p.Name, detail, p.Help) + } + } + if len(r.Outputs) > 0 { + fmt.Fprintln(stdout, "\noutputs:") + for _, o := range r.Outputs { + fmt.Fprintf(stdout, " %-14s %s\n", o.Name, o.Help) + } + } + if r.Health != nil { + fmt.Fprintf(stdout, "\nhealth: %s (timeout %s)\n", r.Health.Check, r.Health.Timeout) + } + return ExitOK +} diff --git a/internal/cli/run_update.go b/internal/cli/run_update.go index a652322e..3ac1500c 100644 --- a/internal/cli/run_update.go +++ b/internal/cli/run_update.go @@ -16,6 +16,10 @@ import ( // requested edits core accepted, and applies_at because most of them are // written to vm.toml now and only take effect at the VM's next start. func runUpdate(a *Args, stdout, stderr io.Writer) int { + set, unset, secrets := paramMaps(a.Params) + a.Patch.SetParams = set + a.Patch.UnsetParams = unset + a.Patch.Secrets = secrets v, err := core.Update(a.VM, a.Patch) if err != nil { return a.fail(stdout, stderr, err) diff --git a/internal/cli/run_vm.go b/internal/cli/run_vm.go index b4d76061..649c4179 100644 --- a/internal/cli/run_vm.go +++ b/internal/cli/run_vm.go @@ -146,9 +146,14 @@ func afterStart(a *Args, v core.VM, stdout, stderr io.Writer) int { if !a.Quiet { fmt.Fprintf(stdout, "applying recipes to %s...\n", a.VM) } + redactor, err := newSecretRedactor(v.Paths.Dir, stdout) + if err != nil { + return a.fail(stdout, stderr, err) + } done := make(chan error, 1) go func() { done <- core.Apply(context.Background(), a.VM, core.ApplyOpts{}) }() - if err := streamFile(v.Paths.ApplyLog, stdout, done); err != nil { + if err := streamFile(v.Paths.ApplyLog, redactor, done); err != nil { + _ = redactor.Flush() if errors.Is(err, core.ErrProvisionInProgress) { // A concurrent `apply` already holds the lock; that run owns the // error. `up` still started the VM, so this is not a failure of @@ -158,6 +163,9 @@ func afterStart(a *Args, v core.VM, stdout, stderr io.Writer) int { } return a.fail(stdout, stderr, err) } + if err := redactor.Flush(); err != nil { + return a.fail(stdout, stderr, err) + } fmt.Fprintf(stdout, "%s: recipes applied\n", a.VM) return ExitOK } @@ -275,6 +283,9 @@ func runRM(a *Args, stdin io.Reader, stdout, stderr io.Writer) int { // what came back. There was no create subcommand before core existed, because // everything it needed lived inside the TUI's form. func runCreate(a *Args, stdout, stderr io.Writer) int { + set, _, secrets := paramMaps(a.Params) + a.Spec.Params = set + a.Spec.Secrets = secrets v, err := core.Create(a.Spec) if err != nil { if a.JSON { diff --git a/internal/cli/subcommands_test.go b/internal/cli/subcommands_test.go index 569968d7..997799cb 100644 --- a/internal/cli/subcommands_test.go +++ b/internal/cli/subcommands_test.go @@ -1,6 +1,7 @@ package cli import ( + "encoding/json" "os" "path/filepath" "strings" @@ -143,6 +144,71 @@ func TestLogsVMTailsAndSelectsWhich(t *testing.T) { } } +// Both public VM log selectors pass through the same redaction boundary. Keep +// this at the CLI sink so a correct core reader cannot be bypassed by JSON +// line handling or tailing. +func TestLogsJSONRedactsStoredSecretsForConsoleAndApply(t *testing.T) { + dir := cliRoot(t) + v := saveVM(t, &config.VM{Name: "work", Mode: "live", RAM: 1024, CPUs: 1, SSHPort: 2200}) + const sentinel = "cli-logs-secret-sentinel" + if err := config.SaveSecrets(v.Dir, config.Secrets{"docker": {"authkey": sentinel}}); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(dir, "work", "console.log"), []byte("console "+sentinel+"\n"), 0o644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(dir, "work", "last-provision.log"), []byte("apply "+sentinel+"\n"), 0o644); err != nil { + t.Fatal(err) + } + + for _, which := range []core.Which{core.WhichConsole, core.WhichApply} { + t.Run(string(which), func(t *testing.T) { + code, objs := runJSON(t, "logs", "work", "--which", string(which)) + if code != ExitOK { + t.Fatalf("exit = %d: %v", code, objs) + } + lines, _ := dataOf(t, objs)["lines"].([]any) + if len(lines) != 1 { + t.Fatalf("lines = %#v, want one redacted line", lines) + } + line, _ := lines[0].(string) + if strings.Contains(line, sentinel) { + t.Fatalf("logs %s leak stored secret %q: %q", which, sentinel, line) + } + if !strings.Contains(line, "") { + t.Errorf("logs %s = %q, want the redaction marker", which, line) + } + if _, err := json.Marshal(objs); err != nil { + t.Fatalf("logs %s result is not JSON-marshalable: %v", which, err) + } + }) + } +} + +// The CLI must preserve the secret-store refusal and identify the VM rather +// than falling back to raw console bytes when a reader is insecure. +func TestLogsJSONPropagatesInsecureSecretStoreWithVMContext(t *testing.T) { + cliRoot(t) + v := saveVM(t, &config.VM{Name: "work", Mode: "live", RAM: 1024, CPUs: 1, SSHPort: 2200}) + path := filepath.Join(v.Dir, config.SecretsName) + if err := os.WriteFile(path, []byte("docker.authkey = \"sentinel\"\n"), 0o644); err != nil { + t.Fatal(err) + } + if err := os.Chmod(path, 0o644); err != nil { + t.Fatal(err) + } + + code, objs := runJSON(t, "logs", "work", "--which", "console") + if code != ExitFail { + t.Fatalf("exit = %d, want %d: %v", code, ExitFail, objs) + } + errObj, _ := result(t, objs)["error"].(map[string]any) + message, _ := errObj["message"].(string) + if !strings.Contains(message, "work") || !strings.Contains(message, "secrets.toml: mode 0644") { + t.Errorf("error.message = %q, want VM context and secret-file mode", message) + } +} + // The no-name form must keep tailing stoat's own log, not become a usage // error now that the positional exists. func TestLogsWithNoVMStillTailsStoatsOwnLog(t *testing.T) { diff --git a/internal/cli/update_test.go b/internal/cli/update_test.go index 3cbf8d78..b20fa70e 100644 --- a/internal/cli/update_test.go +++ b/internal/cli/update_test.go @@ -88,6 +88,45 @@ func TestParseUpdateNeedsAtLeastOneFlag(t *testing.T) { } } +func TestParseUpdateRecipeContractFlagsMatchE2EInvocation(t *testing.T) { + a, err := Parse([]string{"update", "work", "--recipes", "xfce,docker,redaction", "--set", "docker.user=dev", "--secret", "redaction.token"}) + if err != nil { + t.Fatal(err) + } + if a.Patch.Recipes == nil || !reflect.DeepEqual(*a.Patch.Recipes, []string{"xfce", "docker", "redaction"}) { + t.Fatalf("Recipes = %v, want comma-separated three-recipe list", a.Patch.Recipes) + } + if len(a.Params) != 2 || a.Params[0].Recipe != "docker" || a.Params[0].Param != "user" || a.Params[1].Recipe != "redaction" || !a.Params[1].Secret { + t.Fatalf("Params = %+v, want non-secret set plus secret target", a.Params) + } +} + +// Keep every command shape used by scripts/e2e.sh reachable through the +// public parser. This catches a stale script flag before a real VM run; it +// deliberately stops at Parse and never starts QEMU or executes a guest. +func TestParseE2ECommandShapes(t *testing.T) { + commands := [][]string{ + {"create", "e2e", "--image", "alpine", "--mode", "disk", "--ram", "2048", "--cpus", "2", "--recipes", "xfce"}, + {"up", "e2e"}, + {"exec", "e2e", "--", "sh", "-c", "test -s /mnt/work/.installed"}, + {"update", "e2e", "--recipes", "xfce,docker,redaction", "--set", "docker.user=dev", "--secret", "redaction.token"}, + {"apply", "e2e"}, + {"wait", "e2e", "--healthy", "--timeout", "90s"}, + {"get", "e2e"}, + {"update", "e2e", "--set", "docker.user=e2e-rerun"}, + {"apply", "e2e", "--dry-run"}, + {"down", "e2e"}, + {"rm", "e2e", "-y"}, + } + for _, argv := range commands { + t.Run(strings.Join(argv, " "), func(t *testing.T) { + if _, err := Parse(argv); err != nil { + t.Fatalf("Parse(%q) = %v", argv, err) + } + }) + } +} + // End to end against a real data root: the field asked for changes, and the // one that was never mentioned survives. This is the regression that a // Parse-only test cannot prove, since it is core.Update that does the write. diff --git a/internal/cli/wait_test.go b/internal/cli/wait_test.go index 274052d0..0b3d018c 100644 --- a/internal/cli/wait_test.go +++ b/internal/cli/wait_test.go @@ -1,10 +1,15 @@ package cli import ( + "net" + "os" + "path/filepath" "testing" + "time" "github.com/novusedge/stoat/internal/cli/wire" "github.com/novusedge/stoat/internal/config" + "github.com/novusedge/stoat/internal/core" ) // TestWaitMissingVM covers the ordinary core.Get-style not_found path: @@ -75,6 +80,10 @@ func TestParseWaitUsageErrors(t *testing.T) { for _, args := range [][]string{ {"wait", "work", "--until", "bogus"}, {"wait", "work", "--timeout", "0"}, + {"wait", "work", "--healthy", "--until", "reachable"}, + {"wait", "work", "--healthy", "--until", "applied"}, + {"wait", "work", "--healthy", "--until", "stopped"}, + {"wait", "work", "--until", "reachable", "--healthy"}, } { if _, err := Parse(args); err == nil { t.Errorf("Parse(%v) accepted, want a usage error", args) @@ -82,6 +91,19 @@ func TestParseWaitUsageErrors(t *testing.T) { } } +func TestParseWaitHealthySelectsTheHealthEvent(t *testing.T) { + a, err := Parse([]string{"wait", "work", "--healthy", "--timeout", "7s"}) + if err != nil { + t.Fatal(err) + } + if a.Until != core.UntilHealthy { + t.Fatalf("Until = %q, want %q", a.Until, core.UntilHealthy) + } + if a.Timeout != 7*time.Second { + t.Fatalf("Timeout = %s, want 7s", a.Timeout) + } +} + func TestWaitUsageErrorsUnderJSON(t *testing.T) { cliRoot(t) for _, args := range [][]string{ @@ -99,3 +121,74 @@ func TestWaitUsageErrorsUnderJSON(t *testing.T) { } } } + +// A health check may have observed a real failing verdict, but a caller's +// shorter JSON timeout still owns the boundary. The result must remain the +// machine-readable timeout rather than exposing the intermediate failure. +func TestWaitHealthyJSONDeadlineAfterObservedFailureIsTimeout(t *testing.T) { + dir := cliRoot(t) + recipeDir := filepath.Join(dir, "recipes", "healthy-timeout") + if err := os.MkdirAll(recipeDir, 0o755); err != nil { + t.Fatal(err) + } + manifest := "schema = 3\nname = \"healthy-timeout\"\nscript = \"install.sh\"\n\n[health]\ncheck = \"docker info\"\ntimeout = \"2s\"\n" + if err := os.WriteFile(filepath.Join(recipeDir, "recipe.toml"), []byte(manifest), 0o644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(recipeDir, "install.sh"), []byte("#!/bin/sh\nexit 0\n"), 0o755); err != nil { + t.Fatal(err) + } + bin := t.TempDir() + sshScript := "#!/bin/sh\ncat >/dev/null\nprintf '%s\\n' 'health still failing' >&2\nexit 1\n" + if err := os.WriteFile(filepath.Join(bin, "ssh"), []byte(sshScript), 0o755); err != nil { + t.Fatal(err) + } + t.Setenv("PATH", bin+string(os.PathListSeparator)+os.Getenv("PATH")) + port, stopSSH := cliFakeSSHD(t) + defer stopSSH() + v := &config.VM{ + Name: "work", Mode: "live", OS: "alpine", RAM: 1024, CPUs: 1, + SSHPort: port, Recipes: []string{"healthy-timeout"}, + Applied: map[string]config.AppliedRecipe{"healthy-timeout": {}}, + } + if err := v.Save(); err != nil { + t.Fatal(err) + } + defer fakeRunning(t, v)() + + code, objs := runJSON(t, "wait", "work", "--healthy", "--timeout", "100ms") + if code != ExitFail { + t.Fatalf("exit = %d, want %d", code, ExitFail) + } + res := result(t, objs) + errObj, _ := res["error"].(map[string]any) + if errObj["code"] != string(wire.CodeTimeout) { + t.Fatalf("error.code = %v, want %q after observed health failure", errObj["code"], wire.CodeTimeout) + } +} + +func cliFakeSSHD(t *testing.T) (int, func()) { + t.Helper() + l, err := net.Listen("tcp", "127.0.0.1:0") + if err != nil { + t.Fatal(err) + } + done := make(chan struct{}) + go func() { + for { + conn, acceptErr := l.Accept() + if acceptErr != nil { + return + } + go func() { + _, _ = conn.Write([]byte("SSH-2.0-fake\r\n")) + <-done + _ = conn.Close() + }() + } + }() + return l.Addr().(*net.TCPAddr).Port, func() { + close(done) + _ = l.Close() + } +} diff --git a/internal/cli/wire/dto.go b/internal/cli/wire/dto.go index b37a57fc..5eb87ba3 100644 --- a/internal/cli/wire/dto.go +++ b/internal/cli/wire/dto.go @@ -2,6 +2,8 @@ package wire import ( "encoding/base64" + "sort" + "time" "unicode/utf8" "github.com/novusedge/stoat/internal/core" @@ -99,6 +101,72 @@ type VM struct { Error string `json:"error,omitempty"` } +// RecipeState is one recipe's redacted per-VM state. +type RecipeState struct { + Name string `json:"name"` + Applied bool `json:"applied"` + Version string `json:"version"` + At string `json:"at"` + Health string `json:"health"` + Params map[string]string `json:"params"` + Outputs map[string]string `json:"outputs"` +} + +// VMStatus is the get/vm_status payload. RecipeStates is additive so the +// existing VM.recipes string list remains compatible with contract v2. +type VMStatus struct { + VM + Health string `json:"health"` + RecipeStates []RecipeState `json:"recipes_detail"` +} + +// VMStatusResult is the named result for `get --json`. +type VMStatusResult struct { + VM VMStatus `json:"vm"` +} + +// FromVMStatus converts the stored VM status into its additive wire shape. +func FromVMStatus(v core.VM, graphical bool) VMStatus { + health := string(v.Health) + if health == "" { + health = string(core.HealthUnknown) + } + out := VMStatus{VM: FromVM(v, graphical), Health: health, RecipeStates: []RecipeState{}} + for _, state := range v.RecipeStates { + params := nonNilMap(state.Params) + redacted := make(map[string]string, len(params)+len(state.SecretNames)) + for name, value := range params { + redacted[name] = value + } + for _, name := range state.SecretNames { + if redacted[name] != core.SecretUnset { + redacted[name] = core.SecretSet + } + } + at := "" + if !state.At.IsZero() { + at = state.At.UTC().Format(time.RFC3339) + } + stateHealth := state.Health + if stateHealth == "" { + stateHealth = string(core.HealthUnknown) + } + out.RecipeStates = append(out.RecipeStates, RecipeState{ + Name: state.Name, Applied: state.Applied, Version: state.Version, + At: at, Health: stateHealth, Params: redacted, + Outputs: nonNilMap(state.Outputs), + }) + } + return out +} + +func nonNilMap(m map[string]string) map[string]string { + if m == nil { + return map[string]string{} + } + return m +} + // FromVM takes graphical (core.GraphicalSession) rather than calling it, // keeping this constructor pure: FromVMs would otherwise re-answer a // host-wide question once per VM in the list, and a test of this file would @@ -264,23 +332,124 @@ func FromPruneItems(ps []core.PruneItem) []PruneItem { // until reachable" after an apply answers about the guest before or after the // restart. type Recipe struct { - Name string `json:"name"` - Description string `json:"description"` - Reboot bool `json:"reboot"` - Depends []string `json:"depends"` - Runtime string `json:"runtime"` + Name string `json:"name"` + Description string `json:"description"` + Schema int `json:"schema"` + Params []RecipeParam `json:"params"` + Outputs []RecipeOutput `json:"outputs"` + Health *RecipeHealth `json:"health"` + Reboot bool `json:"reboot"` + Depends []string `json:"depends"` + Runtime string `json:"runtime"` +} + +// RecipeParam is one named recipe parameter in a machine-readable schema. +type RecipeParam struct { + Name string `json:"name"` + Type string `json:"type"` + Required bool `json:"required"` + Default string `json:"default"` + Values []string `json:"values"` + Help string `json:"help"` +} + +// RecipeOutput is one named recipe output in a machine-readable schema. +type RecipeOutput struct { + Name string `json:"name"` + Help string `json:"help"` +} + +// RecipeHealth is a recipe's declared health check. +type RecipeHealth struct { + Check string `json:"check"` + Timeout string `json:"timeout"` +} + +// RecipeSchema is one recipe's machine-readable contract. +type RecipeSchema struct { + Name string `json:"name"` + Description string `json:"description"` + Schema int `json:"schema"` + Runtime string `json:"runtime"` + Reboot bool `json:"reboot"` + Depends []string `json:"depends"` + Params []RecipeParam `json:"params"` + Outputs []RecipeOutput `json:"outputs"` + Health *RecipeHealth `json:"health"` +} + +// RecipeShowResult is the named JSON envelope for `recipe show`. +type RecipeShowResult struct { + Recipe RecipeSchema `json:"recipe"` +} + +// FromRecipeSchema converts a core recipe contract to the named wire shape. +// Params and outputs remain sorted named lists so repeated calls are stable. +func FromRecipeSchema(r core.Recipe) RecipeSchema { + s := RecipeSchema{ + Name: r.Name, Description: r.Description, Schema: r.Schema, + Runtime: r.Runtime, Reboot: r.Reboot, Depends: nonNil(r.Depends), + Params: []RecipeParam{}, Outputs: []RecipeOutput{}, + } + for _, p := range r.Params { + s.Params = append(s.Params, RecipeParam{ + Name: p.Name, Type: p.Type, Required: p.Required, + Default: p.Default, Values: nonNil(p.Values), Help: p.Help, + }) + } + for _, o := range r.Outputs { + s.Outputs = append(s.Outputs, RecipeOutput{Name: o.Name, Help: o.Help}) + } + sort.Slice(s.Params, func(i, j int) bool { return s.Params[i].Name < s.Params[j].Name }) + sort.Slice(s.Outputs, func(i, j int) bool { return s.Outputs[i].Name < s.Outputs[j].Name }) + if r.Health != nil { + s.Health = &RecipeHealth{Check: r.Health.Check, Timeout: r.Health.Timeout} + } + return s } func FromRecipe(r core.Recipe) Recipe { return Recipe{ Name: r.Name, Description: r.Description, + Schema: r.Schema, + Params: fromRecipeParams(r.Params), + Outputs: fromRecipeOutputs(r.Outputs), + Health: fromRecipeHealth(r.Health), Reboot: r.Reboot, Depends: nonNil(r.Depends), Runtime: r.Runtime, } } +func fromRecipeParams(params []core.RecipeParam) []RecipeParam { + out := make([]RecipeParam, 0, len(params)) + for _, p := range params { + out = append(out, RecipeParam{ + Name: p.Name, Type: p.Type, Required: p.Required, + Default: p.Default, Values: nonNil(p.Values), Help: p.Help, + }) + } + sort.Slice(out, func(i, j int) bool { return out[i].Name < out[j].Name }) + return nonNil(out) +} + +func fromRecipeOutputs(outputs []core.RecipeOutput) []RecipeOutput { + out := make([]RecipeOutput, 0, len(outputs)) + for _, o := range outputs { + out = append(out, RecipeOutput{Name: o.Name, Help: o.Help}) + } + sort.Slice(out, func(i, j int) bool { return out[i].Name < out[j].Name }) + return nonNil(out) +} + +func fromRecipeHealth(health *core.RecipeHealthSpec) *RecipeHealth { + if health == nil { + return nil + } + return &RecipeHealth{Check: health.Check, Timeout: health.Timeout} +} + func FromRecipes(rs []core.Recipe) []Recipe { out := make([]Recipe, len(rs)) for i, r := range rs { diff --git a/internal/cli/wire/dto_test.go b/internal/cli/wire/dto_test.go index e9244634..384915d8 100644 --- a/internal/cli/wire/dto_test.go +++ b/internal/cli/wire/dto_test.go @@ -127,7 +127,7 @@ func TestRecipeGolden(t *testing.T) { Runtime: "sh", } got := marshal(t, FromRecipe(r)) - want := `{"name":"xfce","description":"XFCE desktop environment","reboot":true,"depends":["devtools"],"runtime":"sh"}` + want := `{"name":"xfce","description":"XFCE desktop environment","schema":0,"params":[],"outputs":[],"health":null,"reboot":true,"depends":["devtools"],"runtime":"sh"}` if got != want { t.Errorf("got %s\nwant %s", got, want) } @@ -138,12 +138,78 @@ func TestRecipeGolden(t *testing.T) { // on null. func TestRecipeNilDependsIsEmptyList(t *testing.T) { got := marshal(t, FromRecipe(core.Recipe{Name: "xfce"})) - want := `{"name":"xfce","description":"","reboot":false,"depends":[],"runtime":""}` + want := `{"name":"xfce","description":"","schema":0,"params":[],"outputs":[],"health":null,"reboot":false,"depends":[],"runtime":""}` if got != want { t.Errorf("got %s\nwant %s", got, want) } } +func TestFromRecipeSchemaShape(t *testing.T) { + got := FromRecipeSchema(core.Recipe{ + Name: "docker", Description: "Docker engine", Schema: 3, Runtime: "sh", + Params: []core.RecipeParam{ + {Name: "channel", Type: "enum", Default: "stable", Values: []string{"stable", "test"}}, + {Name: "authkey", Type: "secret", Required: true}, + }, + Outputs: []core.RecipeOutput{{Name: "socket", Help: "path of the docker socket"}}, + Health: &core.RecipeHealthSpec{Check: "docker info", Timeout: "30s"}, + }) + b, err := json.Marshal(got) + if err != nil { + t.Fatal(err) + } + want := `{"name":"docker","description":"Docker engine","schema":3,"runtime":"sh","reboot":false,"depends":[],"params":[` + + `{"name":"authkey","type":"secret","required":true,"default":"","values":[],"help":""},` + + `{"name":"channel","type":"enum","required":false,"default":"stable","values":["stable","test"],"help":""}],` + + `"outputs":[{"name":"socket","help":"path of the docker socket"}],` + + `"health":{"check":"docker info","timeout":"30s"}}` + if string(b) != want { + t.Errorf("got %s\nwant %s", b, want) + } +} + +// A recipe without a health check emits null, never an empty object: a +// consumer must distinguish "no check declared" from a blank check. +func TestFromRecipeSchemaNullHealth(t *testing.T) { + b, err := json.Marshal(FromRecipeSchema(core.Recipe{Name: "xfce", Schema: 2})) + if err != nil { + t.Fatal(err) + } + if !strings.Contains(string(b), `"health":null`) { + t.Errorf("got %s", b) + } +} + +// List and show are two views of one projection. Their shared recipe fields +// must agree, including sorted, non-null parameter and output lists. +func TestRecipeListAndShowProjectionAgree(t *testing.T) { + r := core.Recipe{ + Name: "docker", Description: "Docker", Schema: 3, Runtime: "sh", + Depends: []string{"base"}, + Params: []core.RecipeParam{{Name: "user", Type: "string", Default: "dev"}}, + Outputs: []core.RecipeOutput{{Name: "socket", Help: "socket"}}, + Health: &core.RecipeHealthSpec{Check: "docker info", Timeout: "30s"}, + } + list := FromRecipe(r) + show := FromRecipeSchema(r) + if list.Name != show.Name || list.Description != show.Description || list.Schema != show.Schema || list.Runtime != show.Runtime || list.Reboot != show.Reboot { + t.Fatalf("list=%+v show=%+v disagree on shared recipe fields", list, show) + } + listParams, _ := json.Marshal(list.Params) + showParams, _ := json.Marshal(show.Params) + if string(listParams) != string(showParams) { + t.Fatalf("list and show params disagree: list=%s show=%s", listParams, showParams) + } + listOutputs, _ := json.Marshal(list.Outputs) + showOutputs, _ := json.Marshal(show.Outputs) + if string(listOutputs) != string(showOutputs) { + t.Fatalf("list and show outputs disagree: list=%s show=%s", listOutputs, showOutputs) + } + if strings.Contains(marshal(t, list), "null") || strings.Contains(marshal(t, show), "null") { + t.Fatalf("recipe projections contain a null list: list=%s show=%s", marshal(t, list), marshal(t, show)) + } +} + func TestApplyPlanGolden(t *testing.T) { p := core.ApplyPlan{Name: "xfce", Action: "run", Reason: "never applied"} got := marshal(t, FromApplyPlan(p)) @@ -224,6 +290,83 @@ func TestEmptySlicesMarshalAsEmptyArrayNeverNull(t *testing.T) { } } +func TestFromVMStatusRedactsSecrets(t *testing.T) { + got := FromVMStatus(core.VM{ + Name: "work", OS: "alpine", Mode: "live", + RecipeStates: []core.RecipeState{{ + Name: "docker", Applied: true, Version: "1.2.0", Health: string(core.HealthOK), + Params: map[string]string{"user": "dev", "authkey": core.SecretSet}, + SecretNames: []string{"authkey"}, + Outputs: map[string]string{"socket": "/var/run/docker.sock"}, + }}, + Health: core.HealthOK, + }, false) + + if len(got.RecipeStates) != 1 { + t.Fatalf("recipe states = %#v, want one redacted state", got.RecipeStates) + } + p := got.RecipeStates[0].Params + if p["authkey"] != "" { + t.Errorf("authkey = %q, want ", p["authkey"]) + } + if p["user"] != "dev" { + t.Errorf("user = %q, want dev", p["user"]) + } +} + +// The wire redactor keys off the manifest's secret-name list, not the value +// core happened to provide: a raw core secret must not cross this boundary. +func TestFromVMStatusRedactsEvenWhenCorePassesARawSecret(t *testing.T) { + got := FromVMStatus(core.VM{ + Name: "work", + RecipeStates: []core.RecipeState{{ + Name: "tailscale", Applied: true, + Params: map[string]string{"authkey": "tskey-SENTINEL"}, SecretNames: []string{"authkey"}, + }}, + }, false) + if len(got.RecipeStates) != 1 { + t.Fatalf("recipe states = %#v, want one redacted state", got.RecipeStates) + } + if v := got.RecipeStates[0].Params["authkey"]; v != "" { + t.Errorf("authkey = %q, want ", v) + } +} + +func TestFromVMStatusPreservesSetAndUnsetSecretMarkers(t *testing.T) { + got := FromVMStatus(core.VM{ + Name: "work", + RecipeStates: []core.RecipeState{{ + Name: "docker", + Params: map[string]string{"set_token": core.SecretSet, "unset_token": core.SecretUnset}, + SecretNames: []string{"set_token", "unset_token"}, + }}, + }, false) + if len(got.RecipeStates) != 1 { + t.Fatalf("recipe states = %#v, want one state", got.RecipeStates) + } + params := got.RecipeStates[0].Params + if params["set_token"] != core.SecretSet || params["unset_token"] != core.SecretUnset { + t.Fatalf("secret markers = %#v, want set/unset markers", params) + } +} + +// Empty status maps marshal as {}, never null: a caller iterating them must +// not branch on a second representation of "no values". +func TestVMStatusEmptyMapsAreObjects(t *testing.T) { + b, err := json.Marshal(FromVMStatus(core.VM{ + Name: "work", RecipeStates: []core.RecipeState{{Name: "xfce"}}, + }, false)) + if err != nil { + t.Fatal(err) + } + if len(FromVMStatus(core.VM{Name: "work"}, false).RecipeStates) != 0 { + t.Fatal("empty recipe states were not normalized") + } + if !strings.Contains(string(b), `"params":{}`) || !strings.Contains(string(b), `"outputs":{}`) { + t.Errorf("got %s", b) + } +} + // --- exec's non-UTF-8 handling (§4) --- func TestExecResultPlainUTF8(t *testing.T) { diff --git a/internal/cloudinit/cloudinit.go b/internal/cloudinit/cloudinit.go index 4df09929..45a33c2b 100644 --- a/internal/cloudinit/cloudinit.go +++ b/internal/cloudinit/cloudinit.go @@ -295,7 +295,10 @@ func Seed(v *config.VM, pubkey string, recipeBodies []string) (string, error) { } seedDir := filepath.Join(v.OvlDir(), "seed") - if err := os.MkdirAll(seedDir, 0o755); err != nil { + if err := os.MkdirAll(seedDir, 0o700); err != nil { + return "", err + } + if err := os.Chmod(seedDir, 0o700); err != nil { return "", err } @@ -303,21 +306,53 @@ func Seed(v *config.VM, pubkey string, recipeBodies []string) (string, error) { if err != nil { return "", err } - if err := os.WriteFile(filepath.Join(seedDir, "user-data"), []byte(ud), 0o644); err != nil { + if err := writePrivateFile(filepath.Join(seedDir, "user-data"), []byte(ud)); err != nil { return "", err } metaData := fmt.Sprintf(metaDataTemplate, "stoat-"+v.Name, v.Name) - if err := os.WriteFile(filepath.Join(seedDir, "meta-data"), []byte(metaData), 0o644); err != nil { + if err := writePrivateFile(filepath.Join(seedDir, "meta-data"), []byte(metaData)); err != nil { return "", err } isoPath := filepath.Join(v.OvlDir(), "seed.iso") - cmd := exec.Command("xorriso", "-as", "mkisofs", "-o", isoPath, "-V", "CIDATA", "-J", "-r", seedDir) + iso, err := os.OpenFile(isoPath, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0o600) + if err != nil { + return "", err + } + if err := iso.Chmod(0o600); err != nil { + _ = iso.Close() + return "", err + } + if err := iso.Close(); err != nil { + return "", err + } + xorrisoArgs := []string{"-as", "mkisofs", "-o", isoPath, "-V", "CIDATA", "-J", "-r", seedDir} + commandArgs := append([]string{"-c", "umask 0077; exec \"$@\"", "stoat-xorriso", "xorriso"}, xorrisoArgs...) + cmd := exec.Command("sh", commandArgs...) out, err := cmd.CombinedOutput() if err != nil { return "", fmt.Errorf("xorriso: %w: %s", err, out) } + if err := os.Chmod(isoPath, 0o600); err != nil { + return "", err + } return isoPath, nil } + +func writePrivateFile(path string, data []byte) error { + f, err := os.OpenFile(path, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0o600) + if err != nil { + return err + } + if err := f.Chmod(0o600); err != nil { + _ = f.Close() + return err + } + if _, err := f.Write(data); err != nil { + _ = f.Close() + return err + } + return f.Close() +} diff --git a/internal/cloudinit/cloudinit_test.go b/internal/cloudinit/cloudinit_test.go index 57c5b3b0..e81b0d08 100644 --- a/internal/cloudinit/cloudinit_test.go +++ b/internal/cloudinit/cloudinit_test.go @@ -265,6 +265,65 @@ func TestSeedMergesCloudRecipe(t *testing.T) { } } +func TestSeedSecretArtifactsArePrivate(t *testing.T) { + root := t.TempDir() + t.Setenv("STOAT_HOME", root) + bin := t.TempDir() + // The stand-in deliberately unlinks and recreates the ISO, inheriting the + // caller's umask. Seed must protect the replacement inode before xorriso + // writes bytes. + modeFile := filepath.Join(root, "xorriso-create-mode") + modeFileQ := shellQuoteCloudinitTest(modeFile) + xorriso := "#!/bin/sh\nwhile [ $# -gt 0 ]; do\n if [ \"$1\" = \"-o\" ]; then out=$2; shift 2; else shift; fi\ndone\nrm -f \"$out\"\n: > \"$out\"\nstat -c '%a' \"$out\" > " + modeFileQ + "\nprintf 'private seed' > \"$out\"\n" + if err := os.WriteFile(filepath.Join(bin, "xorriso"), []byte(xorriso), 0o755); err != nil { + t.Fatal(err) + } + t.Setenv("PATH", bin+string(os.PathListSeparator)+os.Getenv("PATH")) + + const sentinel = "cloud-secret-value" + v := &config.VM{ + Name: "cloudy", Mode: "cloud", OS: "ubuntu", Dir: filepath.Join(root, "cloudy"), + } + if _, err := Seed(v, testPubkey, []string{"#cloud-config\nruncmd:\n - echo " + sentinel + "\n"}); err != nil { + t.Fatal(err) + } + createdMode, err := os.ReadFile(modeFile) + if err != nil { + t.Fatal(err) + } + if strings.TrimSpace(string(createdMode)) != "600" { + t.Fatalf("xorriso replacement mode before payload = %q, want 600", createdMode) + } + seedDir := filepath.Join(v.OvlDir(), "seed") + for _, item := range []struct { + path string + want os.FileMode + }{ + {seedDir, 0o700}, + {filepath.Join(seedDir, "user-data"), 0o600}, + {filepath.Join(v.OvlDir(), "seed.iso"), 0o600}, + } { + info, err := os.Stat(item.path) + if err != nil { + t.Fatalf("stat %s: %v", item.path, err) + } + if got := info.Mode().Perm(); got != item.want { + t.Errorf("%s mode = %#o, want %#o", item.path, got, item.want) + } + } + b, err := os.ReadFile(filepath.Join(seedDir, "user-data")) + if err != nil { + t.Fatal(err) + } + if !strings.Contains(string(b), sentinel) { + t.Fatal("private user-data seed lost the required secret-bearing recipe") + } +} + +func shellQuoteCloudinitTest(value string) string { + return "'" + strings.ReplaceAll(value, "'", "'\\''") + "'" +} + // TestSeedArchiveHeaderIsFirstLine pins what NoCloud checks to recognise a // cloud-config-archive: "#cloud-config-archive" must be the first line of // the file, verbatim. Same shape as the "#cloud-config" match this package diff --git a/internal/cloudinit/scripts.go b/internal/cloudinit/scripts.go index 6ceb58b2..aeb8c303 100644 --- a/internal/cloudinit/scripts.go +++ b/internal/cloudinit/scripts.go @@ -1,7 +1,10 @@ package cloudinit import ( + "encoding/hex" "fmt" + "sort" + "strconv" "strings" "github.com/novusedge/stoat/internal/guest" @@ -17,12 +20,18 @@ const scriptDir = "/var/lib/stoat/recipes" // extension. const MarkerDir = "/var/lib/stoat/.applied" +// SecretsEnvPath is the transient guest path used for cloud-init recipe +// secrets. It is written mode 0600 and removed after all recipe commands run. +const SecretsEnvPath = "/run/stoat/secrets.env" + // Script pairs a recipe's Name with the body WrapScripts should run for it, // i.e. the manifest's Name and manifest.ScriptContent(osName) for the guest // being provisioned. type Script struct { Name string Content string + Env []string + Secrets map[string]string } // WrapScripts renders scripts into a #cloud-config fragment: each script's @@ -46,8 +55,25 @@ func WrapScripts(scripts []Script, prelude string) string { var wf, rc strings.Builder wf.WriteString("write_files:\n") rc.WriteString("runcmd:\n") + hasSecrets := false + for _, s := range scripts { + if len(s.Secrets) > 0 { + hasSecrets = true + break + } + } + if hasSecrets { + fmt.Fprintf(&wf, " - path: %s\n", SecretsEnvPath) + wf.WriteString(" permissions: '0600'\n") + wf.WriteString(" content: |\n") + wf.WriteString(indentBlock(secretEnv(scripts))) + } if prelude != "" { - rc.WriteString(fmt.Sprintf(" - sh -c %s\n", guest.ShQuote(prelude+"stoat_pkg_setup"))) + setup := "sh -c " + guest.ShQuote(prelude+"stoat_pkg_setup") + // YAML plain scalars cannot contain the unindented newlines in a + // guest prelude. Encode the complete shell command as a YAML string; + // the parser restores those newlines before cloud-init invokes sh. + rc.WriteString(fmt.Sprintf(" - %s\n", strconv.Quote(setup))) } for _, s := range scripts { path := fmt.Sprintf("%s/%s.sh", scriptDir, s.Name) @@ -60,12 +86,91 @@ func WrapScripts(scripts []Script, prelude string) string { // leaves no marker for a script that failed, so a failed recipe stays // pending instead of being recorded as applied. marker := fmt.Sprintf("%s/%s", MarkerDir, s.Name) - fmt.Fprintf(&rc, " - %s && mkdir -p %s && touch %s\n", path, MarkerDir, marker) + command := recipeCommand(s, path, marker) + if len(s.Env) > 0 || len(s.Secrets) > 0 { + command = strconv.Quote(command) + } + fmt.Fprintf(&rc, " - %s\n", command) + } + if hasSecrets { + fmt.Fprintf(&rc, " - rm -f %s\n", SecretsEnvPath) } return "#cloud-config\n" + wf.String() + rc.String() } +func secretEnv(scripts []Script) string { + var b strings.Builder + for _, s := range scripts { + names := make([]string, 0, len(s.Secrets)) + for name := range s.Secrets { + names = append(names, name) + } + sort.Strings(names) + for _, name := range names { + key := namespacedSecret(s.Name, name) + fmt.Fprintf(&b, "%s=%s\n", key, guest.ShQuote(s.Secrets[name])) + } + } + return b.String() +} + +func namespacedSecret(recipe, param string) string { + if plainNamespacePart(recipe) && plainNamespacePart(param) { + return "STOAT_PARAM_" + strings.ToUpper(recipe) + "_" + strings.ToUpper(param) + } + return "STOAT_PARAM_X" + hex.EncodeToString([]byte(recipe)) + "_" + hex.EncodeToString([]byte(param)) +} + +// plainNamespacePart retains the historical readable spelling for the +// ordinary lower-case recipe names and parameter names already in use. Any +// punctuation, underscore, or uppercase uses the pair encoding above, which +// makes the recipe/param boundary unambiguous and preserves case identity. +func plainNamespacePart(value string) bool { + if value == "" { + return false + } + for _, r := range value { + if !((r >= 'a' && r <= 'z') || (r >= '0' && r <= '9')) { + return false + } + } + return true +} + +func recipeCommand(s Script, path, marker string) string { + var b strings.Builder + if len(s.Secrets) > 0 { + fmt.Fprintf(&b, ". %s && ", SecretsEnvPath) + } + for _, entry := range s.Env { + key, value, ok := strings.Cut(entry, "=") + if !ok || key == "" { + continue + } + fmt.Fprintf(&b, "export %s=%s && ", key, guest.ShQuote(value)) + } + names := make([]string, 0, len(s.Secrets)) + for name := range s.Secrets { + names = append(names, name) + } + sort.Strings(names) + for _, name := range names { + key := "STOAT_PARAM_" + strings.ToUpper(name) + fmt.Fprintf(&b, "export %s=\"$%s\" && ", key, namespacedSecret(s.Name, name)) + } + output := "/tmp/.stoat-out/" + s.Name + if len(s.Env) > 0 { + fmt.Fprintf(&b, "STOAT_OUTPUT=%s && export STOAT_OUTPUT && mkdir -p /tmp/.stoat-out && chmod 700 /tmp/.stoat-out && : > \"$STOAT_OUTPUT\" && ", guest.ShQuote(output)) + } + fmt.Fprintf(&b, "%s && mkdir -p %s && ", path, MarkerDir) + if len(s.Env) > 0 { + fmt.Fprintf(&b, "if [ -f %s ]; then cp %s %s.out; fi && ", output, output, marker) + } + fmt.Fprintf(&b, "touch %s", marker) + return b.String() +} + // indentBlock indents body by six spaces, the depth a YAML block scalar // needs under write_files' "content: |": two for the write_files list item, // two more for content: under path/permissions, two more for the scalar diff --git a/internal/cloudinit/scripts_test.go b/internal/cloudinit/scripts_test.go index e9963768..31bd83f3 100644 --- a/internal/cloudinit/scripts_test.go +++ b/internal/cloudinit/scripts_test.go @@ -1,9 +1,13 @@ package cloudinit import ( + "os" + "os/exec" + "path/filepath" "strings" "testing" + "github.com/novusedge/stoat/internal/guest" "gopkg.in/yaml.v3" ) @@ -155,11 +159,324 @@ func TestWrapScriptsFragmentMergesIntoSeed(t *testing.T) { // how the cloudinit path keeps the package index refresh and the recipe // verbs behaving the same as the ssh path. func TestWrapScriptsRunsSetupFirst(t *testing.T) { - got := WrapScripts([]Script{{Name: "x", Content: "#!/bin/sh\necho hi\n"}}, "P\n") - if !strings.Contains(got, "runcmd:\n - sh -c 'P\nstoat_pkg_setup'\n") { - t.Errorf("setup not first in runcmd:\n%s", got) + body := WrapScripts([]Script{{Name: "x", Content: "#!/bin/sh\necho hi\n"}}, "P\n") + f := parseWrapped(t, body) + if len(f.Runcmd) != 2 { + t.Fatalf("runcmd = %v, want setup followed by one recipe", f.Runcmd) + } + if got, want := f.Runcmd[0], "sh -c 'P\nstoat_pkg_setup'"; got != want { + t.Errorf("setup command = %q, want preserved shell command %q", got, want) + } + if !strings.Contains(f.Runcmd[1], "/var/lib/stoat/recipes/x.sh") { + t.Errorf("recipe command = %q, want x recipe after setup", f.Runcmd[1]) + } + if len(f.WriteFiles) != 1 { + t.Fatalf("write_files = %v, want one recipe script", f.WriteFiles) + } + if got, want := f.WriteFiles[0].Content, "#!/bin/sh\nP\necho hi\n"; got != want { + t.Errorf("script content = %q, want prelude after shebang %q", got, want) + } +} + +// Debian's registered prelude contains real multiline shell commands. The +// public wrapper must serialize that setup command as one valid YAML scalar so +// cloud-init can retain and execute the package setup before the recipe. +func TestWrapScriptsSerializesActualDebianPrelude(t *testing.T) { + o, ok := guest.Lookup("debian") + if !ok { + t.Fatal("debian guest definition missing") + } + prelude := guest.Prelude(o, "sh") + body := WrapScripts([]Script{{ + Name: "docker", + Content: "#!/bin/sh\nset -eu\nstoat_pkg_install ca-certificates\n", + }}, prelude) + f := parseWrapped(t, body) + if len(f.Runcmd) < 2 { + t.Fatalf("runcmd = %v, want setup and recipe commands", f.Runcmd) + } + if !strings.Contains(f.Runcmd[0], "stoat_pkg_setup") || !strings.Contains(f.Runcmd[0], "apt-get update") { + t.Fatalf("setup command lost Debian prelude semantics: %q", f.Runcmd[0]) + } + if !strings.Contains(f.Runcmd[1], "/var/lib/stoat/recipes/docker.sh") { + t.Fatalf("recipe command missing after setup: %q", f.Runcmd[1]) + } +} + +func TestWrapScriptsNamespacesSecretsAndRemovesTheSecretFileLast(t *testing.T) { + body := WrapScripts([]Script{ + { + Name: "docker", Content: "#!/bin/sh\necho docker\n", + Env: []string{"STOAT_RECIPE=docker", "STOAT_PARAM_USER=dev"}, + Secrets: map[string]string{"authkey": "docker-secret"}, + }, + { + Name: "tailscale", Content: "#!/bin/sh\necho tailscale\n", + Env: []string{"STOAT_RECIPE=tailscale"}, + Secrets: map[string]string{"authkey": "tailscale-secret"}, + }, + }, "") + f := parseWrapped(t, body) + + var secretFile *struct { + Path string + Permissions string + Content string + } + for i := range f.WriteFiles { + wf := f.WriteFiles[i] + if wf.Path == SecretsEnvPath { + copy := struct { + Path string + Permissions string + Content string + }{wf.Path, wf.Permissions, wf.Content} + secretFile = © + } + } + if secretFile == nil { + t.Fatalf("no %s write_files entry:\n%s", SecretsEnvPath, body) + } + if secretFile.Permissions != "0600" { + t.Errorf("secrets permissions = %q, want 0600", secretFile.Permissions) + } + for _, want := range []string{ + "STOAT_PARAM_DOCKER_AUTHKEY", "docker-secret", + "STOAT_PARAM_TAILSCALE_AUTHKEY", "tailscale-secret", + } { + if !strings.Contains(secretFile.Content, want) { + t.Errorf("secret file missing %q:\n%s", want, secretFile.Content) + } + } + if len(f.Runcmd) != 3 { + t.Fatalf("runcmd = %v, want two recipes plus final cleanup", f.Runcmd) + } + if !strings.Contains(f.Runcmd[0], "STOAT_PARAM_DOCKER_AUTHKEY") || !strings.Contains(f.Runcmd[1], "STOAT_PARAM_TAILSCALE_AUTHKEY") { + t.Errorf("recipe wrappers do not select their namespaced secrets: %v", f.Runcmd[:2]) + } + if got := f.Runcmd[len(f.Runcmd)-1]; got != "rm -f "+SecretsEnvPath { + t.Errorf("last runcmd = %q, want secret cleanup", got) + } +} + +func TestWrapScriptsWithoutSecretsWritesNoSecretFile(t *testing.T) { + f := parseWrapped(t, WrapScripts([]Script{{Name: "xfce", Content: "#!/bin/sh\n"}}, "")) + for _, wf := range f.WriteFiles { + if wf.Path == SecretsEnvPath { + t.Fatalf("a recipe with no secrets wrote %s", SecretsEnvPath) + } + } + for _, cmd := range f.Runcmd { + if strings.Contains(cmd, SecretsEnvPath) { + t.Fatalf("a recipe with no secrets references %s: %q", SecretsEnvPath, cmd) + } } - if !strings.Contains(got, " #!/bin/sh\n P\n echo hi\n") { - t.Errorf("prelude not after the shebang:\n%s", got) +} + +func TestWrapScriptsFailureCannotWriteSuccessMarker(t *testing.T) { + f := parseWrapped(t, WrapScripts([]Script{{ + Name: "docker", Content: "#!/bin/sh\nexit 1\n", + Env: []string{"STOAT_RECIPE=docker"}, + }}, "")) + if len(f.Runcmd) != 1 { + t.Fatalf("runcmd = %v, want one recipe command", f.Runcmd) + } + cmd := f.Runcmd[0] + marker := MarkerDir + "/docker" + if !strings.Contains(cmd, "/tmp/.stoat-out/docker") { + t.Errorf("recipe output was not copied before success marking: %q", cmd) + } + markerAt := strings.Index(cmd, marker) + if markerAt < 0 || !strings.Contains(cmd[:markerAt], "&&") { + t.Errorf("success marker is not gated by the recipe/output commands: %q", cmd) + } + if strings.Contains(cmd, "; touch "+marker) { + t.Errorf("success marker is unconditional after a semicolon: %q", cmd) + } +} + +func TestWrapScriptsExecutesRecipeOutputAndGatesMarker(t *testing.T) { + for _, tt := range []struct { + name string + script string + success bool + }{ + { + name: "success", + script: "#!/bin/sh\nset -eu\nprintf '%s\\n' 'socket=/run/demo.sock' > \"$STOAT_OUTPUT\"\n", + success: true, + }, + { + name: "failure", + script: "#!/bin/sh\nset -eu\nprintf '%s\\n' 'socket=/run/demo.sock' > \"$STOAT_OUTPUT\"\nexit 1\n", + }, + } { + t.Run(tt.name, func(t *testing.T) { + body := WrapScripts([]Script{{ + Name: "demo", + Content: tt.script, + Env: []string{"STOAT_RECIPE=demo"}, + }}, "") + f := parseWrapped(t, body) + if len(f.WriteFiles) != 1 || len(f.Runcmd) != 1 { + t.Fatalf("generated files/commands = %d/%d, want one each:\n%s", len(f.WriteFiles), len(f.Runcmd), body) + } + + harness := t.TempDir() + writeFiles := map[string]string{ + "/var/lib/stoat/recipes": filepath.Join(harness, "recipes"), + "/var/lib/stoat/.applied": filepath.Join(harness, "applied"), + "/tmp/.stoat-out": filepath.Join(harness, "out"), + } + for _, wf := range f.WriteFiles { + path := relocateGuestPath(wf.Path, writeFiles) + if !pathWithin(path, harness) { + t.Fatalf("write_files path escaped harness: %q", path) + } + if err := os.MkdirAll(filepath.Dir(path), 0o700); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(path, []byte(wf.Content), 0o755); err != nil { + t.Fatal(err) + } + } + + command := relocateGuestPath(f.Runcmd[0], writeFiles) + if !strings.Contains(command, harness) { + t.Fatalf("generated command was not relocated into harness: %q", command) + } + cmd := exec.Command("sh", "-eu", "-c", command) + if err := cmd.Run(); (err == nil) != tt.success { + t.Fatalf("generated recipe command error = %v, success = %v", err, tt.success) + } + + marker := filepath.Join(harness, "applied", "demo") + copied := marker + ".out" + if tt.success { + if _, err := os.Stat(marker); err != nil { + t.Fatalf("success marker missing: %v", err) + } + got, err := os.ReadFile(copied) + if err != nil { + t.Fatalf("copied output missing: %v", err) + } + if string(got) != "socket=/run/demo.sock\n" { + t.Fatalf("copied output = %q", got) + } + } else { + if _, err := os.Stat(marker); !os.IsNotExist(err) { + t.Errorf("failure marker stat = %v, want absent", err) + } + if _, err := os.Stat(copied); !os.IsNotExist(err) { + t.Errorf("failure copied output stat = %v, want absent", err) + } + } + }) + } +} + +// Recipe names are user-controlled directory names and may contain a dash. +// The wrapper must translate each namespaced secret into a shell-safe, +// collision-free environment variable before invoking the child script. This +// executes dashed, underscored, and literal escape-shaped names so a +// normalization scheme that aliases them cannot silently deliver one recipe's +// secret to the other. +func TestWrapScriptsExecutesHyphenatedSecretsAndCleansUp(t *testing.T) { + scripts := []Script{ + { + Name: "my-recipe", + Content: "#!/bin/sh\nset -eu\ntest \"$STOAT_PARAM_TOKEN\" = hyphen-secret\n", + Secrets: map[string]string{"token": "hyphen-secret"}, + }, + { + Name: "my_recipe", + Content: "#!/bin/sh\nset -eu\ntest \"$STOAT_PARAM_TOKEN\" = underscore-secret\n", + Secrets: map[string]string{"token": "underscore-secret"}, + }, + { + Name: "my_2drecipe", + Content: "#!/bin/sh\nset -eu\ntest \"$STOAT_PARAM_TOKEN\" = escape-shaped-secret\n", + Secrets: map[string]string{"token": "escape-shaped-secret"}, + }, + { + Name: "foo", + Content: "#!/bin/sh\nset -eu\ntest \"$STOAT_PARAM_TOKEN\" = lower-case-secret\n", + Secrets: map[string]string{"token": "lower-case-secret"}, + }, + { + Name: "Foo", + Content: "#!/bin/sh\nset -eu\ntest \"$STOAT_PARAM_TOKEN\" = upper-case-secret\n", + Secrets: map[string]string{"token": "upper-case-secret"}, + }, + { + Name: "a-b", + Content: "#!/bin/sh\nset -eu\ntest \"$STOAT_PARAM_C\" = pair-left-secret\n", + Secrets: map[string]string{"c": "pair-left-secret"}, + }, + { + Name: "a", + Content: "#!/bin/sh\nset -eu\ntest \"$STOAT_PARAM_B_C\" = pair-right-secret\n", + Secrets: map[string]string{"b_c": "pair-right-secret"}, + }, + } + f := parseWrapped(t, WrapScripts(scripts, "")) + if len(f.Runcmd) != len(scripts)+1 { + t.Fatalf("runcmd = %v, want two recipes plus cleanup", f.Runcmd) + } + + harness := t.TempDir() + paths := map[string]string{ + "/var/lib/stoat/recipes": filepath.Join(harness, "recipes"), + "/var/lib/stoat/.applied": filepath.Join(harness, "applied"), + "/run/stoat": filepath.Join(harness, "run", "stoat"), } + for _, wf := range f.WriteFiles { + path := relocateGuestPath(wf.Path, paths) + if !pathWithin(path, harness) { + t.Fatalf("write_files path escaped harness: %q", path) + } + if err := os.MkdirAll(filepath.Dir(path), 0o700); err != nil { + t.Fatal(err) + } + perm := os.FileMode(0o600) + if wf.Permissions == "0755" { + perm = 0o755 + } + if err := os.WriteFile(path, []byte(wf.Content), perm); err != nil { + t.Fatal(err) + } + } + + for i := range scripts { + command := relocateGuestPath(f.Runcmd[i], paths) + cmd := exec.Command("sh", "-eu", "-c", command) + output, err := cmd.CombinedOutput() + if err != nil { + t.Fatalf("recipe %q command failed: %v\n%s\noutput:\n%s", scripts[i].Name, err, command, output) + } + } + cleanup := relocateGuestPath(f.Runcmd[len(f.Runcmd)-1], paths) + if cleanup != "rm -f "+paths["/run/stoat"]+"/secrets.env" { + t.Fatalf("last runcmd = %q, want secret cleanup", f.Runcmd[len(f.Runcmd)-1]) + } + if err := exec.Command("sh", "-eu", "-c", cleanup).Run(); err != nil { + t.Fatalf("secret cleanup failed: %v", err) + } + if _, err := os.Stat(filepath.Join(paths["/run/stoat"], "secrets.env")); !os.IsNotExist(err) { + t.Fatalf("secret file remains after final cleanup: %v", err) + } +} + +func relocateGuestPath(value string, replacements map[string]string) string { + for _, path := range []string{"/var/lib/stoat/recipes", "/var/lib/stoat/.applied", "/tmp/.stoat-out", "/run/stoat"} { + if replacement, ok := replacements[path]; ok { + value = strings.ReplaceAll(value, path, replacement) + } + } + return value +} + +func pathWithin(path, root string) bool { + rel, err := filepath.Rel(root, path) + return err == nil && rel != ".." && !strings.HasPrefix(rel, ".."+string(filepath.Separator)) } diff --git a/internal/config/config.go b/internal/config/config.go index 46a183b8..1d33189d 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -13,7 +13,6 @@ import ( "strings" "time" - "github.com/BurntSushi/toml" "github.com/novusedge/stoat/internal/tomlx" ) @@ -36,24 +35,28 @@ type PortForward struct { // with an empty string, never equal to a current script's hash, so that // recipe re-runs once and then carries a real hash from then on. type AppliedRecipe struct { - Version string `toml:"version"` - Hash string `toml:"hash"` - At time.Time `toml:"at"` + Version string `toml:"version"` + Hash string `toml:"hash"` + ScriptHash string `toml:"script_hash"` + At time.Time `toml:"at"` + Outputs map[string]string `toml:"outputs,omitempty" comment:"written by stoat; do not edit"` + Health string `toml:"health"` } // VM is one virtual machine. vm.toml is authoritative; there is no cache. type VM struct { - Name string `toml:"name"` - Mode string `toml:"mode"` // "live" | "disk" | "cloud" - OS string `toml:"os"` - ISO string `toml:"iso"` // relative to the data root - RAM int `toml:"ram"` // MB - CPUs int `toml:"cpus"` - Disk string `toml:"disk"` // disk mode only, e.g. "8G" - Installed bool `toml:"installed"` // disk mode only; flips boot order - Share string `toml:"share"` // host dir exposed as /mnt/host - SSHPort int `toml:"sshport"` - Recipes []string `toml:"recipes"` + Name string `toml:"name"` + Mode string `toml:"mode"` // "live" | "disk" | "cloud" + OS string `toml:"os"` + ISO string `toml:"iso"` // relative to the data root + RAM int `toml:"ram"` // MB + CPUs int `toml:"cpus"` + Disk string `toml:"disk"` // disk mode only, e.g. "8G" + Installed bool `toml:"installed"` // disk mode only; flips boot order + Share string `toml:"share"` // host dir exposed as /mnt/host + SSHPort int `toml:"sshport"` + Recipes []string `toml:"recipes,omitempty"` + Params map[string]map[string]string `toml:"params,omitempty" comment:"written by stoat; do not edit"` // Display is the user's screen preference: "" or "auto" (default), // "window", or "vnc". "auto" opens a real qemu window on a graphical @@ -73,7 +76,7 @@ type VM struct { // vm.toml immediately but has no effect on the live process. See // core.Forward and core.ErrAppliesAtNextStart, which exist so a caller // cannot mistake "saved" for "live". - Forwards []PortForward `toml:"forwards"` + Forwards []PortForward `toml:"forwards,omitempty"` // Backend is the provisioning backend: "apkovl" | "cloudinit" | "ssh". // Written by the form at creation time; dispatch elsewhere in stoat @@ -103,11 +106,53 @@ type VM struct { AllowExec bool `toml:"allow_exec"` // Applied tracks which recipes have been run on this VM, keyed by recipe name. - Applied map[string]AppliedRecipe `toml:"applied"` + Applied map[string]AppliedRecipe `toml:"applied,omitempty" comment:"written by stoat; do not edit"` Dir string `toml:"-"` // absolute path to the VM directory } +// Param reads one stored recipe parameter. +func (v *VM) Param(recipe, name string) (string, bool) { + if v == nil { + return "", false + } + values, ok := v.Params[recipe] + if !ok { + return "", false + } + value, ok := values[name] + return value, ok +} + +// SetParam stores one non-secret recipe parameter. +func (v *VM) SetParam(recipe, name, value string) { + if v.Params == nil { + v.Params = make(map[string]map[string]string) + } + if v.Params[recipe] == nil { + v.Params[recipe] = make(map[string]string) + } + v.Params[recipe][name] = value +} + +// UnsetParam removes one stored recipe parameter. +func (v *VM) UnsetParam(recipe, name string) { + if v == nil || v.Params == nil { + return + } + values, ok := v.Params[recipe] + if !ok { + return + } + delete(values, name) + if len(values) == 0 { + delete(v.Params, recipe) + } + if len(v.Params) == 0 { + v.Params = nil + } +} + // Root is the data root: $STOAT_HOME, or ~/.stoat. func Root() string { if r := os.Getenv("STOAT_HOME"); r != "" { @@ -196,12 +241,7 @@ func (v *VM) Save() error { if err := os.MkdirAll(v.Dir, 0o755); err != nil { return err } - f, err := os.Create(v.path()) - if err != nil { - return err - } - defer func() { _ = f.Close() }() - return toml.NewEncoder(f).Encode(v) + return tomlx.Encode(v.path(), v) } // Load reads one VM by name. diff --git a/internal/config/encode_test.go b/internal/config/encode_test.go new file mode 100644 index 00000000..97fd2950 --- /dev/null +++ b/internal/config/encode_test.go @@ -0,0 +1,88 @@ +package config + +import ( + "os" + "path/filepath" + "strings" + "testing" + "time" +) + +// The comment is the contract: a human editing vm.toml must be able to see +// which tables stoat rewrites on every apply. +func TestSaveCommentsTheAppliedTable(t *testing.T) { + t.Setenv("STOAT_HOME", t.TempDir()) + v := &VM{ + Name: "work", Mode: "live", RAM: 1024, CPUs: 1, SSHPort: 2200, + Recipes: []string{"docker"}, + Applied: map[string]AppliedRecipe{ + "docker": {Version: "1.2.0", Hash: "sha256:abc", At: time.Unix(0, 0).UTC()}, + }, + } + if err := v.Save(); err != nil { + t.Fatal(err) + } + b, err := os.ReadFile(filepath.Join(Root(), "work", "vm.toml")) + if err != nil { + t.Fatal(err) + } + lines := strings.Split(string(b), "\n") + for i, l := range lines { + if !strings.HasPrefix(strings.TrimSpace(l), "[applied") { + continue + } + for j := i - 1; j >= 0; j-- { + if strings.TrimSpace(lines[j]) == "" { + continue + } + if !strings.Contains(lines[j], "written by stoat; do not edit") { + t.Fatalf("line before %q is %q, want the stoat-owned comment", l, lines[j]) + } + return + } + t.Fatalf("nothing precedes %q", l) + } + t.Fatalf("no applied table in:\n%s", b) +} + +// An absent allow_exec key means true. A vm.toml written before the field +// existed must not lose exec. +func TestLoadDefaultsAllowExecTrue(t *testing.T) { + root := t.TempDir() + t.Setenv("STOAT_HOME", root) + dir := filepath.Join(root, "old") + if err := os.MkdirAll(dir, 0o755); err != nil { + t.Fatal(err) + } + body := "name = \"old\"\nmode = \"live\"\nram = 1024\ncpus = 1\nsshport = 2201\n" + if err := os.WriteFile(filepath.Join(dir, "vm.toml"), []byte(body), 0o644); err != nil { + t.Fatal(err) + } + v, err := Load("old") + if err != nil { + t.Fatal(err) + } + if !v.AllowExec { + t.Error("AllowExec = false, want true for an absent key") + } +} + +func TestLoadKeepsExplicitAllowExecFalse(t *testing.T) { + root := t.TempDir() + t.Setenv("STOAT_HOME", root) + dir := filepath.Join(root, "locked") + if err := os.MkdirAll(dir, 0o755); err != nil { + t.Fatal(err) + } + body := "name = \"locked\"\nmode = \"live\"\nram = 1024\ncpus = 1\nsshport = 2202\nallow_exec = false\n" + if err := os.WriteFile(filepath.Join(dir, "vm.toml"), []byte(body), 0o644); err != nil { + t.Fatal(err) + } + v, err := Load("locked") + if err != nil { + t.Fatal(err) + } + if v.AllowExec { + t.Error("AllowExec = true, want the explicit false") + } +} diff --git a/internal/config/params_test.go b/internal/config/params_test.go new file mode 100644 index 00000000..aa053af8 --- /dev/null +++ b/internal/config/params_test.go @@ -0,0 +1,103 @@ +package config + +import ( + "os" + "path/filepath" + "strings" + "testing" + "time" +) + +func TestParamsRoundTrip(t *testing.T) { + t.Setenv("STOAT_HOME", t.TempDir()) + v := &VM{Name: "work", Mode: "live", RAM: 1024, CPUs: 1, SSHPort: 2200} + v.SetParam("docker", "user", "dev") + v.SetParam("docker", "channel", "stable") + v.Recipes = []string{"docker"} + v.Applied = map[string]AppliedRecipe{ + "docker": { + Version: "1.2.0", Hash: "sha256:abc", At: time.Unix(0, 0).UTC(), + Outputs: map[string]string{"socket": "/var/run/docker.sock"}, + Health: "ok", + }, + } + if err := v.Save(); err != nil { + t.Fatal(err) + } + b, err := os.ReadFile(filepath.Join(Root(), "work", "vm.toml")) + if err != nil { + t.Fatal(err) + } + text := string(b) + if !strings.Contains(text, `recipes = ["docker"]`) { + t.Errorf("recipes array was not preserved:\n%s", text) + } + if strings.Contains(text, "[recipes.docker]") { + t.Errorf("params were written under the recipes array:\n%s", text) + } + if !strings.Contains(text, "[applied.docker.outputs]") { + t.Errorf("outputs were not written as an applied subtable:\n%s", text) + } + lines := strings.Split(text, "\n") + for i, line := range lines { + trimmed := strings.TrimSpace(line) + if !strings.HasPrefix(trimmed, "[params") && !strings.HasPrefix(trimmed, "[applied") { + continue + } + j := i - 1 + for j >= 0 && strings.TrimSpace(lines[j]) == "" { + j-- + } + if j < 0 || !strings.Contains(lines[j], "written by stoat; do not edit") { + t.Errorf("table %q lacks the stoat-owned comment", trimmed) + } + } + got, err := Load("work") + if err != nil { + t.Fatal(err) + } + if u, ok := got.Param("docker", "user"); !ok || u != "dev" { + t.Errorf("user = %q %v, want dev true", u, ok) + } + if got.Applied["docker"].Outputs["socket"] != "/var/run/docker.sock" { + t.Errorf("outputs = %v", got.Applied["docker"].Outputs) + } + if got.Applied["docker"].Health != "ok" { + t.Errorf("health = %q, want ok", got.Applied["docker"].Health) + } +} + +func TestParamAccessorsTrackEmptyValues(t *testing.T) { + v := &VM{} + v.SetParam("docker", "user", "") + if got, ok := v.Param("docker", "user"); !ok || got != "" { + t.Errorf("empty value = %q %v, want empty true", got, ok) + } + v.UnsetParam("docker", "user") + if _, ok := v.Param("docker", "user"); ok { + t.Error("unset parameter remains present") + } + if _, ok := v.Params["docker"]; ok { + t.Error("empty recipe table remains after unsetting its last value") + } +} + +func TestUnsetParamDropsTheTableWhenEmpty(t *testing.T) { + t.Setenv("STOAT_HOME", t.TempDir()) + v := &VM{Name: "work", Mode: "live", RAM: 1024, CPUs: 1, SSHPort: 2200} + v.SetParam("docker", "user", "dev") + v.UnsetParam("docker", "user") + if _, ok := v.Params["docker"]; ok { + t.Error("an empty recipe table stays in vm.toml") + } + if err := v.Save(); err != nil { + t.Fatal(err) + } + b, err := os.ReadFile(filepath.Join(Root(), "work", "vm.toml")) + if err != nil { + t.Fatal(err) + } + if strings.Contains(string(b), "[params.docker]") { + t.Errorf("empty table written:\n%s", b) + } +} diff --git a/internal/config/secrets.go b/internal/config/secrets.go new file mode 100644 index 00000000..e26646f2 --- /dev/null +++ b/internal/config/secrets.go @@ -0,0 +1,104 @@ +package config + +import ( + "fmt" + "os" + "path/filepath" + "sort" + + "github.com/novusedge/stoat/internal/tomlx" +) + +// SecretsName is the file holding secret recipe parameter values. +const SecretsName = "secrets.toml" + +// Secrets maps a recipe name to its secret parameter values. +type Secrets map[string]map[string]string + +// SecretsPath returns the path to this VM's secrets file. +func (v *VM) SecretsPath() string { return filepath.Join(v.Dir, SecretsName) } + +// LoadSecrets reads one VM's secret parameter values. +func LoadSecrets(dir string) (Secrets, error) { + path := filepath.Join(dir, SecretsName) + info, err := os.Stat(path) + if os.IsNotExist(err) { + return Secrets{}, nil + } + if err != nil { + return nil, err + } + if perm := info.Mode().Perm(); perm&^0o600 != 0 { + return nil, fmt.Errorf("%s: mode %#o, want 0600", SecretsName, perm) + } + secrets := Secrets{} + if err := tomlx.Decode(path, &secrets, tomlx.Reject); err != nil { + return nil, err + } + return secrets, nil +} + +// SaveSecrets writes one VM's secret parameter values. +func SaveSecrets(dir string, s Secrets) error { + path := filepath.Join(dir, SecretsName) + clean := make(Secrets, len(s)) + for recipe, values := range s { + if len(values) == 0 { + continue + } + copyValues := make(map[string]string, len(values)) + for name, value := range values { + copyValues[name] = value + } + clean[recipe] = copyValues + } + if len(clean) == 0 { + err := os.Remove(path) + if os.IsNotExist(err) { + return nil + } + return err + } + + tmp, err := os.CreateTemp(dir, ".secrets-*") + if err != nil { + return err + } + tmpPath := tmp.Name() + keep := false + defer func() { + if !keep { + _ = os.Remove(tmpPath) + } + }() + if err := tmp.Chmod(0o600); err != nil { + _ = tmp.Close() + return err + } + if err := tmp.Close(); err != nil { + return err + } + if err := tomlx.Encode(tmpPath, clean); err != nil { + return err + } + if err := os.Chmod(tmpPath, 0o600); err != nil { + return err + } + if err := os.Rename(tmpPath, path); err != nil { + return err + } + keep = true + return nil +} + +// Names returns the names of a recipe's set secret parameters. +func (s Secrets) Names(recipe string) []string { + var names []string + for name, value := range s[recipe] { + if value != "" { + names = append(names, name) + } + } + sort.Strings(names) + return names +} diff --git a/internal/config/secrets_test.go b/internal/config/secrets_test.go new file mode 100644 index 00000000..d29b0055 --- /dev/null +++ b/internal/config/secrets_test.go @@ -0,0 +1,120 @@ +package config + +import ( + "os" + "path/filepath" + "reflect" + "slices" + "strings" + "testing" +) + +func TestSecretsRoundTripAt0600(t *testing.T) { + dir := t.TempDir() + if got, want := (&VM{Dir: dir}).SecretsPath(), filepath.Join(dir, SecretsName); got != want { + t.Errorf("SecretsPath = %q, want %q", got, want) + } + s := Secrets{ + "empty": {}, + "docker": {"zkey": "z", "authkey": "tskey-abc", "unset": ""}, + } + wantInput := Secrets{ + "empty": {}, + "docker": {"zkey": "z", "authkey": "tskey-abc", "unset": ""}, + } + if err := SaveSecrets(dir, s); err != nil { + t.Fatal(err) + } + if !reflect.DeepEqual(s, wantInput) { + t.Errorf("SaveSecrets mutated its input: got %#v, want %#v", s, wantInput) + } + fi, err := os.Stat(filepath.Join(dir, SecretsName)) + if err != nil { + t.Fatal(err) + } + if fi.Mode().Perm() != 0o600 { + t.Errorf("mode = %o, want 600", fi.Mode().Perm()) + } + got, err := LoadSecrets(dir) + if err != nil { + t.Fatal(err) + } + if got["docker"]["authkey"] != "tskey-abc" { + t.Errorf("got %v", got) + } + if names := got.Names("docker"); !slices.Equal(names, []string{"authkey", "zkey"}) { + t.Errorf("Names = %v", names) + } +} + +func TestSaveSecretsProtectsExistingWideFile(t *testing.T) { + dir := t.TempDir() + path := filepath.Join(dir, SecretsName) + if err := os.WriteFile(path, []byte("docker.authkey = \"old-secret\"\n"), 0o644); err != nil { + t.Fatal(err) + } + if err := SaveSecrets(dir, Secrets{"docker": {"authkey": "new-secret"}}); err != nil { + t.Fatal(err) + } + fi, err := os.Stat(path) + if err != nil { + t.Fatal(err) + } + if fi.Mode().Perm() != 0o600 { + t.Errorf("mode = %o, want 600 after replacing an existing file", fi.Mode().Perm()) + } + got, err := LoadSecrets(dir) + if err != nil { + t.Fatal(err) + } + if got["docker"]["authkey"] != "new-secret" { + t.Errorf("got %v, want the replacement secret", got) + } +} + +func TestLoadSecretsRefusesWideModeWithoutSecretValue(t *testing.T) { + dir := t.TempDir() + path := filepath.Join(dir, SecretsName) + sentinel := "tskey-secret-sentinel" + if err := os.WriteFile(path, []byte("docker.authkey = \""+sentinel+"\"\n"), 0o644); err != nil { + t.Fatal(err) + } + _, err := LoadSecrets(dir) + if err == nil || !strings.Contains(err.Error(), "secrets.toml: mode 0644, want 0600") { + t.Fatalf("err = %v, want the mode refusal", err) + } + if strings.Contains(err.Error(), sentinel) { + t.Fatalf("error exposes the secret value: %v", err) + } +} + +func TestLoadSecretsMissingFileIsEmpty(t *testing.T) { + got, err := LoadSecrets(t.TempDir()) + if err != nil { + t.Fatal(err) + } + if len(got) != 0 { + t.Errorf("got %v, want no secrets", got) + } +} + +// The spec writes "docker.authkey"; TOML reads it as a nested table. Both +// spellings must load the same. +func TestLoadSecretsAcceptsDottedAndTableForms(t *testing.T) { + for _, body := range []string{ + "docker.authkey = \"x\"\n", + "[docker]\nauthkey = \"x\"\n", + } { + dir := t.TempDir() + if err := os.WriteFile(filepath.Join(dir, SecretsName), []byte(body), 0o600); err != nil { + t.Fatal(err) + } + got, err := LoadSecrets(dir) + if err != nil { + t.Fatalf("%q: %v", body, err) + } + if got["docker"]["authkey"] != "x" { + t.Errorf("%q: got %v", body, got) + } + } +} diff --git a/internal/core/access.go b/internal/core/access.go index 8e6ff76b..9a51390c 100644 --- a/internal/core/access.go +++ b/internal/core/access.go @@ -8,6 +8,8 @@ import ( "os" "path/filepath" "slices" + "sort" + "strings" "github.com/novusedge/stoat/internal/config" "github.com/novusedge/stoat/internal/sshx" @@ -77,6 +79,10 @@ func Logs(name string, which Which) (io.ReadCloser, error) { case err != nil: return nil, err } + secrets, err := config.LoadSecrets(v.Dir) + if err != nil { + return nil, fmt.Errorf("%s: %w", name, err) + } path := v.ProvisionLogPath() if which == WhichConsole { @@ -90,5 +96,29 @@ func Logs(name string, which Which) (io.ReadCloser, error) { if err != nil { return nil, err } - return f, nil + b, readErr := io.ReadAll(f) + closeErr := f.Close() + if readErr != nil { + return nil, readErr + } + if closeErr != nil { + return nil, closeErr + } + return io.NopCloser(bytes.NewReader([]byte(redactLog(string(b), secrets)))), nil +} + +func redactLog(value string, secrets config.Secrets) string { + var values []string + for _, recipe := range secrets { + for _, secret := range recipe { + if secret != "" { + values = append(values, secret) + } + } + } + sort.Slice(values, func(i, j int) bool { return len(values[i]) > len(values[j]) }) + for _, secret := range values { + value = strings.ReplaceAll(value, secret, "") + } + return value } diff --git a/internal/core/access_test.go b/internal/core/access_test.go index 5386e3b2..c2f0c43f 100644 --- a/internal/core/access_test.go +++ b/internal/core/access_test.go @@ -5,6 +5,7 @@ import ( "io" "os" "path/filepath" + "strings" "testing" "github.com/novusedge/stoat/internal/config" @@ -118,6 +119,53 @@ func TestLogsReturnsWrittenBytes(t *testing.T) { } } +// Logs is a public reader boundary. Stored secret values may appear in either +// backend's output, so the reader must redact them before a CLI, MCP, or TUI +// can expose the bytes. +func TestLogsRedactsStoredSecretValues(t *testing.T) { + v := vm(t, "work", "") + const sentinel = "logs-secret-sentinel" + if err := config.SaveSecrets(v.Dir, config.Secrets{"docker": {"authkey": sentinel}}); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(v.ProvisionLogPath(), []byte("docker authkey="+sentinel+"\n"), 0o644); err != nil { + t.Fatal(err) + } + + r, err := Logs(v.Name, WhichApply) + if err != nil { + t.Fatal(err) + } + defer func() { _ = r.Close() }() + b, err := io.ReadAll(r) + if err != nil { + t.Fatal(err) + } + if strings.Contains(string(b), sentinel) { + t.Fatalf("apply log leaks stored secret %q: %q", sentinel, b) + } + if !strings.Contains(string(b), "") { + t.Errorf("apply log = %q, want the redaction marker", b) + } +} + +// A malformed or insecure secret store must fail the reader with VM context; +// silently opening raw log bytes would make the mode check meaningless. +func TestLogsRefusesInsecureSecretStoreWithVMContext(t *testing.T) { + v := vm(t, "work", "") + path := filepath.Join(v.Dir, config.SecretsName) + if err := os.WriteFile(path, []byte("docker.authkey = \"sentinel\"\n"), 0o644); err != nil { + t.Fatal(err) + } + if err := os.Chmod(path, 0o644); err != nil { + t.Fatal(err) + } + + if _, err := Logs(v.Name, WhichApply); err == nil || !strings.Contains(err.Error(), "work") || !strings.Contains(err.Error(), "secrets.toml: mode 0644") { + t.Fatalf("Logs error = %v, want VM context and secret-file mode", err) + } +} + // A VM that was never started or provisioned has neither log file. That is // normal, not an error (see Logs' doc comment), so this must come back as // an empty, already-EOF reader rather than a failure. diff --git a/internal/core/apply.go b/internal/core/apply.go index 2a0b465a..dceefd79 100644 --- a/internal/core/apply.go +++ b/internal/core/apply.go @@ -6,6 +6,7 @@ import ( "fmt" "os" "os/exec" + "sort" "strings" "time" @@ -172,12 +173,14 @@ func applyLocked(ctx context.Context, v *config.VM, opts ApplyOpts) error { // A cloudinit VM ran its recipes from the seed at first boot and never // populated v.Applied. Read the marker files cloud-init left behind, so // filterByRunMode can skip a "once" recipe instead of re-running it. - if err := discoverCloudInitApplied(ctx, v); err != nil { + discoveryWarnings, err := discoverCloudInitApplied(ctx, v) + if err != nil { return err } runTargets, manifests, err := filterByRunMode(v, targets, explicit) if err != nil { + appendProvisionWarnings(v, discoveryWarnings) return err } if len(runTargets) == 0 { @@ -186,8 +189,12 @@ func applyLocked(ctx context.Context, v *config.VM, opts ApplyOpts) error { // nobody named explicitly. This is not a failure. It is the run // mode doing what it declared, so this stays a no-op like the // "nothing to run at all" case above. + appendProvisionWarnings(v, discoveryWarnings) return nil } + if v.Applied == nil { + v.Applied = make(map[string]config.AppliedRecipe, len(runTargets)) + } // run is v with Recipes narrowed to runTargets. config.VM is a plain // value struct (internal/config/config.go) with no mutex and no owned @@ -199,8 +206,10 @@ func applyLocked(ctx context.Context, v *config.VM, opts ApplyOpts) error { run := *v run.Recipes = runTargets - if err := sshx.Provision(ctx, &run); err != nil { - return err + provisionErr := sshx.Provision(ctx, &run) + appendProvisionWarnings(v, discoveryWarnings) + if provisionErr != nil { + return provisionErr } // Provision runs runTargets in order and stops at the first failure, so @@ -208,7 +217,6 @@ func applyLocked(ctx context.Context, v *config.VM, opts ApplyOpts) error { // (one ManifestFor found a manifest for) gets its applied state // recorded. A v1 recipe has no version to record and stays out of // Applied, as it always has. - var changed bool rebootRecipe, needsReboot := "", false for _, name := range runTargets { m, ok := manifests[name] @@ -218,12 +226,19 @@ func applyLocked(ctx context.Context, v *config.VM, opts ApplyOpts) error { if v.Applied == nil { v.Applied = make(map[string]config.AppliedRecipe, len(runTargets)) } - hash, err := recipes.ScriptHash(name, v.OS) + hash, err := recipeHashFor(v, m) if err != nil { return err } - v.Applied[name] = config.AppliedRecipe{Version: m.Version, Hash: hash, At: time.Now()} - changed = true + scriptHash, err := recipes.ScriptHash(name, v.OS) + if err != nil { + return err + } + provisioned := run.Applied[name] + v.Applied[name] = config.AppliedRecipe{ + Version: m.Version, Hash: hash, ScriptHash: scriptHash, At: time.Now(), + Outputs: provisioned.Outputs, Health: string(HealthUnknown), + } if m.Reboot && !needsReboot { rebootRecipe, needsReboot = name, true } @@ -242,10 +257,22 @@ func applyLocked(ctx context.Context, v *config.VM, opts ApplyOpts) error { } } - if !changed { - return nil + verdicts, healthErr := healthChecksForVM(ctx, v, runTargets) + if healthErr != nil { + if saveErr := v.Save(); saveErr != nil { + return saveErr + } + return healthErr } - return v.Save() + if err := v.Save(); err != nil { + return err + } + for _, verdict := range verdicts { + if verdict.Status == HealthFailed { + return fmt.Errorf("%s: %s", verdict.Name, verdict.Detail) + } + } + return nil } // rebootAndWait reboots v's guest over ssh and waits for it to come back. @@ -287,7 +314,7 @@ const rebootSettle = 2 * time.Second // fatal to the reboot itself, so this drops the error rather than aborting // an otherwise successful apply over a log write. func appendProvisionLog(v *config.VM, s string) { - f, err := os.OpenFile(v.ProvisionLogPath(), os.O_APPEND|os.O_WRONLY, 0o644) + f, err := os.OpenFile(v.ProvisionLogPath(), os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0o644) if err != nil { return } @@ -295,6 +322,12 @@ func appendProvisionLog(v *config.VM, s string) { _, _ = f.WriteString(s) } +func appendProvisionWarnings(v *config.VM, warnings []string) { + for _, warning := range warnings { + appendProvisionLog(v, warning+"\n") + } +} + // discoverCloudInitApplied rebuilds v.Applied for a cloudinit VM from the // marker files cloud-init left after first boot. It runs over ssh, so the VM // must be reachable; applyLocked calls it only after the qemu.Running check. @@ -308,35 +341,97 @@ func appendProvisionLog(v *config.VM, s string) { // The recorded Hash comes from the current script on disk, not from whatever // cloud-init ran at creation. That is benign: recipes are idempotent, and a // script that has since changed reruns on this same Apply anyway. -func discoverCloudInitApplied(ctx context.Context, v *config.VM) error { +func discoverCloudInitApplied(ctx context.Context, v *config.VM) ([]string, error) { if backend.For(v).Name() != "cloudinit" || len(v.Applied) > 0 { - return nil + return nil, nil } - out, err := exec.CommandContext(ctx, "ssh", sshx.Args(v, "ls -1 "+cloudinit.MarkerDir+" 2>/dev/null")...).Output() + script := fmt.Sprintf("for marker in %s/*; do case \"$marker\" in *.out) continue;; esac; [ -f \"$marker\" ] || continue; name=$(basename \"$marker\"); printf '===%%s\\n' \"$name\"; cat \"$marker.out\" 2>/dev/null; done", cloudinit.MarkerDir) + out, err := exec.CommandContext(ctx, "ssh", sshx.Args(v, script)...).Output() if err != nil { - return nil // marker dir missing or a transient ssh error; discover nothing + return nil, nil // marker dir missing or a transient ssh error; discover nothing } + secrets, err := config.LoadSecrets(v.Dir) + if err != nil { + return nil, err + } var applied map[string]config.AppliedRecipe - for _, name := range strings.Fields(string(out)) { - hash, err := recipes.ScriptHash(name, v.OS) - if err != nil { + var warnings []string + outputs := cloudInitOutputs(string(out)) + names := make([]string, 0, len(outputs)) + for name := range outputs { + names = append(names, name) + } + sort.Strings(names) + for _, name := range names { + body := outputs[name] + m, ok, manifestErr := recipes.ManifestFor(name) + if manifestErr != nil || !ok { continue // a marker for a recipe no longer on disk } - ver := "" - if m, ok, _ := recipes.ManifestFor(name); ok { - ver = m.Version + hash, hashErr := recipeHashFor(v, m) + if hashErr != nil { + hash, _ = recipes.ScriptHash(name, v.OS) + } + scriptHash, _ := recipes.ScriptHash(name, v.OS) + values, undeclared := sshx.ParseOutputs(m.Outputs, redactCloudSecrets(body, secrets[name])) + for _, output := range undeclared { + warnings = append(warnings, fmt.Sprintf("%s: output %q is not declared", name, output)) } if applied == nil { applied = make(map[string]config.AppliedRecipe) } - applied[name] = config.AppliedRecipe{Version: ver, Hash: hash, At: time.Now()} + applied[name] = config.AppliedRecipe{ + Version: m.Version, Hash: hash, ScriptHash: scriptHash, + At: time.Now(), Outputs: values, Health: string(HealthUnknown), + } } if applied == nil { - return nil + return warnings, nil } v.Applied = applied - return v.Save() + if err := v.Save(); err != nil { + return warnings, err + } + return warnings, nil +} + +func cloudInitOutputs(body string) map[string]string { + out := map[string]string{} + var name string + var value strings.Builder + for _, line := range strings.Split(body, "\n") { + if strings.HasPrefix(line, "===") { + if name != "" { + out[name] = value.String() + } + name = strings.TrimSpace(strings.TrimPrefix(line, "===")) + value.Reset() + continue + } + if name != "" { + value.WriteString(line) + value.WriteByte('\n') + } + } + if name != "" { + out[name] = value.String() + } + return out +} + +func redactCloudSecrets(value string, secrets map[string]string) string { + names := make([]string, 0, len(secrets)) + for _, secret := range secrets { + if secret != "" { + names = append(names, secret) + } + } + sort.Slice(names, func(i, j int) bool { return len(names[i]) > len(names[j]) }) + for _, secret := range names { + value = strings.ReplaceAll(value, secret, "") + } + return value } // filterByRunMode narrows targets to the recipes that should actually run, @@ -427,13 +522,16 @@ func planRecipes(v *config.VM, targets []string, explicit map[string]bool) ([]re } case "once": if applied, done := v.Applied[name]; done { - hash, err := recipes.ScriptHash(name, v.OS) + hash, err := recipeHashFor(v, m) if err != nil { return nil, nil, err } - if applied.Hash == hash { + switch { + case applied.Hash == hash: run, reason = false, "already applied" - } else { + case scriptUnchanged(v, m, applied): + reason = "params changed" + default: reason = "script changed" } } else { @@ -458,6 +556,50 @@ func planRecipes(v *config.VM, targets []string, explicit map[string]bool) ([]re return decisions, manifests, nil } +// recipeHashFor computes the current combined hash for a VM's recipe. Only +// declared secret parameters with non-empty stored values affect the hash; +// stale keys in secrets.toml do not. +func recipeHashFor(v *config.VM, m recipes.Manifest) (string, error) { + if len(m.Params) == 0 { + return recipes.ScriptHash(m.Name, v.OS) + } + secrets, err := config.LoadSecrets(v.Dir) + if err != nil { + return "", err + } + params, err := recipes.Resolve(m, v.Params[m.Name], secrets[m.Name]) + if err != nil { + return "", err + } + nonSecret := make(map[string]string, len(params)) + for name, value := range params { + if m.Params[name].Type != "secret" { + nonSecret[name] = value + } + } + setSecrets := make(map[string]bool) + for _, name := range m.SecretNames() { + if secrets[m.Name][name] != "" { + setSecrets[name] = true + } + } + secretNames := make([]string, 0, len(setSecrets)) + for name := range setSecrets { + secretNames = append(secretNames, name) + } + return recipes.RecipeHash(m.Name, v.OS, nonSecret, secretNames) +} + +// scriptUnchanged distinguishes a parameter change from a changed recipe +// body. Entries written before ScriptHash existed cannot make that claim. +func scriptUnchanged(v *config.VM, m recipes.Manifest, applied config.AppliedRecipe) bool { + if applied.ScriptHash == "" { + return false + } + body, err := recipes.ScriptHash(m.Name, v.OS) + return err == nil && body == applied.ScriptHash +} + // dependencyError explains why dependent cannot run: its dependency dep is // neither running this pass nor already applied. A dep that is a configured // "manual" recipe was skipped because nobody named it; anything else is a dep @@ -475,6 +617,12 @@ func dependencyError(dependent, dep string, manifests map[string]recipes.Manifes type Recipe struct { Name string // recipe name, matches the directory name Description string // from recipe.toml + // Schema is the recipe.toml format version exposed to machine callers. + Schema int + // Params, Outputs and Health describe the recipe contract. + Params []RecipeParam + Outputs []RecipeOutput + Health *RecipeHealthSpec // Reboot says the guest needs a restart before this recipe's effect is // visible. A caller that waits for "reachable" after an apply sees the // pre-reboot sshd and reads it as done. @@ -487,6 +635,68 @@ type Recipe struct { Runtime string } +// RecipeParam is one declared recipe parameter. +type RecipeParam struct { + Name string + Type string + Default string + Help string + Required bool + Values []string +} + +// RecipeOutput is one declared recipe output. +type RecipeOutput struct { + Name string + Help string +} + +// RecipeHealthSpec is a recipe's declared health check. +type RecipeHealthSpec struct { + Check string + Timeout string +} + +// RecipeShow is the host-side lookup for one recipe's contract. +func RecipeShow(name string) (Recipe, error) { + m, ok, err := recipes.ManifestFor(name) + if err != nil { + return Recipe{}, err + } + if !ok { + return Recipe{}, fmt.Errorf("%w: no such recipe %q", ErrNotFound, name) + } + return fromManifest(m), nil +} + +// fromManifest is the one projection shared by recipe list and recipe show. +// Keeping the conversion here prevents the two caller surfaces from growing +// different views of the same manifest over time. +func fromManifest(m recipes.Manifest) Recipe { + r := Recipe{ + Name: m.Name, Description: m.Description, Schema: m.Schema, + Reboot: m.Reboot, Depends: m.Depends, Runtime: m.Runtime, + Params: []RecipeParam{}, Outputs: []RecipeOutput{}, + } + for _, p := range m.SortedParams() { + r.Params = append(r.Params, RecipeParam{ + Name: p.Name, Type: p.Type, Default: p.Default, Help: p.Help, + Required: p.Required, Values: append([]string{}, p.Values...), + }) + } + for _, o := range m.SortedOutputs() { + r.Outputs = append(r.Outputs, RecipeOutput{Name: o.Name, Help: o.Help}) + } + if m.Health.Check != "" { + timeout := m.Health.Timeout + if timeout == "" { + timeout = recipes.DefaultHealthTimeout.String() + } + r.Health = &RecipeHealthSpec{Check: m.Health.Check, Timeout: timeout} + } + return r +} + // RecipeFilter selects the recipes Recipes returns: the set // recipes.List(OS, Backend) would offer a VM with that OS and backend. // Backend is accepted for API compatibility but ignored in v2 (all recipes @@ -508,13 +718,7 @@ func Recipes(f RecipeFilter) ([]Recipe, error) { var out []Recipe for _, m := range manifests { if recipes.MatchesVM(&m, f.OS) { - out = append(out, Recipe{ - Name: m.Name, - Description: m.Description, - Reboot: m.Reboot, - Depends: m.Depends, - Runtime: m.Runtime, - }) + out = append(out, fromManifest(m)) } } return out, nil diff --git a/internal/core/apply_reboot_test.go b/internal/core/apply_reboot_test.go index e83aef67..a8f2d73c 100644 --- a/internal/core/apply_reboot_test.go +++ b/internal/core/apply_reboot_test.go @@ -88,6 +88,79 @@ func TestApplyRebootsAfterARecipeThatDeclaresIt(t *testing.T) { } } +func TestApplyRunsHealthOnlyAfterRebootAndReachability(t *testing.T) { + dir := root(t) + writeSchema3RebootHealthRecipe(t, dir, "xfce") + sequence := filepath.Join(dir, "ssh-sequence") + installSequenceSSH(t, sequence) + + port, stop := fakeSSHD(t, 0) + defer stop() + v := &config.VM{ + Name: "work", Mode: "disk", OS: "alpine", Backend: "apkovl", Installed: true, + RAM: 512, CPUs: 1, SSHPort: port, Recipes: []string{"xfce"}, + } + if err := v.Save(); err != nil { + t.Fatal(err) + } + defer fakeRunning(t, v)() + + if err := Apply(context.Background(), v.Name, ApplyOpts{}); err != nil { + t.Fatalf("Apply: %v", err) + } + events, err := os.ReadFile(sequence) + if err != nil { + t.Fatal(err) + } + lines := strings.Fields(string(events)) + index := func(want string) int { + for i, line := range lines { + if line == want { + return i + } + } + return -1 + } + recipeAt, rebootAt, healthAt := index("recipe"), index("reboot"), index("health") + if recipeAt < 0 || rebootAt < 0 || healthAt < 0 { + t.Fatalf("ssh sequence = %v, want recipe, reboot, and health", lines) + } + if !(recipeAt < rebootAt && rebootAt < healthAt) { + t.Fatalf("ssh sequence = %v, want recipe < reboot < health", lines) + } +} + +func writeSchema3RebootHealthRecipe(t *testing.T, rootDir, name string) { + t.Helper() + d := filepath.Join(rootDir, "recipes", name) + if err := os.MkdirAll(d, 0o755); err != nil { + t.Fatal(err) + } + manifest := "schema = 3\nname = \"" + name + "\"\nversion = \"1.0\"\nscript = \"install.sh\"\nrun = \"always\"\nreboot = true\n\n[health]\ncheck = \"health-check\"\n" + if err := os.WriteFile(filepath.Join(d, "recipe.toml"), []byte(manifest), 0o644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(d, "install.sh"), []byte("#!/bin/sh\necho hi\n"), 0o755); err != nil { + t.Fatal(err) + } +} + +func installSequenceSSH(t *testing.T, sequence string) { + t.Helper() + bin := t.TempDir() + script := "#!/bin/sh\n" + + "last=\"\"\nfor a in \"$@\"; do last=\"$a\"; done\n" + + "if [ \"$last\" = reboot ]; then printf 'reboot\\n' >> " + shellQuoteCoreTest(sequence) + "; exit 0; fi\n" + + "input=$(cat)\n" + + "case \"$*\" in *'cat /tmp/.stoat-out/xfce'*) printf 'outputs\\n' >> " + shellQuoteCoreTest(sequence) + "; exit 0;; esac\n" + + "case \"$input\" in *'health-check'*) printf 'health\\n' >> " + shellQuoteCoreTest(sequence) + "; exit 0;; *'STOAT_RECIPE=xfce'*) printf 'recipe\\n' >> " + shellQuoteCoreTest(sequence) + "; exit 0;; *'stoat_pkg_setup'*) printf 'setup\\n' >> " + shellQuoteCoreTest(sequence) + "; exit 0;; esac\n" + + "printf 'other\\n' >> " + shellQuoteCoreTest(sequence) + "\n" + if err := os.WriteFile(filepath.Join(bin, "ssh"), []byte(script), 0o755); err != nil { + t.Fatal(err) + } + t.Setenv("PATH", bin+string(os.PathListSeparator)+os.Getenv("PATH")) +} + // TestApplyDoesNotRebootALiveVM pins the mode gate: a live VM's root is a // tmpfs the reboot wipes, and a live VM re-applies every boot, so a reboot // here would loop. A reboot=true recipe on a live VM reboots nothing. diff --git a/internal/core/apply_test.go b/internal/core/apply_test.go index e40811a7..1423ea96 100644 --- a/internal/core/apply_test.go +++ b/internal/core/apply_test.go @@ -697,3 +697,101 @@ func TestCheckRecipesReportsCapabilityMismatch(t *testing.T) { t.Errorf("Reason = %q, want it to contain %q", issues[0].Reason, want) } } + +func TestRecipesProjectsSchema3ContractInSortedOrder(t *testing.T) { + dir := root(t) + recipeDir := filepath.Join(dir, "recipes", "docker") + if err := os.MkdirAll(recipeDir, 0o755); err != nil { + t.Fatal(err) + } + manifest := `schema = 3 +name = "docker" +description = "Docker engine" +os = ["alpine"] +script = "install.sh" +runtime = "sh" +depends = ["base"] + +[params.zeta] +type = "string" +default = "z" + +[params.alpha] +type = "int" +default = 2375 + +[outputs] +z-socket = "z" +socket = "socket" + +[health] +check = "docker info" +timeout = "2s" +` + if err := os.WriteFile(filepath.Join(recipeDir, "recipe.toml"), []byte(manifest), 0o644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(recipeDir, "install.sh"), []byte("#!/bin/sh\n"), 0o755); err != nil { + t.Fatal(err) + } + got, err := Recipes(RecipeFilter{OS: "alpine", Backend: "apkovl"}) + if err != nil { + t.Fatal(err) + } + var docker Recipe + for _, recipe := range got { + if recipe.Name == "docker" { + docker = recipe + break + } + } + if docker.Name == "" { + t.Fatalf("Recipes omitted docker: %+v", got) + } + if docker.Schema != 3 || docker.Description != "Docker engine" || docker.Health == nil || docker.Health.Check != "docker info" { + t.Fatalf("docker contract = %+v, want schema/description/health projection", docker) + } + if len(docker.Params) != 2 || docker.Params[0].Name != "alpha" || docker.Params[1].Name != "zeta" { + t.Fatalf("params = %+v, want sorted [alpha zeta]", docker.Params) + } + if len(docker.Outputs) != 2 || docker.Outputs[0].Name != "socket" || docker.Outputs[1].Name != "z-socket" { + t.Fatalf("outputs = %+v, want sorted [socket z-socket]", docker.Outputs) + } +} + +func TestRecipeShowProjectsNamedManifestContract(t *testing.T) { + dir := root(t) + recipeDir := filepath.Join(dir, "recipes", "docker") + if err := os.MkdirAll(recipeDir, 0o755); err != nil { + t.Fatal(err) + } + manifest := `schema = 3 +name = "docker" +description = "Docker engine" +script = "install.sh" + +[params.authkey] +type = "secret" +required = true + +[outputs] +socket = "path" + +[health] +check = "docker info" +timeout = "30s" +` + if err := os.WriteFile(filepath.Join(recipeDir, "recipe.toml"), []byte(manifest), 0o644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(recipeDir, "install.sh"), []byte("#!/bin/sh\n"), 0o755); err != nil { + t.Fatal(err) + } + got, err := RecipeShow("docker") + if err != nil { + t.Fatal(err) + } + if got.Name != "docker" || got.Schema != 3 || len(got.Params) != 1 || got.Params[0].Name != "authkey" || len(got.Outputs) != 1 || got.Health == nil { + t.Fatalf("RecipeShow = %+v, want named schema contract", got) + } +} diff --git a/internal/core/cloudinit_recipe_test.go b/internal/core/cloudinit_recipe_test.go new file mode 100644 index 00000000..fe96b767 --- /dev/null +++ b/internal/core/cloudinit_recipe_test.go @@ -0,0 +1,90 @@ +package core + +import ( + "context" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/novusedge/stoat/internal/config" +) + +func TestApplyDiscoversCloudInitOutputsAndSkipsTheRecipe(t *testing.T) { + dir := root(t) + recipeDir := filepath.Join(dir, "recipes", "docker") + if err := os.MkdirAll(recipeDir, 0o755); err != nil { + t.Fatal(err) + } + manifest := "schema = 3\nname = \"docker\"\nscript = \"install.sh\"\n" + if err := os.WriteFile(filepath.Join(recipeDir, "recipe.toml"), []byte(manifest), 0o644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(recipeDir, "install.sh"), []byte("#!/bin/sh\necho should-not-run\n"), 0o755); err != nil { + t.Fatal(err) + } + + vmDir := filepath.Join(dir, "cloudy") + port, stopSSH := fakeSSHD(t, 0) + defer stopSSH() + v := &config.VM{ + Name: "cloudy", Dir: vmDir, Mode: "cloud", Backend: "cloudinit", OS: "alpine", + RAM: 512, CPUs: 1, SSHPort: port, Recipes: []string{"docker"}, + } + if err := v.Save(); err != nil { + t.Fatal(err) + } + const secret = "cloud-discovery-secret" + if err := config.SaveSecrets(vmDir, config.Secrets{"docker": {"authkey": secret}}); err != nil { + t.Fatal(err) + } + defer fakeRunning(t, v)() + count := filepath.Join(vmDir, "recipe-count") + installCloudMarkerSSH(t, count, secret) + + if err := Apply(context.Background(), v.Name, ApplyOpts{}); err != nil { + t.Fatal(err) + } + got, err := config.Load(v.Name) + if err != nil { + t.Fatal(err) + } + outputs := got.Applied["docker"].Outputs + if outputs["rogue"] != "" || outputs["empty"] != "" { + t.Fatalf("cloud-init outputs = %v, want redacted rogue and empty output", outputs) + } + vmToml, err := os.ReadFile(filepath.Join(vmDir, "vm.toml")) + if err != nil { + t.Fatal(err) + } + if strings.Contains(string(vmToml), secret) { + t.Fatalf("cloud-init output secret leaked into vm.toml: %s", vmToml) + } + applyLog, err := os.ReadFile(v.ProvisionLogPath()) + if err != nil { + t.Fatalf("apply log missing discovery warnings: %v", err) + } + for _, name := range []string{"rogue", "empty"} { + if !strings.Contains(string(applyLog), `docker: output "`+name+`" is not declared`) { + t.Errorf("apply log missing undeclared %s warning: %s", name, applyLog) + } + } + b, err := os.ReadFile(count) + if err == nil && strings.Contains(string(b), "STOAT_RECIPE=docker") { + t.Fatalf("cloud-init marker discovery reran the already-applied recipe: %s", b) + } +} + +func installCloudMarkerSSH(t *testing.T, count, secret string) { + t.Helper() + bin := t.TempDir() + script := "#!/bin/sh\ninput=$(cat)\ncase \"$*\" in *'.applied'*) printf '===docker\\nrogue=%s\\nempty=\\n' " + shellQuoteCoreTest(secret) + ";; esac\ncase \"$input\" in *'STOAT_RECIPE=docker'*) printf '%s\\n' \"$input\" >> " + shellQuoteCoreTest(count) + ";; esac\nexit 0\n" + if err := os.WriteFile(filepath.Join(bin, "ssh"), []byte(script), 0o755); err != nil { + t.Fatal(err) + } + t.Setenv("PATH", bin+string(os.PathListSeparator)+os.Getenv("PATH")) +} + +func shellQuoteCoreTest(s string) string { + return "'" + strings.ReplaceAll(s, "'", `\'\''`) + "'" +} diff --git a/internal/core/core.go b/internal/core/core.go index ef344fb3..bf1b6948 100644 --- a/internal/core/core.go +++ b/internal/core/core.go @@ -67,6 +67,8 @@ type Spec struct { Disk string // qemu-img size, absolute only ("8G", never "+8G") Share string Recipes []string + Params map[string]map[string]string + Secrets config.Secrets // Display is the screen preference to record in vm.toml: "" or "auto" // (default), "window", or "vnc". validateDisplay is the single check @@ -121,6 +123,16 @@ func Create(s Spec) (VM, error) { if err := v.Save(); err != nil { return VM{}, err } + if err := applyParamEdits(v, Patch{SetParams: s.Params, Secrets: s.Secrets}); err != nil { + _ = os.RemoveAll(v.Dir) + return VM{}, err + } + if len(s.Params) > 0 { + if err := v.Save(); err != nil { + _ = os.RemoveAll(v.Dir) + return VM{}, err + } + } if v.Mode == "disk" { out, err := exec.Command("qemu-img", "create", "-f", "qcow2", v.DiskPath(), v.Disk).CombinedOutput() if err != nil { diff --git a/internal/core/health.go b/internal/core/health.go index 5913e8f5..5e432d5a 100644 --- a/internal/core/health.go +++ b/internal/core/health.go @@ -1,5 +1,16 @@ package core +import ( + "context" + "fmt" + "strings" + "time" + + "github.com/novusedge/stoat/internal/config" + "github.com/novusedge/stoat/internal/recipes" + "github.com/novusedge/stoat/internal/sshx" +) + // Health is a recipe's health-check result. The recipe contract writes the // checks that report it; this declares the values they may report. type Health string @@ -12,3 +23,104 @@ const ( // Healths returns every declared health value. func Healths() []Health { return []Health{HealthOK, HealthFailed, HealthUnknown} } + +// RecipeHealth is one recipe health verdict. +type RecipeHealth struct { + Name string + Status Health + Detail string +} + +// healthChecksForVM runs checks for the named recipes in order and records +// each verdict on an existing applied entry. It does not save v. If ctx ends +// during a later check, completed verdicts are returned with the context error +// so callers can preserve an earlier failure's actionable detail. +func healthChecksForVM(ctx context.Context, v *config.VM, names []string) ([]RecipeHealth, error) { + out := make([]RecipeHealth, 0, len(names)) + for _, name := range names { + if err := ctx.Err(); err != nil { + return out, err + } + m, ok, err := recipes.ManifestFor(name) + if err != nil { + return out, err + } + verdict := RecipeHealth{Name: name, Status: HealthUnknown} + if ok && m.Health.Check != "" { + text, runErr := sshx.RunCheck(ctx, v, m.Health.Check, m.Health.Duration()) + if runErr != nil { + stored, loadErr := config.LoadSecrets(v.Dir) + if loadErr != nil { + if err := ctx.Err(); err != nil { + return out, err + } + return out, loadErr + } + verdict.Status = HealthFailed + detail := redactCloudSecrets(text, stored[name]) + verdict.Detail = fmt.Sprintf("health check failed after %s: %s", m.Health.Duration(), lastLine(detail)) + } else { + verdict.Status = HealthOK + } + } + if a, recorded := v.Applied[name]; recorded { + a.Health = string(verdict.Status) + v.Applied[name] = a + } + out = append(out, verdict) + if err := ctx.Err(); err != nil { + return out, err + } + } + return out, nil +} + +// HealthChecks checks the named VM's applied recipes in configured order. +func HealthChecks(ctx context.Context, name string) ([]RecipeHealth, error) { + v, err := load(name) + if err != nil { + return nil, err + } + names := make([]string, 0, len(v.Applied)) + for _, recipe := range v.Recipes { + if _, ok := v.Applied[recipe]; ok { + names = append(names, recipe) + } + } + return healthChecksForVM(ctx, v, names) +} + +// VMHealth folds verdicts into one VM result. +func VMHealth(rs []RecipeHealth) Health { + status := HealthUnknown + for _, result := range rs { + if result.Status == HealthFailed { + return HealthFailed + } + if result.Status == HealthOK { + status = HealthOK + } + } + return status +} + +// HealthTimeout is the longest declared health check among applied recipes. +func HealthTimeout(v *config.VM) time.Duration { + var longest time.Duration + for name := range v.Applied { + if m, ok, _ := recipes.ManifestFor(name); ok && m.Health.Check != "" && m.Health.Duration() > longest { + longest = m.Health.Duration() + } + } + return longest +} + +func lastLine(s string) string { + lines := strings.Split(strings.TrimRight(s, "\n"), "\n") + for i := len(lines) - 1; i >= 0; i-- { + if line := strings.TrimSpace(lines[i]); line != "" { + return line + } + } + return "" +} diff --git a/internal/core/health_test.go b/internal/core/health_test.go new file mode 100644 index 00000000..b302dd36 --- /dev/null +++ b/internal/core/health_test.go @@ -0,0 +1,218 @@ +package core + +import ( + "context" + "errors" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/novusedge/stoat/internal/config" +) + +func TestVMHealthFolds(t *testing.T) { + tests := []struct { + name string + in []RecipeHealth + want Health + }{ + {"empty", nil, HealthUnknown}, + {"all unknown", []RecipeHealth{{Status: HealthUnknown}}, HealthUnknown}, + {"one ok", []RecipeHealth{{Status: HealthOK}, {Status: HealthUnknown}}, HealthOK}, + {"one failed", []RecipeHealth{{Status: HealthOK}, {Status: HealthFailed}}, HealthFailed}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + if got := VMHealth(tt.in); got != tt.want { + t.Errorf("VMHealth = %q, want %q", got, tt.want) + } + }) + } +} + +func TestApplyFailsOnHealthCheckAndPersistsResult(t *testing.T) { + dir := root(t) + writeHealthRecipeWithOutput(t, dir) + const secret = "health-output-secret" + port, stopSSH := fakeSSHD(t, 0) + defer stopSSH() + v := &config.VM{ + Name: "work", Mode: "live", OS: "alpine", Backend: "apkovl", + RAM: 512, CPUs: 1, SSHPort: port, Recipes: []string{"docker"}, + } + if err := v.Save(); err != nil { + t.Fatal(err) + } + if err := config.SaveSecrets(v.Dir, config.Secrets{"docker": {"authkey": secret}}); err != nil { + t.Fatal(err) + } + defer fakeRunning(t, v)() + installHealthSSH(t, false, t.TempDir()) + + err := Apply(context.Background(), v.Name, ApplyOpts{}) + if err == nil { + t.Fatal("Apply succeeded with a failing health check") + } + if !strings.Contains(err.Error(), "docker: health check failed after 30s: cannot connect to the docker daemon") { + t.Errorf("err = %v", err) + } + got, err := config.Load(v.Name) + if err != nil { + t.Fatal(err) + } + if got.Applied["docker"].Health != string(HealthFailed) { + t.Errorf("health = %q, want failed", got.Applied["docker"].Health) + } + if got.Applied["docker"].Outputs["captured"] != "" { + t.Errorf("captured output = %q, want redacted", got.Applied["docker"].Outputs["captured"]) + } + vmToml, err := os.ReadFile(filepath.Join(got.Dir, "vm.toml")) + if err != nil { + t.Fatal(err) + } + if strings.Contains(string(vmToml), secret) { + t.Fatalf("health output secret leaked into vm.toml: %s", vmToml) + } + plan, err := PlanApply(v.Name, ApplyOpts{}) + if err != nil { + t.Fatal(err) + } + if len(plan) != 1 || plan[0].Action != "skip" { + t.Fatalf("plan after failed health = %+v, want skip", plan) + } +} + +func TestApplyWithoutHealthCheckRecordsUnknown(t *testing.T) { + dir := root(t) + writeHealthRecipe(t, dir, false) + port, stopSSH := fakeSSHD(t, 0) + defer stopSSH() + v := &config.VM{ + Name: "work", Mode: "live", OS: "alpine", Backend: "apkovl", + RAM: 512, CPUs: 1, SSHPort: port, Recipes: []string{"docker"}, + } + if err := v.Save(); err != nil { + t.Fatal(err) + } + defer fakeRunning(t, v)() + installHealthSSH(t, true) + + if err := Apply(context.Background(), v.Name, ApplyOpts{}); err != nil { + t.Fatal(err) + } + got, err := config.Load(v.Name) + if err != nil { + t.Fatal(err) + } + if got.Applied["docker"].Health != string(HealthUnknown) { + t.Errorf("health = %q, want unknown", got.Applied["docker"].Health) + } +} + +func TestHealthChecksPropagatesCancellation(t *testing.T) { + dir := root(t) + writeHealthRecipe(t, dir, true) + v := &config.VM{Name: "work", Dir: filepath.Join(dir, "work"), OS: "alpine", Recipes: []string{"docker"}, Applied: map[string]config.AppliedRecipe{"docker": {}}} + if err := v.Save(); err != nil { + t.Fatal(err) + } + ctx, cancel := context.WithCancel(context.Background()) + cancel() + _, err := HealthChecks(ctx, v.Name) + if !errors.Is(err, context.Canceled) { + t.Fatalf("HealthChecks error = %v, want context.Canceled", err) + } +} + +func TestRecipeHealthTimeoutKeepsRawDeclarationAcrossListAndShow(t *testing.T) { + dir := root(t) + writeHealthRecipeWithTimeoutNamed(t, dir, "health-sixty", "60s") + writeHealthRecipeWithTimeoutNamed(t, dir, "health-default", "") + + shown, err := RecipeShow("health-sixty") + if err != nil { + t.Fatal(err) + } + if shown.Health == nil || shown.Health.Timeout != "60s" { + t.Fatalf("RecipeShow health = %+v, want raw 60s timeout", shown.Health) + } + defaultShown, err := RecipeShow("health-default") + if err != nil { + t.Fatal(err) + } + if defaultShown.Health == nil || defaultShown.Health.Timeout != "30s" { + t.Fatalf("RecipeShow omitted timeout = %+v, want 30s", defaultShown.Health) + } + + listed, err := Recipes(RecipeFilter{OS: "alpine"}) + if err != nil { + t.Fatal(err) + } + seen := map[string]RecipeHealthSpec{} + for _, recipe := range listed { + if recipe.Health != nil { + seen[recipe.Name] = *recipe.Health + } + } + if got := seen["health-sixty"].Timeout; got != "60s" { + t.Errorf("Recipes health-sixty timeout = %q, want 60s", got) + } + if got := seen["health-default"].Timeout; got != "30s" { + t.Errorf("Recipes health-default timeout = %q, want 30s", got) + } +} + +func writeHealthRecipe(t *testing.T, rootDir string, withHealth bool) { + t.Helper() + d := filepath.Join(rootDir, "recipes", "docker") + if err := os.MkdirAll(d, 0o755); err != nil { + t.Fatal(err) + } + manifest := "schema = 3\nname = \"docker\"\nscript = \"install.sh\"\n" + if withHealth { + manifest += "\n[health]\ncheck = \"docker info\"\n" + } + if err := os.WriteFile(filepath.Join(d, "recipe.toml"), []byte(manifest), 0o644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(d, "install.sh"), []byte("#!/bin/sh\necho provisioned\n"), 0o755); err != nil { + t.Fatal(err) + } +} + +func writeHealthRecipeWithOutput(t *testing.T, rootDir string) { + t.Helper() + d := filepath.Join(rootDir, "recipes", "docker") + if err := os.MkdirAll(d, 0o755); err != nil { + t.Fatal(err) + } + manifest := "schema = 3\nname = \"docker\"\nscript = \"install.sh\"\n\n[params.authkey]\ntype = \"secret\"\nrequired = true\n\n[health]\ncheck = \"docker info\"\n" + if err := os.WriteFile(filepath.Join(d, "recipe.toml"), []byte(manifest), 0o644); err != nil { + t.Fatal(err) + } + script := "#!/bin/sh\nprintf 'captured=%s\\n' \"$STOAT_PARAM_AUTHKEY\" > \"$STOAT_OUTPUT\"\n" + if err := os.WriteFile(filepath.Join(d, "install.sh"), []byte(script), 0o755); err != nil { + t.Fatal(err) + } +} + +func installHealthSSH(t *testing.T, succeedHealth bool, outputRoots ...string) { + t.Helper() + bin := t.TempDir() + check := "echo 'cannot connect to the docker daemon' >&2\nexit 1" + if succeedHealth { + check = "exit 0" + } + script := "#!/bin/sh\ninput=$(cat)\ncase \"$input\" in *'docker info'*)\n" + check + "\n;; esac\n" + if len(outputRoots) > 0 { + root := strings.ReplaceAll(outputRoots[0], "#", "\\#") + script += "case \"$input\" in *'STOAT_RECIPE=docker'*) safe=$(printf '%s' \"$input\" | sed 's#/tmp/.stoat-out#" + root + "#g'); printf '%s' \"$safe\" | sh -s; exit $?;; esac\n" + script += "case \"$*\" in *'cat /tmp/.stoat-out/docker'*) cat " + filepath.Join(outputRoots[0], "docker") + ";; esac\n" + } + script += "exit 0\n" + if err := os.WriteFile(filepath.Join(bin, "ssh"), []byte(script), 0o755); err != nil { + t.Fatal(err) + } + t.Setenv("PATH", bin+string(os.PathListSeparator)+os.Getenv("PATH")) +} diff --git a/internal/core/recipe_params_test.go b/internal/core/recipe_params_test.go new file mode 100644 index 00000000..12c763da --- /dev/null +++ b/internal/core/recipe_params_test.go @@ -0,0 +1,264 @@ +package core + +import ( + "bytes" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/novusedge/stoat/internal/config" + "github.com/novusedge/stoat/internal/recipes" +) + +const paramRecipeManifest = `schema = 3 +name = "docker" +script = "install.sh" + +[params.user] +type = "string" +default = "dev" + +[params.port] +type = "int" +default = 2375 + +[params.authkey] +type = "secret" +required = true +` + +func writeParamRecipe(t *testing.T, rootDir string) { + t.Helper() + d := filepath.Join(rootDir, "recipes", "docker") + if err := os.MkdirAll(d, 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(d, "recipe.toml"), []byte(paramRecipeManifest), 0o644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(d, "install.sh"), []byte("#!/bin/sh\necho provisioned\n"), 0o755); err != nil { + t.Fatal(err) + } +} + +func TestPlanApplyReportsParamsChangedAndIgnoresSecretValueChanges(t *testing.T) { + dir := root(t) + writeParamRecipe(t, dir) + + scriptHash, err := recipes.ScriptHash("docker", "alpine") + if err != nil { + t.Fatal(err) + } + params := map[string]string{"user": "dev", "port": "2375"} + combined, err := recipes.RecipeHash("docker", "alpine", params, []string{"authkey"}) + if err != nil { + t.Fatal(err) + } + v := &config.VM{ + Name: "work", Mode: "live", OS: "alpine", Backend: "apkovl", + RAM: 512, CPUs: 1, SSHPort: 2200, + Recipes: []string{"docker"}, Params: map[string]map[string]string{"docker": params}, + Applied: map[string]config.AppliedRecipe{"docker": { + Version: "1.0", Hash: combined, ScriptHash: scriptHash, + }}, + } + if err := v.Save(); err != nil { + t.Fatal(err) + } + if err := config.SaveSecrets(v.Dir, config.Secrets{"docker": {"authkey": "first-secret", "stale": "stale-secret"}}); err != nil { + t.Fatal(err) + } + + plan, err := PlanApply(v.Name, ApplyOpts{}) + if err != nil { + t.Fatal(err) + } + if len(plan) != 1 || plan[0].Action != "skip" { + t.Fatalf("initial plan = %+v, want skip", plan) + } + + if err := config.SaveSecrets(v.Dir, config.Secrets{"docker": {"authkey": "changed-secret", "stale": "changed-stale-secret"}}); err != nil { + t.Fatal(err) + } + plan, err = PlanApply(v.Name, ApplyOpts{}) + if err != nil { + t.Fatal(err) + } + if len(plan) != 1 || plan[0].Action != "skip" { + t.Fatalf("secret-only plan = %+v, want skip", plan) + } + reloaded, err := config.Load(v.Name) + if err != nil { + t.Fatal(err) + } + if reloaded.Applied["docker"].Hash != combined { + t.Fatalf("stale undeclared secret changed applied hash to %q, want %q", reloaded.Applied["docker"].Hash, combined) + } + + v.SetParam("docker", "user", "bob") + if err := v.Save(); err != nil { + t.Fatal(err) + } + plan, err = PlanApply(v.Name, ApplyOpts{}) + if err != nil { + t.Fatal(err) + } + if len(plan) != 1 || plan[0].Action != "run" || plan[0].Reason != "params changed" { + t.Fatalf("non-secret change plan = %+v, want run/params changed", plan) + } +} + +func TestCreatePersistsParamsAndSecretsUnderTheVMDirectory(t *testing.T) { + dir := root(t) + haveImage(t, dir, "alpine-virt-3.24.1-x86_64.iso") + writeParamRecipe(t, dir) + + _, err := Create(Spec{ + Name: "work", Image: "alpine-virt-3.24.1-x86_64.iso", Recipes: []string{"docker"}, + Params: map[string]map[string]string{"docker": {"user": "alice"}}, + Secrets: config.Secrets{"docker": {"authkey": "secret-value"}}, + }) + if err != nil { + t.Fatal(err) + } + + v, err := config.Load("work") + if err != nil { + t.Fatal(err) + } + if got, ok := v.Param("docker", "user"); !ok || got != "alice" { + t.Errorf("stored user = %q/%v, want alice/true", got, ok) + } + contents, err := os.ReadFile(filepath.Join(v.Dir, "vm.toml")) + if err != nil { + t.Fatal(err) + } + if strings.Contains(string(contents), "secret-value") { + t.Fatal("vm.toml contains the secret value") + } + secrets, err := config.LoadSecrets(v.Dir) + if err != nil { + t.Fatal(err) + } + if got := secrets["docker"]["authkey"]; got != "secret-value" { + t.Errorf("stored secret = %q, want secret-value", got) + } + info, err := os.Stat(v.SecretsPath()) + if err != nil { + t.Fatal(err) + } + if got := info.Mode().Perm(); got != 0o600 { + t.Errorf("secrets mode = %#o, want 0600", got) + } +} + +func TestUpdateValidatesAndUnsetsSecretAndNonSecretParams(t *testing.T) { + dir := root(t) + haveImage(t, dir, "alpine-virt-3.24.1-x86_64.iso") + writeParamRecipe(t, dir) + if _, err := Create(Spec{ + Name: "work", Image: "alpine-virt-3.24.1-x86_64.iso", Recipes: []string{"docker"}, + Params: map[string]map[string]string{"docker": {"user": "alice", "port": "2375"}}, + Secrets: config.Secrets{"docker": {"authkey": "old-secret"}}, + }); err != nil { + t.Fatal(err) + } + + if _, err := Update("work", Patch{ + SetParams: map[string]map[string]string{"docker": {"user": "bob"}}, + UnsetParams: map[string][]string{"docker": {"port", "authkey"}}, + }); err != nil { + t.Fatal(err) + } + v, err := config.Load("work") + if err != nil { + t.Fatal(err) + } + if got, ok := v.Param("docker", "user"); !ok || got != "bob" { + t.Errorf("user = %q/%v, want bob/true", got, ok) + } + if _, ok := v.Param("docker", "port"); ok { + t.Error("unset non-secret port remained in vm.toml") + } + secrets, err := config.LoadSecrets(v.Dir) + if err != nil { + t.Fatal(err) + } + if _, ok := secrets["docker"]["authkey"]; ok { + t.Error("unset secret authkey remained in secrets.toml") + } + + if _, err := Update("work", Patch{UnsetParams: map[string][]string{"docker": {"missing"}}}); err == nil { + t.Fatal("unknown parameter unset was accepted") + } + v, err = config.Load("work") + if err != nil { + t.Fatal(err) + } + if got, ok := v.Param("docker", "user"); !ok || got != "bob" { + t.Errorf("failed update changed user = %q/%v", got, ok) + } +} + +func TestUpdateSecretEditRollsBackWhenLaterFieldIsInvalid(t *testing.T) { + dir := root(t) + haveImage(t, dir, "alpine-virt-3.24.1-x86_64.iso") + writeParamRecipe(t, dir) + cases := []struct { + name string + patch func(*string) Patch + }{ + {name: "display", patch: func(secret *string) Patch { + return Patch{Secrets: config.Secrets{"docker": {"authkey": *secret}}, Display: ptr("invalid")} + }}, + {name: "ssh port", patch: func(secret *string) Patch { + return Patch{Secrets: config.Secrets{"docker": {"authkey": *secret}}, SSHPort: ptr(80)} + }}, + {name: "disk", patch: func(secret *string) Patch { + return Patch{Secrets: config.Secrets{"docker": {"authkey": *secret}}, Disk: ptr("not-a-size")} + }}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + name := "rollback-" + strings.ReplaceAll(tc.name, " ", "-") + if _, err := Create(Spec{ + Name: name, Image: "alpine-virt-3.24.1-x86_64.iso", Recipes: []string{"docker"}, + Params: map[string]map[string]string{"docker": {"user": "alice"}}, + Secrets: config.Secrets{"docker": {"authkey": "old-secret"}}, + }); err != nil { + t.Fatal(err) + } + v, err := config.Load(name) + if err != nil { + t.Fatal(err) + } + vmBefore, err := os.ReadFile(filepath.Join(v.Dir, "vm.toml")) + if err != nil { + t.Fatal(err) + } + secretsBefore, err := os.ReadFile(v.SecretsPath()) + if err != nil { + t.Fatal(err) + } + newSecret := "new-secret-" + tc.name + if _, err := Update(name, tc.patch(&newSecret)); err == nil { + t.Fatal("invalid update succeeded") + } + vmAfter, err := os.ReadFile(filepath.Join(v.Dir, "vm.toml")) + if err != nil { + t.Fatal(err) + } + secretsAfter, err := os.ReadFile(v.SecretsPath()) + if err != nil { + t.Fatal(err) + } + if !bytes.Equal(vmAfter, vmBefore) { + t.Errorf("vm.toml changed after rejected update:\nbefore:\n%s\nafter:\n%s", vmBefore, vmAfter) + } + if !bytes.Equal(secretsAfter, secretsBefore) { + t.Errorf("secrets.toml changed after rejected update:\nbefore:\n%s\nafter:\n%s", secretsBefore, secretsAfter) + } + }) + } +} diff --git a/internal/core/update.go b/internal/core/update.go index be2914b2..3cf77bb2 100644 --- a/internal/core/update.go +++ b/internal/core/update.go @@ -5,10 +5,12 @@ import ( "fmt" "os" "os/exec" + "path/filepath" "strings" "github.com/novusedge/stoat/internal/config" "github.com/novusedge/stoat/internal/qemu" + "github.com/novusedge/stoat/internal/recipes" ) // ErrImmutableField is returned by Update when a Patch changes a field with no @@ -53,7 +55,10 @@ type Patch struct { // the list untouched, matching every other field's "nil means don't // touch" rule; there is no other way to tell "clear the list" and // "didn't mention it" apart with a bare []string. - Recipes *[]string + Recipes *[]string + SetParams map[string]map[string]string + UnsetParams map[string][]string + Secrets config.Secrets // Installed is meaningful only for a disk-mode VM: it tracks whether the OS // is installed to disk.qcow2 yet. qemu.Start flips it true once disk.qcow2 @@ -110,8 +115,9 @@ func Update(name string, p Patch) (VM, error) { if err != nil { return VM{}, err } + work := cloneConfigVM(v) - if err := checkImmutable(v, p); err != nil { + if err := checkImmutable(work, p); err != nil { return VM{}, err } @@ -119,40 +125,53 @@ func Update(name string, p Patch) (VM, error) { if *p.RAM < 256 { return VM{}, fmt.Errorf("%w: ram must be at least 256 MB", ErrInvalidSpec) } - v.RAM = *p.RAM + work.RAM = *p.RAM } if p.CPUs != nil { if *p.CPUs < 1 { return VM{}, fmt.Errorf("%w: cpus must be at least 1", ErrInvalidSpec) } - v.CPUs = *p.CPUs + work.CPUs = *p.CPUs } if p.Share != nil { - v.Share = strings.TrimSpace(*p.Share) + work.Share = strings.TrimSpace(*p.Share) } if p.Recipes != nil { - v.Recipes = *p.Recipes + work.Recipes = append([]string(nil), (*p.Recipes)...) + } + var stored config.Secrets + if hasParamEdits(p) { + stored, err = config.LoadSecrets(work.Dir) + if err != nil { + return VM{}, err + } + } + var stagedSecrets config.Secrets + var secretTouched bool + err = stageParamEdits(work, p, stored, &stagedSecrets, &secretTouched) + if err != nil { + return VM{}, err } if p.Installed != nil { - v.Installed = *p.Installed + work.Installed = *p.Installed } if p.Display != nil { if err := validateDisplay(*p.Display); err != nil { return VM{}, err } - v.Display = *p.Display + work.Display = *p.Display } - if p.SSHPort != nil && *p.SSHPort != v.SSHPort { - if err := validateSSHPort(v, *p.SSHPort); err != nil { + if p.SSHPort != nil && *p.SSHPort != work.SSHPort { + if err := validateSSHPort(work, *p.SSHPort); err != nil { return VM{}, err } - v.SSHPort = *p.SSHPort + work.SSHPort = *p.SSHPort } resizeTo := "" - if p.Disk != nil && *p.Disk != v.Disk { - resizeTo, err = validateDiskGrow(v, *p.Disk) + if p.Disk != nil && *p.Disk != work.Disk { + resizeTo, err = validateDiskGrow(work, *p.Disk) if err != nil { return VM{}, err } @@ -162,17 +181,293 @@ func Update(name string, p Patch) (VM, error) { // saveEdit. If qemu-img fails, vm.toml still describes the disk that // exists, not one that doesn't. if resizeTo != "" { - out, err := exec.Command("qemu-img", "resize", v.DiskPath(), resizeTo).CombinedOutput() + out, err := exec.Command("qemu-img", "resize", work.DiskPath(), resizeTo).CombinedOutput() if err != nil { return VM{}, fmt.Errorf("qemu-img resize: %s", strings.TrimSpace(string(out))) } - v.Disk = resizeTo + work.Disk = resizeTo } - if err := v.Save(); err != nil { + if err := commitUpdate(v, work, stagedSecrets, secretTouched); err != nil { return VM{}, err } - return fromConfig(v), nil + return fromConfig(work), nil +} + +// applyParamEdits validates and applies parameter changes. Non-secret values +// stay in vm.toml; secret values stay in secrets.toml and are removed when an +// unset edit names a secret parameter. +func applyParamEdits(v *config.VM, p Patch) error { + stored, err := config.LoadSecrets(v.Dir) + if err != nil { + return err + } + var staged config.Secrets + var touched bool + if err := stageParamEdits(v, p, stored, &staged, &touched); err != nil { + return err + } + if touched { + return config.SaveSecrets(v.Dir, staged) + } + return nil +} + +func stageParamEdits(v *config.VM, p Patch, stored config.Secrets, stagedOut *config.Secrets, touchedOut *bool) error { + *stagedOut = cloneSecrets(stored) + *touchedOut = false + if len(p.SetParams) == 0 && len(p.UnsetParams) == 0 && len(p.Secrets) == 0 { + return nil + } + + for recipe, values := range p.SetParams { + m, err := manifestForVM(v, recipe) + if err != nil { + return err + } + for name, value := range values { + param, ok := m.Params[name] + if !ok { + if err := recipes.Validate(m, name, value); err != nil { + return fmt.Errorf("%w: %v", ErrInvalidSpec, err) + } + } + if ok && param.Type == "secret" { + return fmt.Errorf("%w: %s.%s is a secret; use --secret", ErrInvalidSpec, recipe, name) + } + if err := recipes.Validate(m, name, value); err != nil { + return fmt.Errorf("%w: %v", ErrInvalidSpec, err) + } + } + } + + secretTouched := len(p.Secrets) > 0 + for recipe, names := range p.UnsetParams { + m, err := manifestForVM(v, recipe) + if err != nil { + return err + } + for _, name := range names { + if _, ok := m.Params[name]; !ok { + return fmt.Errorf("%w: %s.%s is not declared", ErrInvalidSpec, recipe, name) + } + if m.Params[name].Type == "secret" { + secretTouched = true + } + } + } + for recipe, values := range p.Secrets { + m, err := manifestForVM(v, recipe) + if err != nil { + return err + } + for name, value := range values { + param, ok := m.Params[name] + if !ok { + return fmt.Errorf("%w: %s.%s is not declared", ErrInvalidSpec, recipe, name) + } + if param.Type != "secret" { + return fmt.Errorf("%w: %s.%s is not a secret param", ErrInvalidSpec, recipe, name) + } + if value == "" { + return fmt.Errorf("%w: %s.%s secret is empty", ErrInvalidSpec, recipe, name) + } + } + } + + for recipe, values := range p.SetParams { + for name, value := range values { + v.SetParam(recipe, name, value) + } + } + for recipe, names := range p.UnsetParams { + m, _ := manifestForVM(v, recipe) + for _, name := range names { + if m.Params[name].Type != "secret" { + v.UnsetParam(recipe, name) + } + } + } + staged := cloneSecrets(stored) + if secretTouched { + for recipe, names := range p.UnsetParams { + m, _ := manifestForVM(v, recipe) + for _, name := range names { + if m.Params[name].Type == "secret" { + delete(staged[recipe], name) + } + } + } + for recipe, values := range p.Secrets { + if staged[recipe] == nil { + staged[recipe] = map[string]string{} + } + for name, value := range values { + staged[recipe][name] = value + } + } + } + *stagedOut = staged + *touchedOut = secretTouched + return nil +} + +func hasParamEdits(p Patch) bool { + return len(p.SetParams) > 0 || len(p.UnsetParams) > 0 || len(p.Secrets) > 0 +} + +func cloneSecrets(in config.Secrets) config.Secrets { + if in == nil { + return config.Secrets{} + } + out := make(config.Secrets, len(in)) + for recipe, values := range in { + if values == nil { + continue + } + out[recipe] = make(map[string]string, len(values)) + for name, value := range values { + out[recipe][name] = value + } + } + return out +} + +func cloneConfigVM(in *config.VM) *config.VM { + out := *in + out.Recipes = append([]string(nil), in.Recipes...) + out.Forwards = append([]config.PortForward(nil), in.Forwards...) + if in.Params != nil { + out.Params = make(map[string]map[string]string, len(in.Params)) + for recipe, values := range in.Params { + out.Params[recipe] = make(map[string]string, len(values)) + for name, value := range values { + out.Params[recipe][name] = value + } + } + } + return &out +} + +// commitUpdate stages both on-disk representations, then swaps them into +// place with backups so a failure of the second replacement restores the +// first. The original inodes retain their modes and ownership on rollback. +func commitUpdate(original, updated *config.VM, secrets config.Secrets, secretTouched bool) error { + stageDir, err := os.MkdirTemp(original.Dir, ".update-stage-") + if err != nil { + return err + } + defer func() { _ = os.RemoveAll(stageDir) }() + + stagedVM := cloneConfigVM(updated) + stagedVM.Dir = stageDir + if err := stagedVM.Save(); err != nil { + return err + } + vmTarget := filepath.Join(original.Dir, "vm.toml") + // The previous single-file Save opened vm.toml for writing, so a + // read-only target failed even when its directory allowed replacement. + // Probe that same permission boundary before the atomic swap; otherwise a + // rename would silently bypass the target's mode and turn a failed update + // into a successful one. + if f, err := os.OpenFile(vmTarget, os.O_WRONLY, 0); err != nil { + return err + } else if err := f.Close(); err != nil { + return err + } + if info, statErr := os.Stat(vmTarget); statErr == nil { + if err := os.Chmod(filepath.Join(stageDir, "vm.toml"), info.Mode().Perm()); err != nil { + return err + } + } + + stagedSecrets := filepath.Join(stageDir, config.SecretsName) + secretTarget := filepath.Join(original.Dir, config.SecretsName) + if secretTouched { + if err := config.SaveSecrets(stageDir, secrets); err != nil { + return err + } + if info, statErr := os.Stat(secretTarget); statErr == nil { + if _, stageErr := os.Stat(stagedSecrets); stageErr == nil { + if err := os.Chmod(stagedSecrets, info.Mode().Perm()); err != nil { + return err + } + } + } + } + + vmBackup := filepath.Join(stageDir, "vm.toml.old") + secretBackup := filepath.Join(stageDir, config.SecretsName+".old") + vmHadOld := false + secretHadOld := false + vmInstalled := false + secretInstalled := false + rollback := func() { + if secretInstalled { + _ = os.Remove(secretTarget) + } + if vmInstalled { + _ = os.Remove(vmTarget) + } + if secretHadOld { + _ = os.Rename(secretBackup, secretTarget) + } + if vmHadOld { + _ = os.Rename(vmBackup, vmTarget) + } + } + + if err := os.Rename(vmTarget, vmBackup); err != nil { + return err + } + vmHadOld = true + if secretTouched { + if _, statErr := os.Stat(secretTarget); statErr == nil { + if err := os.Rename(secretTarget, secretBackup); err != nil { + rollback() + return err + } + secretHadOld = true + } else if !os.IsNotExist(statErr) { + rollback() + return statErr + } + } + if err := os.Rename(filepath.Join(stageDir, "vm.toml"), vmTarget); err != nil { + rollback() + return err + } + vmInstalled = true + if secretTouched { + if _, statErr := os.Stat(stagedSecrets); statErr == nil { + if err := os.Rename(stagedSecrets, secretTarget); err != nil { + rollback() + return err + } + secretInstalled = true + } else if !os.IsNotExist(statErr) { + rollback() + return statErr + } + } + return nil +} + +func manifestForVM(v *config.VM, recipe string) (recipes.Manifest, error) { + for _, name := range v.Recipes { + if name != recipe { + continue + } + m, ok, err := recipes.ManifestFor(recipe) + if err != nil { + return recipes.Manifest{}, err + } + if !ok { + return recipes.Manifest{}, fmt.Errorf("%w: recipe %q has no recipe.toml", ErrRecipeNotApplicable, recipe) + } + return m, nil + } + return recipes.Manifest{}, fmt.Errorf("%w: %s is not one of %s's recipes", ErrRecipeNotApplicable, recipe, v.Name) } // validateSSHPort checks a candidate ssh port by reusing validateForwards diff --git a/internal/core/vm.go b/internal/core/vm.go index bc43f57c..baf9be0c 100644 --- a/internal/core/vm.go +++ b/internal/core/vm.go @@ -12,6 +12,7 @@ import ( "github.com/novusedge/stoat/internal/guest" "github.com/novusedge/stoat/internal/iso" "github.com/novusedge/stoat/internal/qemu" + "github.com/novusedge/stoat/internal/recipes" ) // State is List/Get's answer at call time. It is never cached; it comes @@ -81,6 +82,28 @@ type AppliedRecipe struct { Version string Hash string At time.Time + Health string + Outputs map[string]string +} + +// SecretSet and SecretUnset are the redacted states readers expose for +// recipe secret parameters. +const ( + SecretSet = "" + SecretUnset = "" +) + +// RecipeState is one recipe's stored per-VM state. Secret values are never +// carried here; SecretNames lets a wire boundary verify redaction. +type RecipeState struct { + Name string + Applied bool + Version string + At time.Time + Health string + Params map[string]string + SecretNames []string + Outputs map[string]string } // VM answers "what is this VM doing right now". It is not the on-disk @@ -110,11 +133,13 @@ type VM struct { // which keeps ticking between reloads. StartedAt time.Time - RAM int - CPUs int - Disk string - Share string - Recipes []string + RAM int + CPUs int + Disk string + Share string + Recipes []string + RecipeStates []RecipeState + Health Health SSHPort int SSHUser string @@ -215,7 +240,7 @@ func checkGuest(v *config.VM) error { // fromConfig builds the point-in-time view for a VM that parsed cleanly. // State and Paths are the two things config.VM cannot answer for itself. -func fromConfig(v *config.VM) VM { +func fromConfigUnchecked(v *config.VM) VM { state := StateStopped if qemu.Running(v) { state = StateRunning @@ -257,6 +282,82 @@ func fromConfig(v *config.VM) VM { } } +// fromConfigChecked adds the stored recipe state that requires reading the +// protected secret store. Public readers use this form so a security error +// cannot be mistaken for an empty VM state. +func fromConfigChecked(v *config.VM) (VM, error) { + out := fromConfigUnchecked(v) + states, err := recipeStates(v) + if err != nil { + return VM{}, fmt.Errorf("%s: %w", filepath.Base(v.Dir), err) + } + out.RecipeStates = states + verdicts := make([]RecipeHealth, 0, len(states)) + for _, state := range states { + status := HealthUnknown + if state.Health != "" { + status = Health(state.Health) + } + verdicts = append(verdicts, RecipeHealth{Name: state.Name, Status: status}) + } + out.Health = VMHealth(verdicts) + return out, nil +} + +// fromConfig retains the value-only helper used by mutating operations. +// Public Get and List call fromConfigChecked and propagate secret-store +// failures instead. +func fromConfig(v *config.VM) VM { + out, _ := fromConfigChecked(v) + return out +} + +// recipeStates projects one state for every configured recipe. Secret values +// are replaced before this data leaves the core status layer. +func recipeStates(v *config.VM) ([]RecipeState, error) { + secrets, err := config.LoadSecrets(v.Dir) + if err != nil { + return nil, err + } + out := make([]RecipeState, 0, len(v.Recipes)) + for _, name := range v.Recipes { + applied, done := v.Applied[name] + state := RecipeState{ + Name: name, Applied: done, Version: applied.Version, At: applied.At, + Health: applied.Health, Params: map[string]string{}, Outputs: map[string]string{}, + } + if state.Health == "" { + state.Health = string(HealthUnknown) + } + for key, value := range applied.Outputs { + state.Outputs[key] = value + } + manifest, ok, manifestErr := recipes.ManifestFor(name) + if manifestErr != nil || !ok { + out = append(out, state) + continue + } + for _, param := range manifest.SortedParams() { + if param.Type == "secret" { + state.SecretNames = append(state.SecretNames, param.Name) + if secrets[name][param.Name] == "" { + state.Params[param.Name] = SecretUnset + } else { + state.Params[param.Name] = SecretSet + } + continue + } + if value, given := v.Params[name][param.Name]; given { + state.Params[param.Name] = value + } else { + state.Params[param.Name] = param.Default + } + } + out = append(out, state) + } + return out, nil +} + // applied converts config.VM.Applied to core's own AppliedRecipe, so core.VM // never carries a config type (see AppliedRecipe's doc comment). A nil input // returns nil rather than an empty map, matching config.VM.Applied's own @@ -267,7 +368,11 @@ func applied(m map[string]config.AppliedRecipe) map[string]AppliedRecipe { } out := make(map[string]AppliedRecipe, len(m)) for k, v := range m { - out[k] = AppliedRecipe{Version: v.Version, Hash: v.Hash, At: v.At} + outputs := make(map[string]string, len(v.Outputs)) + for name, value := range v.Outputs { + outputs[name] = value + } + out[k] = AppliedRecipe{Version: v.Version, Hash: v.Hash, At: v.At, Health: v.Health, Outputs: outputs} } return out } @@ -333,7 +438,11 @@ func List() ([]VM, error) { out = append(out, VM{Name: filepath.Base(cv.Dir), State: StateBroken, Error: err.Error()}) continue } - out = append(out, fromConfig(cv)) + view, err := fromConfigChecked(cv) + if err != nil { + return nil, err + } + out = append(out, view) } broken, err := config.ListBroken() @@ -365,7 +474,7 @@ func Get(name string) (VM, error) { if err != nil { return VM{}, err } - return fromConfig(v), nil + return fromConfigChecked(v) } // Start launches VM name. It wraps qemu.Start; the actual work (pidfile, diff --git a/internal/core/vm_test.go b/internal/core/vm_test.go index 81bd1ab0..734fb01d 100644 --- a/internal/core/vm_test.go +++ b/internal/core/vm_test.go @@ -82,6 +82,87 @@ func TestGetKnownVM(t *testing.T) { } } +// A VM with an unreadable secrets file must fail with its VM context instead +// of becoming an apparently empty or healthy VM. +func TestGetRefusesInsecureSecretsWithVMContext(t *testing.T) { + root(t) + v := &config.VM{Name: "work", Mode: "live", RAM: 1024, CPUs: 1, SSHPort: 2201, Recipes: []string{"docker"}} + if err := v.Save(); err != nil { + t.Fatal(err) + } + path := filepath.Join(v.Dir, config.SecretsName) + if err := os.WriteFile(path, []byte("docker.authkey = \"sentinel\"\n"), 0o644); err != nil { + t.Fatal(err) + } + if err := os.Chmod(path, 0o644); err != nil { + t.Fatal(err) + } + + _, err := Get(v.Name) + if err == nil || !strings.Contains(err.Error(), "work") || !strings.Contains(err.Error(), "secrets.toml: mode 0644") { + t.Fatalf("Get error = %v, want VM context and secret-file mode", err) + } +} + +// List applies the same secret-store read policy as Get; one insecure VM must +// not silently disappear from the result. +func TestListRefusesInsecureSecretsWithVMContext(t *testing.T) { + root(t) + v := &config.VM{Name: "work", Mode: "live", RAM: 1024, CPUs: 1, SSHPort: 2201, Recipes: []string{"docker"}} + if err := v.Save(); err != nil { + t.Fatal(err) + } + path := filepath.Join(v.Dir, config.SecretsName) + if err := os.WriteFile(path, []byte("docker.authkey = \"sentinel\"\n"), 0o644); err != nil { + t.Fatal(err) + } + if err := os.Chmod(path, 0o644); err != nil { + t.Fatal(err) + } + + _, err := List() + if err == nil || !strings.Contains(err.Error(), "work") || !strings.Contains(err.Error(), "secrets.toml: mode 0644") { + t.Fatalf("List error = %v, want VM context and secret-file mode", err) + } +} + +// An absent secrets file remains the valid empty-store case. +func TestGetWithoutSecretsFileRemainsReadable(t *testing.T) { + root(t) + v := &config.VM{Name: "work", Mode: "live", RAM: 1024, CPUs: 1, SSHPort: 2201} + if err := v.Save(); err != nil { + t.Fatal(err) + } + got, err := Get(v.Name) + if err != nil { + t.Fatalf("Get without secrets.toml = %v, want nil", err) + } + if got.Health != HealthUnknown { + t.Errorf("Health = %q, want unknown", got.Health) + } +} + +// Stored health is a host-side status read. Get must not rerun SSH health +// checks merely to render a VM whose applied state already records a result. +func TestGetUsesStoredRecipeHealthWithoutSSH(t *testing.T) { + root(t) + v := &config.VM{ + Name: "work", Mode: "live", RAM: 1024, CPUs: 1, SSHPort: 2201, + Recipes: []string{"docker"}, + Applied: map[string]config.AppliedRecipe{"docker": {Health: string(HealthOK)}}, + } + if err := v.Save(); err != nil { + t.Fatal(err) + } + got, err := Get(v.Name) + if err != nil { + t.Fatal(err) + } + if got.Health != HealthOK { + t.Errorf("Health = %q, want %q", got.Health, HealthOK) + } +} + func TestGetUnknownVM(t *testing.T) { root(t) if _, err := Get("nope"); !errors.Is(err, ErrNotFound) { diff --git a/internal/core/wait.go b/internal/core/wait.go index 79d2a04e..d0145f67 100644 --- a/internal/core/wait.go +++ b/internal/core/wait.go @@ -32,10 +32,14 @@ const ( UntilApplied Until = "applied" // UntilStopped is qemu.Running turning false. UntilStopped Until = "stopped" + // UntilHealthy is every applied recipe's health check passing. + UntilHealthy Until = "healthy" ) // Untils returns every state Wait can block for. -func Untils() []Until { return []Until{UntilReachable, UntilApplied, UntilStopped} } +func Untils() []Until { + return []Until{UntilReachable, UntilApplied, UntilStopped, UntilHealthy} +} // Valid reports whether u is one of Untils(). Wait calls it before it loads // the VM, so a typo fails with the reason rather than with "not found". @@ -85,11 +89,80 @@ func Wait(ctx context.Context, name string, until Until) error { return waitApplied(ctx, v) case UntilStopped: return waitStopped(ctx, v) + case UntilHealthy: + return waitHealthy(ctx, v) default: return waitReachable(ctx, v) } } +// waitHealthy waits for ssh first, then evaluates every applied recipe that +// declares a check until all checks pass or the health budget expires. A +// caller deadline still bounds the operation; cancellation is returned +// unchanged when no health result is available. +func waitHealthy(ctx context.Context, v *config.VM) error { + if err := waitReachable(ctx, v); err != nil { + return err + } + budget := HealthTimeout(v) + if budget <= 0 { + return nil + } + healthCtx, cancel := context.WithTimeout(ctx, budget) + defer cancel() + var first RecipeHealth + for { + verdicts, err := HealthChecks(healthCtx, v.Name) + first = firstHealthFailure(verdicts) + if err != nil { + if callerErr := ctx.Err(); callerErr != nil { + if first.Name != "" { + return fmt.Errorf("%w: %s", callerErr, healthFailure(first)) + } + return callerErr + } + if first.Name != "" && errors.Is(healthCtx.Err(), context.DeadlineExceeded) { + return healthFailure(first) + } + return err + } + if first.Name == "" { + return nil + } + timer := time.NewTimer(pollInterval) + select { + case <-healthCtx.Done(): + if !timer.Stop() { + <-timer.C + } + if ctx.Err() != nil && first.Name != "" { + return fmt.Errorf("%w: %s", ctx.Err(), healthFailure(first)) + } + if first.Name != "" { + return healthFailure(first) + } + return healthCtx.Err() + case <-timer.C: + } + } +} + +func firstHealthFailure(verdicts []RecipeHealth) RecipeHealth { + for _, verdict := range verdicts { + if verdict.Status == HealthFailed { + return verdict + } + } + return RecipeHealth{} +} + +func healthFailure(verdict RecipeHealth) error { + if verdict.Detail == "" { + return fmt.Errorf("%s: health check failed", verdict.Name) + } + return fmt.Errorf("%s: %s", verdict.Name, verdict.Detail) +} + // waitReachable blocks until sshd answers on v's forwarded port. // // A VM whose qemu process is not running is refused immediately, not @@ -116,7 +189,20 @@ func sshBannerUp(ctx context.Context, v *config.VM) bool { return false } defer func() { _ = c.Close() }() - _ = c.SetReadDeadline(time.Now().Add(2 * time.Second)) + deadline := time.Now().Add(2 * time.Second) + if callerDeadline, ok := ctx.Deadline(); ok && callerDeadline.Before(deadline) { + deadline = callerDeadline + } + _ = c.SetReadDeadline(deadline) + readDone := make(chan struct{}) + defer close(readDone) + go func() { + select { + case <-ctx.Done(): + _ = c.Close() + case <-readDone: + } + }() buf := make([]byte, 4) _, err = io.ReadFull(c, buf) return err == nil && string(buf) == "SSH-" diff --git a/internal/core/wait_test.go b/internal/core/wait_test.go index 1ab71022..65dddfc0 100644 --- a/internal/core/wait_test.go +++ b/internal/core/wait_test.go @@ -5,7 +5,9 @@ import ( "errors" "net" "os" + "path/filepath" "strconv" + "strings" "testing" "time" @@ -270,3 +272,325 @@ func TestWaitCtxDeadlineExceeded(t *testing.T) { t.Fatalf("took %s past a 200ms deadline, want well under a second past it", elapsed) } } + +// A VM with no applied recipes that declare health is healthy as soon as ssh +// answers: no later check can change the result. +func TestWaitHealthyWithNoChecksReturnsOnReachable(t *testing.T) { + root(t) + port, stopSSH := fakeSSHD(t, 0) + defer stopSSH() + v := &config.VM{Name: "work", Mode: "live", RAM: 1024, CPUs: 1, SSHPort: port} + if err := v.Save(); err != nil { + t.Fatal(err) + } + defer fakeRunning(t, v)() + + ctx, cancel := context.WithTimeout(context.Background(), time.Second) + defer cancel() + if err := Wait(ctx, v.Name, UntilHealthy); err != nil { + t.Fatalf("Wait healthy = %v, want nil", err) + } +} + +// The first failing recipe is named and retains the check's last output line, +// so a caller can act on the reported failure rather than a generic timeout. +func TestWaitHealthyNamesFirstFailureAndDetail(t *testing.T) { + dir := root(t) + writeHealthRecipe(t, dir, true) + port, stopSSH := fakeSSHD(t, 0) + defer stopSSH() + v := &config.VM{ + Name: "work", Mode: "live", OS: "alpine", RAM: 1024, CPUs: 1, + SSHPort: port, Recipes: []string{"docker"}, + Applied: map[string]config.AppliedRecipe{"docker": {}}, + } + if err := v.Save(); err != nil { + t.Fatal(err) + } + defer fakeRunning(t, v)() + installHealthSSH(t, false) + + ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second) + defer cancel() + err := Wait(ctx, v.Name, UntilHealthy) + if err == nil || !strings.Contains(err.Error(), "docker: health check failed") || !strings.Contains(err.Error(), "cannot connect to the docker daemon") { + t.Fatalf("Wait healthy error = %v, want named check detail", err) + } +} + +// The global healthy deadline honors a declared timeout shorter than the old +// 30s fallback; it must not wait for a separate budget per recipe. +func TestWaitHealthyUsesLongestDeclaredTimeout(t *testing.T) { + dir := root(t) + writeHealthRecipeWithTimeout(t, dir, "50ms") + port, stopSSH := fakeSSHD(t, 0) + defer stopSSH() + v := &config.VM{ + Name: "work", Mode: "live", OS: "alpine", RAM: 1024, CPUs: 1, + SSHPort: port, Recipes: []string{"docker"}, + Applied: map[string]config.AppliedRecipe{"docker": {}}, + } + if err := v.Save(); err != nil { + t.Fatal(err) + } + defer fakeRunning(t, v)() + installHealthSSH(t, false) + + start := time.Now() + ctx, cancel := context.WithTimeout(context.Background(), time.Second) + defer cancel() + err := Wait(ctx, v.Name, UntilHealthy) + if err == nil { + t.Fatal("Wait healthy succeeded with a failing check") + } + if elapsed := time.Since(start); elapsed > 500*time.Millisecond { + t.Fatalf("Wait healthy took %s, want the 50ms health budget", elapsed) + } +} + +// Health checks are evaluated in recipe order, but the caller's one budget is +// the longest declared timeout. A slow first check must not let later checks +// add another full timeout to Wait. +func TestWaitHealthyUsesOneGlobalBudgetForSequentialChecks(t *testing.T) { + dir := root(t) + writeHealthRecipeWithCheckTimeoutNamed(t, dir, "health-one", "150ms", "health-one-check") + writeHealthRecipeWithCheckTimeoutNamed(t, dir, "health-two", "500ms", "health-two-check") + port, stopSSH := fakeSSHD(t, 0) + defer stopSSH() + v := &config.VM{ + Name: "work", Mode: "live", OS: "alpine", RAM: 1024, CPUs: 1, + SSHPort: port, Recipes: []string{"health-one", "health-two"}, + Applied: map[string]config.AppliedRecipe{"health-one": {}, "health-two": {}}, + } + if err := v.Save(); err != nil { + t.Fatal(err) + } + defer fakeRunning(t, v)() + installSequentialHealthSSH(t, filepath.Join(t.TempDir(), "health-calls")) + + start := time.Now() + ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second) + defer cancel() + err := Wait(ctx, v.Name, UntilHealthy) + if err == nil { + t.Fatal("Wait healthy succeeded with blocked checks") + } + if !strings.Contains(err.Error(), "health-one") || !strings.Contains(err.Error(), "first-health-detail") { + t.Fatalf("Wait healthy error = %v, want first failing recipe and detail", err) + } + if elapsed := time.Since(start); elapsed >= 800*time.Millisecond { + t.Fatalf("Wait healthy took %s, want one 500ms global budget rather than sequential budgets", elapsed) + } +} + +// A single probe can time out after writing diagnostic output. Wait must keep +// that named, redacted failure instead of returning a bare context deadline. +func TestWaitHealthyRetainsSingleCheckDetailWhenInternalBudgetExpires(t *testing.T) { + dir := root(t) + const ( + recipe = "single-blocked" + secret = "single-health-blocking-secret-7c2" + ) + writeHealthRecipeWithCheckTimeoutNamed(t, dir, recipe, "100ms", "single-health-check") + port, stopSSH := fakeSSHD(t, 0) + defer stopSSH() + v := &config.VM{ + Name: "work", Mode: "live", OS: "alpine", RAM: 1024, CPUs: 1, + SSHPort: port, Recipes: []string{recipe}, + Applied: map[string]config.AppliedRecipe{recipe: {}}, + } + if err := v.Save(); err != nil { + t.Fatal(err) + } + if err := config.SaveSecrets(v.Dir, config.Secrets{recipe: {"token": secret}}); err != nil { + t.Fatal(err) + } + defer fakeRunning(t, v)() + installSingleBlockingHealthSSH(t, secret) + + ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second) + defer cancel() + err := Wait(ctx, v.Name, UntilHealthy) + if err == nil { + t.Fatal("Wait healthy succeeded with a single check that exceeded its internal budget") + } + if errors.Is(err, context.DeadlineExceeded) { + t.Fatalf("Wait healthy returned bare deadline for internal health timeout: %v", err) + } + for _, want := range []string{recipe, "single-health-detail", ""} { + if !strings.Contains(err.Error(), want) { + t.Fatalf("Wait healthy error = %v, want %q", err, want) + } + } + if strings.Contains(err.Error(), secret) { + t.Fatalf("Wait healthy error leaked stored secret: %v", err) + } +} + +// A child that ignores SIGTERM must still be reaped promptly when a health +// check's context expires. The PID is the fake ssh process itself, so a +// passing implementation cannot leave an owned descendant behind. +func TestHealthCheckReapsTERMIgnoringChildWithinBound(t *testing.T) { + dir := root(t) + writeHealthRecipeWithTimeoutNamed(t, dir, "ignore-term", "100ms") + port, stopSSH := fakeSSHD(t, 0) + defer stopSSH() + v := &config.VM{ + Name: "work", Mode: "live", OS: "alpine", RAM: 1024, CPUs: 1, + SSHPort: port, Recipes: []string{"ignore-term"}, + Applied: map[string]config.AppliedRecipe{"ignore-term": {}}, + } + if err := v.Save(); err != nil { + t.Fatal(err) + } + defer fakeRunning(t, v)() + pidPath := filepath.Join(t.TempDir(), "ssh.pid") + installIgnoringTERMHealthSSH(t, pidPath) + + ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond) + defer cancel() + start := time.Now() + _, err := HealthChecks(ctx, v.Name) + elapsed := time.Since(start) + if !errors.Is(err, context.DeadlineExceeded) { + t.Fatalf("HealthChecks error = %v, want context deadline", err) + } + if elapsed >= 2*time.Second { + t.Fatalf("HealthChecks took %s after child ignored SIGTERM, want bounded reaping", elapsed) + } +} + +// An accepted TCP peer that never sends an SSH banner is not reachable. The +// banner read must nevertheless observe the caller context instead of waiting +// for its independent two-second socket deadline. +func TestWaitReachableSilentPeerHonorsContext(t *testing.T) { + root(t) + l, err := net.Listen("tcp", "127.0.0.1:0") + if err != nil { + t.Fatal(err) + } + defer func() { _ = l.Close() }() + peerDone := make(chan struct{}) + go func() { + for { + conn, acceptErr := l.Accept() + if acceptErr != nil { + return + } + go func() { + <-peerDone + _ = conn.Close() + }() + } + }() + v := &config.VM{ + Name: "work", Mode: "live", RAM: 1024, CPUs: 1, + SSHPort: l.Addr().(*net.TCPAddr).Port, + } + if err := v.Save(); err != nil { + t.Fatal(err) + } + defer fakeRunning(t, v)() + ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond) + defer cancel() + start := time.Now() + err = Wait(ctx, v.Name, UntilReachable) + close(peerDone) + if !errors.Is(err, context.DeadlineExceeded) { + t.Fatalf("Wait silent peer error = %v, want context deadline", err) + } + if elapsed := time.Since(start); elapsed >= time.Second { + t.Fatalf("Wait silent peer took %s, want context-bounded banner read", elapsed) + } +} + +func TestHealthTimeoutUsesLongestDeclaredTimeoutWithoutMinimum(t *testing.T) { + dir := root(t) + writeHealthRecipeWithTimeoutNamed(t, dir, "docker", "50ms") + writeHealthRecipeWithTimeoutNamed(t, dir, "tailscale", "2s") + v := &config.VM{ + Name: "work", OS: "alpine", Recipes: []string{"docker", "tailscale"}, + Applied: map[string]config.AppliedRecipe{"docker": {}, "tailscale": {}}, + } + if got, want := HealthTimeout(v), 2*time.Second; got != want { + t.Fatalf("HealthTimeout = %s, want longest declared timeout %s", got, want) + } + writeHealthRecipeWithTimeoutNamed(t, dir, "docker", "50ms") + v.Applied = map[string]config.AppliedRecipe{"docker": {}} + if got, want := HealthTimeout(v), 50*time.Millisecond; got != want { + t.Fatalf("HealthTimeout = %s, want declared timeout %s (not the 30s default)", got, want) + } +} + +// Parent cancellation survives the reachability and health boundaries rather +// than being converted into a recipe failure. +func TestWaitHealthyPropagatesCancellation(t *testing.T) { + root(t) + v := &config.VM{Name: "work", Mode: "live", RAM: 1024, CPUs: 1, SSHPort: 2399} + if err := v.Save(); err != nil { + t.Fatal(err) + } + defer fakeRunning(t, v)() + ctx, cancel := context.WithCancel(context.Background()) + cancel() + if err := Wait(ctx, v.Name, UntilHealthy); !errors.Is(err, context.Canceled) { + t.Fatalf("Wait healthy error = %v, want context.Canceled", err) + } +} + +func writeHealthRecipeWithTimeout(t *testing.T, rootDir, timeout string) { + writeHealthRecipeWithTimeoutNamed(t, rootDir, "docker", timeout) +} + +func writeHealthRecipeWithTimeoutNamed(t *testing.T, rootDir, name, timeout string) { + writeHealthRecipeWithCheckTimeoutNamed(t, rootDir, name, timeout, "docker info") +} + +func writeHealthRecipeWithCheckTimeoutNamed(t *testing.T, rootDir, name, timeout, check string) { + t.Helper() + d := filepath.Join(rootDir, "recipes", name) + if err := os.MkdirAll(d, 0o755); err != nil { + t.Fatal(err) + } + manifest := "schema = 3\nname = \"" + name + "\"\nscript = \"install.sh\"\n\n[health]\ncheck = \"" + check + "\"\n" + if timeout != "" { + manifest += "timeout = \"" + timeout + "\"\n" + } + if err := os.WriteFile(filepath.Join(d, "recipe.toml"), []byte(manifest), 0o644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(d, "install.sh"), []byte("#!/bin/sh\nexit 0\n"), 0o755); err != nil { + t.Fatal(err) + } +} + +func installSequentialHealthSSH(t *testing.T, callsPath string) { + t.Helper() + bin := t.TempDir() + script := "#!/bin/sh\nbody=$(cat)\ncalls=0\nif [ -f " + shellQuoteCoreTest(callsPath) + " ]; then calls=$(cat " + shellQuoteCoreTest(callsPath) + "); fi\ncalls=$((calls + 1))\nprintf '%s\\n' \"$calls\" > " + shellQuoteCoreTest(callsPath) + "\ncase \"$body\" in\n*health-one-check*) printf '%s\\n' first-health-detail >&2; exit 1;;\n*health-two-check*) if [ \"$calls\" -ge 4 ]; then while :; do :; done; fi; exit 0;;\nesac\nexit 1\n" + if err := os.WriteFile(filepath.Join(bin, "ssh"), []byte(script), 0o755); err != nil { + t.Fatal(err) + } + t.Setenv("PATH", bin+string(os.PathListSeparator)+os.Getenv("PATH")) +} + +func installSingleBlockingHealthSSH(t *testing.T, secret string) { + t.Helper() + bin := t.TempDir() + detail := shellQuoteCoreTest("single-health-detail " + secret) + script := "#!/bin/sh\nbody=$(cat)\ncase \"$body\" in\n*'single-health-check'*) printf '%s\\n' " + detail + " >&2; trap '' TERM; while :; do :; done;;\nesac\nexit 0\n" + if err := os.WriteFile(filepath.Join(bin, "ssh"), []byte(script), 0o755); err != nil { + t.Fatal(err) + } + t.Setenv("PATH", bin+string(os.PathListSeparator)+os.Getenv("PATH")) +} + +func installIgnoringTERMHealthSSH(t *testing.T, pidPath string) { + t.Helper() + bin := t.TempDir() + script := "#!/bin/sh\ncat >/dev/null\nprintf '%s\\n' \"$$\" > " + shellQuoteCoreTest(pidPath) + "\ntrap '' TERM\nwhile :; do :; done\n" + if err := os.WriteFile(filepath.Join(bin, "ssh"), []byte(script), 0o755); err != nil { + t.Fatal(err) + } + t.Setenv("PATH", bin+string(os.PathListSeparator)+os.Getenv("PATH")) +} diff --git a/internal/guest/bundled/alpine.toml b/internal/guest/bundled/alpine.toml index 25906212..7c0816f2 100644 --- a/internal/guest/bundled/alpine.toml +++ b/internal/guest/bundled/alpine.toml @@ -32,5 +32,9 @@ stop = "rc-service {name} stop" restart = "rc-service {name} restart" status = "rc-service {name} status" +[cmd] +download = "wget -O" +useradd = "adduser -D {name}" + [backend.cloudinit] skip_9p = false diff --git a/internal/guest/bundled/arch.toml b/internal/guest/bundled/arch.toml index 9146f588..b0fd447c 100644 --- a/internal/guest/bundled/arch.toml +++ b/internal/guest/bundled/arch.toml @@ -26,5 +26,9 @@ stop = "systemctl stop {name}" restart = "systemctl restart {name}" status = "systemctl status {name}" +[cmd] +download = "curl -fsSL -o" +useradd = "useradd -m -s /bin/bash {name}" + [backend.cloudinit] skip_9p = false diff --git a/internal/guest/bundled/debian.toml b/internal/guest/bundled/debian.toml index 3ec1b9fe..8a9e7817 100644 --- a/internal/guest/bundled/debian.toml +++ b/internal/guest/bundled/debian.toml @@ -29,6 +29,10 @@ stop = "systemctl stop {name}" restart = "systemctl restart {name}" status = "systemctl status {name}" +[cmd] +download = "curl -fsSL -o" +useradd = "useradd -m -s /bin/bash {name}" + # Debian's cloud kernel ships no 9p module; the cloudinit backend must not # rely on a 9p share mount for this guest. [backend.cloudinit] diff --git a/internal/guest/bundled/fedora.toml b/internal/guest/bundled/fedora.toml index 7ad5c097..e951719e 100644 --- a/internal/guest/bundled/fedora.toml +++ b/internal/guest/bundled/fedora.toml @@ -25,5 +25,9 @@ stop = "systemctl stop {name}" restart = "systemctl restart {name}" status = "systemctl status {name}" +[cmd] +download = "curl -fsSL -o" +useradd = "useradd -m -s /bin/bash {name}" + [backend.cloudinit] skip_9p = false diff --git a/internal/guest/bundled/ubuntu.toml b/internal/guest/bundled/ubuntu.toml index cb9e4237..2f85fde4 100644 --- a/internal/guest/bundled/ubuntu.toml +++ b/internal/guest/bundled/ubuntu.toml @@ -29,5 +29,9 @@ stop = "systemctl stop {name}" restart = "systemctl restart {name}" status = "systemctl status {name}" +[cmd] +download = "curl -fsSL -o" +useradd = "useradd -m -s /bin/bash {name}" + [backend.cloudinit] skip_9p = false diff --git a/internal/guest/prelude.go b/internal/guest/prelude.go index 743f1950..ae943f3e 100644 --- a/internal/guest/prelude.go +++ b/internal/guest/prelude.go @@ -98,7 +98,7 @@ func preludePython(o OS) string { fmt.Fprintf(&b, "def stoat_pkg_setup(): _run(\"sh\", \"-c\", %q)\n", setup) fmt.Fprintf(&b, "def stoat_pkg_install(*pkgs): _run(%s, *pkgs)\n", pyArgv(o.Pkg.Install)) for _, v := range verbs(o) { - fmt.Fprintf(&b, "def %s(name): _run(\"sh\", \"-c\", '%s', \"stoat\", name)\n", v.name, shTemplate(v.tmpl)) + fmt.Fprintf(&b, "def %s(*args): _run(\"sh\", \"-c\", '%s', \"stoat\", *args)\n", v.name, shTemplate(v.tmpl)) } return b.String() } diff --git a/internal/guest/prelude_test.go b/internal/guest/prelude_test.go index aa97c91c..1208ab10 100644 --- a/internal/guest/prelude_test.go +++ b/internal/guest/prelude_test.go @@ -61,6 +61,105 @@ func TestPreludePython(t *testing.T) { } } +func TestPreludeDefinesCmdVerbs(t *testing.T) { + for _, name := range []string{"alpine", "debian", "ubuntu", "fedora", "arch"} { + t.Run(name, func(t *testing.T) { + o, ok := Lookup(name) + if !ok { + t.Fatalf("no guest %q", name) + } + got := Prelude(o, "sh") + for _, fn := range []string{"stoat_download()", "stoat_useradd()"} { + if !strings.Contains(got, fn) { + t.Errorf("prelude does not define %s:\n%s", fn, got) + } + } + }) + } +} + +// {name} becomes "$1"; a template with no placeholder gets "$@". This is +// the same rule [svc] follows, so a recipe author learns it once. +func TestPreludeCmdTemplateRules(t *testing.T) { + o := OS{ + Name: "freebsd", Init: "rc", Shell: "/bin/sh", + Cmd: map[string]string{ + "download": "fetch -o", + "useradd": "pw useradd -n {name} -m", + }, + } + got := Prelude(o, "sh") + if !strings.Contains(got, `stoat_download() { fetch -o "$@"; }`) { + t.Errorf("download verb:\n%s", got) + } + if !strings.Contains(got, `stoat_useradd() { pw useradd -n "$1" -m; }`) { + t.Errorf("useradd verb:\n%s", got) + } +} + +// The python prelude defines the same names over subprocess.run. +func TestPythonPreludeDefinesCmdVerbs(t *testing.T) { + o, ok := Lookup("debian") + if !ok { + t.Fatal("bundled debian missing") + } + got := Prelude(o, "python3") + for _, fn := range []string{"def stoat_download(", "def stoat_useradd("} { + if !strings.Contains(got, fn) { + t.Errorf("python prelude does not define %s:\n%s", fn, got) + } + } +} + +func TestPythonPreludeCmdForwardsDownloadArguments(t *testing.T) { + if _, err := exec.LookPath("python3"); err != nil { + t.Fatal("python3 is required to execute the public Python prelude") + } + dir := t.TempDir() + recorded := filepath.Join(dir, "args") + recorder := filepath.Join(dir, "record") + if err := os.WriteFile(recorder, []byte("#!/bin/sh\nprintf '%s\\n' \"$@\" > \"$RECORD\"\n"), 0o755); err != nil { + t.Fatal(err) + } + if err := os.Chmod(recorder, 0o755); err != nil { + t.Fatal(err) + } + o := OS{ + Name: "freebsd", Init: "rc", Shell: "/bin/sh", + Pkg: Pkg{Install: []string{"true"}}, + Cmd: map[string]string{"download": "record"}, + } + body := Prelude(o, "python3") + "\nstoat_download(\"output.bin\", \"https://example.test/a\")\n" + cmd := exec.Command("python3", "-c", body) + cmd.Env = append(os.Environ(), "PATH="+dir+string(os.PathListSeparator)+os.Getenv("PATH"), "RECORD="+recorded) + if out, err := cmd.CombinedOutput(); err != nil { + t.Fatalf("python prelude failed: %v\n%s", err, out) + } + got, err := os.ReadFile(recorded) + if err != nil { + t.Fatal(err) + } + if string(got) != "output.bin\nhttps://example.test/a\n" { + t.Errorf("download args = %q, want output then URL", got) + } +} + +// STOAT_OUTPUT belongs to the per-recipe execution wrapper. The shared +// prelude also runs health and package-setup commands, which must not create +// or truncate a recipe output file. +func TestPreludeDoesNotInitializeStoatOutput(t *testing.T) { + o, ok := Lookup("alpine") + if !ok { + t.Fatal("bundled alpine missing") + } + for _, runtime := range []string{"sh", "python3"} { + got := Prelude(o, runtime) + if strings.Contains(got, "STOAT_OUTPUT") || strings.Contains(got, "/tmp/.stoat-out") { + t.Errorf("%s prelude initializes recipe output state:\n%s", runtime, got) + } + } +} + // WithPrelude inserts after a leading shebang line so the interpreter line // stays first; a body with no shebang gets the prelude in front. func TestWithPreludeKeepsShebangFirst(t *testing.T) { diff --git a/internal/guest/testdata/prelude/alpine.sh b/internal/guest/testdata/prelude/alpine.sh index 0670c7e1..8be41ea3 100644 --- a/internal/guest/testdata/prelude/alpine.sh +++ b/internal/guest/testdata/prelude/alpine.sh @@ -5,5 +5,7 @@ stoat_svc_start() { rc-service "$1" start; } stoat_svc_stop() { rc-service "$1" stop; } stoat_svc_restart() { rc-service "$1" restart; } stoat_svc_status() { rc-service "$1" status; } +stoat_download() { wget -O "$@"; } +stoat_useradd() { adduser -D "$1"; } STOAT_OS=alpine; STOAT_INIT=openrc; STOAT_PKGMGR=apk export STOAT_OS STOAT_INIT STOAT_PKGMGR diff --git a/internal/guest/testdata/prelude/arch.sh b/internal/guest/testdata/prelude/arch.sh index 6f76ba3c..0253b67e 100644 --- a/internal/guest/testdata/prelude/arch.sh +++ b/internal/guest/testdata/prelude/arch.sh @@ -5,5 +5,7 @@ stoat_svc_start() { systemctl start "$1"; } stoat_svc_stop() { systemctl stop "$1"; } stoat_svc_restart() { systemctl restart "$1"; } stoat_svc_status() { systemctl status "$1"; } +stoat_download() { curl -fsSL -o "$@"; } +stoat_useradd() { useradd -m -s /bin/bash "$1"; } STOAT_OS=arch; STOAT_INIT=systemd; STOAT_PKGMGR=pacman export STOAT_OS STOAT_INIT STOAT_PKGMGR diff --git a/internal/guest/testdata/prelude/debian.sh b/internal/guest/testdata/prelude/debian.sh index 236336d4..c90aa6af 100644 --- a/internal/guest/testdata/prelude/debian.sh +++ b/internal/guest/testdata/prelude/debian.sh @@ -5,6 +5,8 @@ stoat_svc_start() { systemctl start "$1"; } stoat_svc_stop() { systemctl stop "$1"; } stoat_svc_restart() { systemctl restart "$1"; } stoat_svc_status() { systemctl status "$1"; } +stoat_download() { curl -fsSL -o "$@"; } +stoat_useradd() { useradd -m -s /bin/bash "$1"; } export DEBIAN_FRONTEND='noninteractive' STOAT_OS=debian; STOAT_INIT=systemd; STOAT_PKGMGR=apt-get export STOAT_OS STOAT_INIT STOAT_PKGMGR diff --git a/internal/guest/testdata/prelude/fedora.sh b/internal/guest/testdata/prelude/fedora.sh index 71a5b2c0..6ea43654 100644 --- a/internal/guest/testdata/prelude/fedora.sh +++ b/internal/guest/testdata/prelude/fedora.sh @@ -5,5 +5,7 @@ stoat_svc_start() { systemctl start "$1"; } stoat_svc_stop() { systemctl stop "$1"; } stoat_svc_restart() { systemctl restart "$1"; } stoat_svc_status() { systemctl status "$1"; } +stoat_download() { curl -fsSL -o "$@"; } +stoat_useradd() { useradd -m -s /bin/bash "$1"; } STOAT_OS=fedora; STOAT_INIT=systemd; STOAT_PKGMGR=dnf export STOAT_OS STOAT_INIT STOAT_PKGMGR diff --git a/internal/guest/testdata/prelude/ubuntu.sh b/internal/guest/testdata/prelude/ubuntu.sh index 524c4a92..0100ea06 100644 --- a/internal/guest/testdata/prelude/ubuntu.sh +++ b/internal/guest/testdata/prelude/ubuntu.sh @@ -5,6 +5,8 @@ stoat_svc_start() { systemctl start "$1"; } stoat_svc_stop() { systemctl stop "$1"; } stoat_svc_restart() { systemctl restart "$1"; } stoat_svc_status() { systemctl status "$1"; } +stoat_download() { curl -fsSL -o "$@"; } +stoat_useradd() { useradd -m -s /bin/bash "$1"; } export DEBIAN_FRONTEND='noninteractive' STOAT_OS=ubuntu; STOAT_INIT=systemd; STOAT_PKGMGR=apt-get export STOAT_OS STOAT_INIT STOAT_PKGMGR diff --git a/internal/recipes/bundled/docker/install-alpine.sh b/internal/recipes/bundled/docker/install-alpine.sh index eb2c6d43..1f2f062f 100755 --- a/internal/recipes/bundled/docker/install-alpine.sh +++ b/internal/recipes/bundled/docker/install-alpine.sh @@ -40,6 +40,15 @@ docker version --format '{{.Server.Version}}' 2>/dev/null | sed 's/^/docker daemon running, version /' || echo "docker installed, but the daemon did not come up: check 'rc-service docker status'" +user="${STOAT_PARAM_USER:-dev}" +if ! id "$user" >/dev/null 2>&1; then + adduser -D "$user" +fi +addgroup "$user" docker 2>/dev/null || true +if [ -n "${STOAT_OUTPUT:-}" ]; then + printf '%s\n' 'socket=/var/run/docker.sock' >> "$STOAT_OUTPUT" +fi + # Live VMs are diskless: the root filesystem is a tmpfs/overlay in RAM, so # every package installed above is gone on reboot. A disk install mounts a # real block device as root, which persists. Detecting it from inside the diff --git a/internal/recipes/bundled/docker/install-arch.sh b/internal/recipes/bundled/docker/install-arch.sh index 109b49a5..f8466770 100755 --- a/internal/recipes/bundled/docker/install-arch.sh +++ b/internal/recipes/bundled/docker/install-arch.sh @@ -18,3 +18,12 @@ done docker version --format '{{.Server.Version}}' 2>/dev/null | sed 's/^/docker daemon running, version /' || echo "docker installed, but the daemon did not come up: check 'systemctl status docker'" + +user="${STOAT_PARAM_USER:-dev}" +if ! id "$user" >/dev/null 2>&1; then + useradd -m -s /bin/bash "$user" +fi +usermod -aG docker "$user" +if [ -n "${STOAT_OUTPUT:-}" ]; then + printf '%s\n' 'socket=/var/run/docker.sock' >> "$STOAT_OUTPUT" +fi diff --git a/internal/recipes/bundled/docker/install-debian.sh b/internal/recipes/bundled/docker/install-debian.sh index 6cd4290c..958704ff 100755 --- a/internal/recipes/bundled/docker/install-debian.sh +++ b/internal/recipes/bundled/docker/install-debian.sh @@ -35,3 +35,12 @@ done docker version --format '{{.Server.Version}}' 2>/dev/null | sed 's/^/docker daemon running, version /' || echo "docker installed, but the daemon did not come up: check 'systemctl status docker'" + +user="${STOAT_PARAM_USER:-dev}" +if ! id "$user" >/dev/null 2>&1; then + useradd -m -s /bin/bash "$user" +fi +usermod -aG docker "$user" +if [ -n "${STOAT_OUTPUT:-}" ]; then + printf '%s\n' 'socket=/var/run/docker.sock' >> "$STOAT_OUTPUT" +fi diff --git a/internal/recipes/bundled/docker/install-fedora.sh b/internal/recipes/bundled/docker/install-fedora.sh index 13adfa23..83c0821d 100755 --- a/internal/recipes/bundled/docker/install-fedora.sh +++ b/internal/recipes/bundled/docker/install-fedora.sh @@ -23,3 +23,12 @@ done docker version --format '{{.Server.Version}}' 2>/dev/null | sed 's/^/docker daemon running, version /' || echo "docker installed, but the daemon did not come up: check 'systemctl status docker'" + +user="${STOAT_PARAM_USER:-dev}" +if ! id "$user" >/dev/null 2>&1; then + useradd -m -s /bin/bash "$user" +fi +usermod -aG docker "$user" +if [ -n "${STOAT_OUTPUT:-}" ]; then + printf '%s\n' 'socket=/var/run/docker.sock' >> "$STOAT_OUTPUT" +fi diff --git a/internal/recipes/bundled/docker/install.sh b/internal/recipes/bundled/docker/install.sh index 3ad91bed..9ce23884 100755 --- a/internal/recipes/bundled/docker/install.sh +++ b/internal/recipes/bundled/docker/install.sh @@ -11,7 +11,7 @@ apt-get install -y ca-certificates curl gnupg install -m 0755 -d /etc/apt/keyrings id=$(. /etc/os-release && echo "$ID") curl -fsSL "https://download.docker.com/linux/$id/gpg" | \ - gpg --dearmor -o /etc/apt/keyrings/docker.gpg + gpg --batch --yes --no-tty --dearmor -o /etc/apt/keyrings/docker.gpg chmod a+r /etc/apt/keyrings/docker.gpg echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \ @@ -35,3 +35,12 @@ done docker version --format '{{.Server.Version}}' 2>/dev/null | sed 's/^/docker daemon running, version /' || echo "docker installed, but the daemon did not come up: check 'systemctl status docker'" + +user="${STOAT_PARAM_USER:-dev}" +if ! id "$user" >/dev/null 2>&1; then + useradd -m -s /bin/bash "$user" +fi +usermod -aG docker "$user" +if [ -n "${STOAT_OUTPUT:-}" ]; then + printf '%s\n' 'socket=/var/run/docker.sock' >> "$STOAT_OUTPUT" +fi diff --git a/internal/recipes/bundled/docker/recipe.toml b/internal/recipes/bundled/docker/recipe.toml index 46ff55c4..489af243 100644 --- a/internal/recipes/bundled/docker/recipe.toml +++ b/internal/recipes/bundled/docker/recipe.toml @@ -3,6 +3,7 @@ description = "Docker engine and the compose plugin" os = ["alpine", "ubuntu", "debian", "fedora", "arch"] stage = "provision" script = "install.sh" +schema = 3 [scripts] alpine = "install-alpine.sh" @@ -10,3 +11,15 @@ ubuntu = "install-debian.sh" debian = "install-debian.sh" fedora = "install-fedora.sh" arch = "install-arch.sh" + +[params.user] +type = "string" +default = "dev" +help = "account to add to the docker group" + +[outputs] +socket = "path of the docker socket" + +[health] +check = "docker info" +timeout = "30s" diff --git a/internal/recipes/bundled/tailscale/install-alpine.sh b/internal/recipes/bundled/tailscale/install-alpine.sh index fb3018e4..a07aaf7a 100755 --- a/internal/recipes/bundled/tailscale/install-alpine.sh +++ b/internal/recipes/bundled/tailscale/install-alpine.sh @@ -2,11 +2,8 @@ # Installs Tailscale and starts the daemon. Runs as root over ssh on a booted # Alpine VM. # -# It deliberately does NOT authenticate. Joining a tailnet needs an auth key, -# and stoat has nowhere to keep one that isn't worse than the alternative: a -# key in vm.toml would sit in plaintext in the data root, and a key baked into -# a recipe would end up in git. So this installs and starts the daemon, then -# tells you the one command to run yourself. +# The auth key is a required secret parameter. It is provided only for this +# invocation, so it never lands in vm.toml or the recipe directory. set -e # -c enables community, where tailscale lives; -1 picks a mirror and refreshes @@ -36,9 +33,10 @@ while [ $i -lt 30 ]; do sleep 1 done +if [ -n "${STOAT_PARAM_AUTHKEY:-}" ]; then + tailscale up --authkey "$STOAT_PARAM_AUTHKEY" +fi echo "tailscale installed and tailscaled running." -echo "To join your tailnet, ssh in and run: tailscale up" -echo "(stoat does not store auth keys, see this recipe's header for why.)" # Live VMs are diskless: the root filesystem is a tmpfs/overlay in RAM, so # every package installed above is gone on reboot. A disk install mounts a diff --git a/internal/recipes/bundled/tailscale/install-arch.sh b/internal/recipes/bundled/tailscale/install-arch.sh index 70872a1a..de765838 100755 --- a/internal/recipes/bundled/tailscale/install-arch.sh +++ b/internal/recipes/bundled/tailscale/install-arch.sh @@ -2,9 +2,8 @@ # Installs Tailscale and starts the daemon. Runs as root over ssh on a booted # Arch VM. # -# Does NOT authenticate. Joining a tailnet needs an auth key, and stoat has -# nowhere to keep one safely. This installs and starts the daemon, then tells -# you the one command to run yourself. +# The auth key is a required secret parameter. It is provided only for this +# invocation, so it never lands in vm.toml or the recipe directory. set -e pacman -Sy --noconfirm tailscale @@ -20,5 +19,7 @@ while [ $i -lt 30 ]; do sleep 1 done +if [ -n "${STOAT_PARAM_AUTHKEY:-}" ]; then + tailscale up --authkey "$STOAT_PARAM_AUTHKEY" +fi echo "tailscale installed and tailscaled running." -echo "To join your tailnet, ssh in and run: tailscale up" diff --git a/internal/recipes/bundled/tailscale/install-debian.sh b/internal/recipes/bundled/tailscale/install-debian.sh index e1793f62..1beb9e12 100755 --- a/internal/recipes/bundled/tailscale/install-debian.sh +++ b/internal/recipes/bundled/tailscale/install-debian.sh @@ -2,9 +2,8 @@ # Installs Tailscale and starts the daemon. Runs as root over ssh on a booted # Ubuntu or Debian VM. # -# Does NOT authenticate. Joining a tailnet needs an auth key, and stoat has -# nowhere to keep one safely. This installs and starts the daemon, then tells -# you the one command to run yourself. +# The auth key is a required secret parameter. It is provided only for this +# invocation, so it never lands in vm.toml or the recipe directory. set -e export DEBIAN_FRONTEND=noninteractive @@ -22,5 +21,7 @@ while [ $i -lt 30 ]; do sleep 1 done +if [ -n "${STOAT_PARAM_AUTHKEY:-}" ]; then + tailscale up --authkey "$STOAT_PARAM_AUTHKEY" +fi echo "tailscale installed and tailscaled running." -echo "To join your tailnet, ssh in and run: tailscale up" diff --git a/internal/recipes/bundled/tailscale/install-fedora.sh b/internal/recipes/bundled/tailscale/install-fedora.sh index e3e1f25d..778f70dd 100755 --- a/internal/recipes/bundled/tailscale/install-fedora.sh +++ b/internal/recipes/bundled/tailscale/install-fedora.sh @@ -2,9 +2,8 @@ # Installs Tailscale and starts the daemon. Runs as root over ssh on a booted # Fedora VM. # -# Does NOT authenticate. Joining a tailnet needs an auth key, and stoat has -# nowhere to keep one safely. This installs and starts the daemon, then tells -# you the one command to run yourself. +# The auth key is a required secret parameter. It is provided only for this +# invocation, so it never lands in vm.toml or the recipe directory. set -e curl -fsSL https://tailscale.com/install.sh | sh @@ -20,5 +19,7 @@ while [ $i -lt 30 ]; do sleep 1 done +if [ -n "${STOAT_PARAM_AUTHKEY:-}" ]; then + tailscale up --authkey "$STOAT_PARAM_AUTHKEY" +fi echo "tailscale installed and tailscaled running." -echo "To join your tailnet, ssh in and run: tailscale up" diff --git a/internal/recipes/bundled/tailscale/install.sh b/internal/recipes/bundled/tailscale/install.sh index cb1cd3fb..b6242344 100755 --- a/internal/recipes/bundled/tailscale/install.sh +++ b/internal/recipes/bundled/tailscale/install.sh @@ -2,9 +2,8 @@ # Installs Tailscale and starts the daemon. Default script for OSes not explicitly # listed in [scripts]. Uses Tailscale's official install script. # -# Does NOT authenticate. Joining a tailnet needs an auth key, and stoat has -# nowhere to keep one safely. This installs and starts the daemon, then tells -# you the one command to run yourself. +# The auth key is a required secret parameter. It is provided only for this +# invocation, so it never lands in vm.toml or the recipe directory. set -e curl -fsSL https://tailscale.com/install.sh | sh @@ -19,5 +18,7 @@ while [ $i -lt 30 ]; do sleep 1 done +if [ -n "${STOAT_PARAM_AUTHKEY:-}" ]; then + tailscale up --authkey "$STOAT_PARAM_AUTHKEY" +fi echo "tailscale installed and tailscaled running." -echo "To join your tailnet, ssh in and run: tailscale up" diff --git a/internal/recipes/bundled/tailscale/recipe.toml b/internal/recipes/bundled/tailscale/recipe.toml index d8e41473..ca589c1a 100644 --- a/internal/recipes/bundled/tailscale/recipe.toml +++ b/internal/recipes/bundled/tailscale/recipe.toml @@ -3,6 +3,7 @@ description = "Tailscale daemon, installed and started (join manually)" os = ["alpine", "ubuntu", "debian", "fedora", "arch"] stage = "provision" script = "install.sh" +schema = 3 [scripts] alpine = "install-alpine.sh" @@ -10,3 +11,12 @@ ubuntu = "install-debian.sh" debian = "install-debian.sh" fedora = "install-fedora.sh" arch = "install-arch.sh" + +[params.authkey] +type = "secret" +required = true +help = "auth key used to join the tailnet" + +[health] +check = "tailscale version" +timeout = "30s" diff --git a/internal/recipes/bundled/xfce/install.sh b/internal/recipes/bundled/xfce/install.sh index db3265c6..9d9d41e0 100644 --- a/internal/recipes/bundled/xfce/install.sh +++ b/internal/recipes/bundled/xfce/install.sh @@ -14,6 +14,9 @@ apk) setup-devd udev udevadm trigger 2>/dev/null || true udevadm settle 2>/dev/null || true + # Alpine's xfce4 metapackage pulls no X server. setup-xorg-base installs + # xorg-server, xinit and xf86-input-libinput, which startx below needs. + setup-xorg-base stoat_pkg_install xfce4 xfce4-terminal dbus-x11 ;; apt-get) diff --git a/internal/recipes/manifest.go b/internal/recipes/manifest.go index 4facaeef..0abf4878 100644 --- a/internal/recipes/manifest.go +++ b/internal/recipes/manifest.go @@ -2,9 +2,14 @@ package recipes import ( "fmt" + "io" "os" "path/filepath" + "regexp" + "sort" + "strconv" "strings" + "time" "github.com/novusedge/stoat/internal/guest" "github.com/novusedge/stoat/internal/tomlx" @@ -14,21 +19,132 @@ import ( // (docs/recipe-spec-v2.md): a directory holding one manifest and one or // more shell scripts, replacing the old flat "..sh" files. type Manifest struct { - Name string `toml:"name"` - Description string `toml:"description"` - Version string `toml:"version"` - OS []string `toml:"os"` - Requires []string `toml:"requires"` - Stage string `toml:"stage"` // "install" | "provision" - Script string `toml:"script"` - Scripts map[string]string `toml:"scripts"` // OS-specific overrides - Auto bool `toml:"auto"` - Run string `toml:"run"` // "once" | "always" | "manual" - Reboot bool `toml:"reboot"` // guest needs a reboot after this recipe to take effect - Runtime string `toml:"runtime"` // "sh" | "python3", the interpreter the script runs under - Depends []string `toml:"depends"` // recipe names that must run before this one - - dir string // recipe directory, set by ParseManifest; scripts resolve against it + Schema int `toml:"schema"` + Name string `toml:"name"` + Description string `toml:"description"` + Version string `toml:"version"` + OS []string `toml:"os"` + Requires []string `toml:"requires"` + Stage string `toml:"stage"` // "install" | "provision" + Script string `toml:"script"` + Scripts map[string]string `toml:"scripts"` // OS-specific overrides + Auto bool `toml:"auto"` + Run string `toml:"run"` // "once" | "always" | "manual" + Reboot bool `toml:"reboot"` // guest needs a reboot after this recipe to take effect + Runtime string `toml:"runtime"` // "sh" | "python3", the interpreter the script runs under + Depends []string `toml:"depends"` // recipe names that must run before this one + ParamsRaw map[string]rawParam `toml:"params"` + Params map[string]Param `toml:"-"` + Outputs map[string]string `toml:"outputs"` + Health Health `toml:"health"` + + dir string // recipe directory, set by ParseManifest; scripts resolve against it + paramOrder []string // parameter declaration order, for interactive forms +} + +// Param is one declared input of a schema-3 recipe. Default is the spelling +// the recipe receives in STOAT_PARAM_, regardless of its type. +type Param struct { + Name string + Type string + Default string + Help string + Required bool + Values []string +} + +// rawParam is the TOML representation of a parameter. TOML preserves the +// literal type of default, so ParseManifest renders it after decoding. +type rawParam struct { + Type string `toml:"type"` + Default any `toml:"default"` + Help string `toml:"help"` + Required bool `toml:"required"` + Values []string `toml:"values"` +} + +// Output is one declared result of a recipe. +type Output struct { + Name string + Help string +} + +// Health is a recipe's health-check command. An empty Check means no check. +type Health struct { + Check string `toml:"check"` + Timeout string `toml:"timeout"` +} + +// DefaultHealthTimeout is used when a health check omits timeout. +const DefaultHealthTimeout = 30 * time.Second + +// Duration returns the configured health timeout. Invalid values fall back to +// the default because ParseManifest rejects them before a manifest is used. +func (h Health) Duration() time.Duration { + if h.Check == "" { + return 0 + } + if h.Timeout == "" { + return DefaultHealthTimeout + } + d, err := time.ParseDuration(h.Timeout) + if err != nil || d <= 0 { + return DefaultHealthTimeout + } + return d +} + +// SecretNames returns the names of secret parameters. +func (m Manifest) SecretNames() []string { + var names []string + for _, p := range m.SortedParams() { + if p.Type == "secret" { + names = append(names, p.Name) + } + } + return names +} + +// SortedParams returns declared parameters in name order. +func (m Manifest) SortedParams() []Param { + params := make([]Param, 0, len(m.Params)) + for _, p := range m.Params { + params = append(params, p) + } + sort.Slice(params, func(i, j int) bool { return params[i].Name < params[j].Name }) + return params +} + +// OrderedParams returns parameters in the order in which their tables appear +// in recipe.toml. Parameters added by a caller without a corresponding table +// are appended by name, so the result remains complete and deterministic. +// Wire projections use SortedParams; this order is only for the interactive +// form, where declaration order is part of the user's input flow. +func (m Manifest) OrderedParams() []Param { + params := make([]Param, 0, len(m.Params)) + seen := make(map[string]bool, len(m.Params)) + for _, name := range m.paramOrder { + if p, ok := m.Params[name]; ok { + params = append(params, p) + seen[name] = true + } + } + for _, p := range m.SortedParams() { + if !seen[p.Name] { + params = append(params, p) + } + } + return params +} + +// SortedOutputs returns declared outputs in name order. +func (m Manifest) SortedOutputs() []Output { + outputs := make([]Output, 0, len(m.Outputs)) + for name, help := range m.Outputs { + outputs = append(outputs, Output{Name: name, Help: help}) + } + sort.Slice(outputs, func(i, j int) bool { return outputs[i].Name < outputs[j].Name }) + return outputs } var validStages = map[string]bool{"install": true, "provision": true} @@ -37,9 +153,14 @@ var validRuns = map[string]bool{"once": true, "always": true, "manual": true} var validRuntimes = map[string]bool{"sh": true, "python3": true} +var paramName = regexp.MustCompile(`^[a-z][a-z0-9_]*$`) + +var validParamTypes = []string{"string", "int", "bool", "enum", "secret"} + // ParseManifest reads and validates a recipe.toml at path. Defaults are // applied before validation: Stage defaults to "provision" (the common -// case, docs/recipe-spec-v2.md's Stages section), Run defaults to "once". +// case, docs/recipe-spec-v2.md's Stages section), Run defaults to "once", +// Runtime to "sh", and Schema to 2 for older manifests. func ParseManifest(path string) (Manifest, error) { var m Manifest if err := tomlx.Decode(path, &m, tomlx.Reject); err != nil { @@ -47,6 +168,15 @@ func ParseManifest(path string) (Manifest, error) { } m.dir = filepath.Dir(path) + schema, schemaSet, err := manifestSchema(path) + if err != nil { + return Manifest{}, err + } + if !schemaSet { + m.Schema = 2 + } else { + m.Schema = schema + } if m.Stage == "" { m.Stage = "provision" } @@ -72,10 +202,161 @@ func ParseManifest(path string) (Manifest, error) { if !validRuntimes[m.Runtime] { return Manifest{}, fmt.Errorf("%s: invalid runtime %q, want %q or %q", path, m.Runtime, "sh", "python3") } + if m.Schema > 3 { + return Manifest{}, fmt.Errorf("%s: schema %d is newer than this stoat (3)", path, m.Schema) + } + if schemaSet && m.Schema != 2 && m.Schema != 3 { + return Manifest{}, fmt.Errorf("%s: schema %d is unsupported; want 2 or 3", path, m.Schema) + } + if m.Schema < 3 && (len(m.ParamsRaw) > 0 || len(m.Outputs) > 0 || m.Health.Check != "" || m.Health.Timeout != "") { + return Manifest{}, fmt.Errorf("%s: params, outputs and health require schema 3", path) + } + if err := m.buildParams(); err != nil { + return Manifest{}, err + } + order, err := manifestParamOrder(path) + if err != nil { + return Manifest{}, err + } + m.paramOrder = order + if err := validateHealth(path, m.Health); err != nil { + return Manifest{}, err + } return m, nil } +// manifestParamOrder reads only table headers. The TOML decoder intentionally +// normalizes params into a map, but the form should follow the author's +// declaration order without changing the sorted public recipe projection. +func manifestParamOrder(path string) ([]string, error) { + b, err := os.ReadFile(path) + if err != nil { + return nil, err + } + var order []string + for _, line := range strings.Split(string(b), "\n") { + line = strings.TrimSpace(line) + if !strings.HasPrefix(line, "[params.") || !strings.HasSuffix(line, "]") { + continue + } + name := strings.TrimSuffix(strings.TrimPrefix(line, "[params."), "]") + name = strings.Trim(name, "\"") + if paramName.MatchString(name) { + order = append(order, name) + } + } + return order, nil +} + +// manifestSchema distinguishes an absent schema from an explicit zero. The +// public Manifest.Schema field remains an int for callers, so a second decode +// into a pointer is the boundary that preserves this distinction. +func manifestSchema(path string) (int, bool, error) { + var raw struct { + Schema *int `toml:"schema"` + } + if err := tomlx.Decode(path, &raw, tomlx.Warn(io.Discard)); err != nil { + return 0, false, err + } + if raw.Schema == nil { + return 0, false, nil + } + return *raw.Schema, true, nil +} + +// buildParams turns raw TOML declarations into the normalized parameter map. +func (m *Manifest) buildParams() error { + m.Params = make(map[string]Param, len(m.ParamsRaw)) + for name, raw := range m.ParamsRaw { + if !paramName.MatchString(name) { + return fmt.Errorf("%s: param %q must match [a-z][a-z0-9_]*", m.Name, name) + } + if !containsString(validParamTypes, raw.Type) { + return fmt.Errorf("%s.%s: type %q is not one of %s", m.Name, name, raw.Type, strings.Join(validParamTypes, ", ")) + } + def, err := renderDefault(raw.Type, raw.Default) + if err != nil { + return fmt.Errorf("%s.%s: %w", m.Name, name, err) + } + if raw.Type == "secret" && raw.Default != nil { + return fmt.Errorf("%s.%s: a secret has no default", m.Name, name) + } + if raw.Type == "enum" { + if len(raw.Values) == 0 { + return fmt.Errorf("%s.%s: an enum needs values", m.Name, name) + } + if raw.Default != nil && !containsString(raw.Values, def) { + return fmt.Errorf("%s.%s: %q is not one of %s", m.Name, name, def, strings.Join(raw.Values, ", ")) + } + } + if raw.Default == nil && !raw.Required { + return fmt.Errorf("%s.%s: needs a default or required = true", m.Name, name) + } + m.Params[name] = Param{ + Name: name, Type: raw.Type, Default: def, Help: raw.Help, + Required: raw.Required, Values: raw.Values, + } + } + return nil +} + +// renderDefault converts TOML's typed literal into the guest string form. +func renderDefault(typ string, v any) (string, error) { + if v == nil { + return "", nil + } + switch typ { + case "int": + switch n := v.(type) { + case int64: + return strconv.FormatInt(n, 10), nil + case int: + return strconv.Itoa(n), nil + default: + return "", fmt.Errorf("default %v is not an integer", v) + } + case "bool": + b, ok := v.(bool) + if !ok { + return "", fmt.Errorf("default %v is not a boolean", v) + } + return strconv.FormatBool(b), nil + default: + s, ok := v.(string) + if !ok { + return "", fmt.Errorf("default %v is not a string", v) + } + return s, nil + } +} + +func containsString(values []string, wanted string) bool { + for _, value := range values { + if value == wanted { + return true + } + } + return false +} + +func validateHealth(path string, h Health) error { + if h.Check == "" && h.Timeout == "" { + return nil + } + if h.Check == "" { + return fmt.Errorf("%s: health.check is required when health.timeout is set", path) + } + if h.Timeout == "" { + return nil + } + d, err := time.ParseDuration(h.Timeout) + if err != nil || d <= 0 { + return fmt.Errorf("%s: health.timeout %q is not a positive duration", path, h.Timeout) + } + return nil +} + // ManifestFor resolves name (an entry in the recipes root, the same // identifier VM.Recipes/ApplyOpts.Only use) to its recipe.toml manifest // (docs/recipe-spec-v2.md). diff --git a/internal/recipes/manifest_v3_test.go b/internal/recipes/manifest_v3_test.go new file mode 100644 index 00000000..e71237dd --- /dev/null +++ b/internal/recipes/manifest_v3_test.go @@ -0,0 +1,237 @@ +package recipes + +import ( + "slices" + "strings" + "testing" + "time" +) + +const v3Manifest = `schema = 3 +name = "docker" +script = "install.sh" + +[params.user] +type = "string" +default = "dev" +help = "account added to the docker group" + +[params.port] +type = "int" +default = 2375 + +[params.tls] +type = "bool" +default = true + +[params.channel] +type = "enum" +values = ["stable", "test"] +default = "stable" + +[params.authkey] +type = "secret" +required = true + +[outputs] +socket = "path of the docker socket" +zsocket = "a second output" + +[health] +check = "docker info" +timeout = "45s" +` + +func TestParseManifestV3(t *testing.T) { + m, err := ParseManifest(writeManifestFile(t, t.TempDir(), v3Manifest)) + if err != nil { + t.Fatal(err) + } + if m.Schema != 3 { + t.Errorf("Schema = %d, want 3", m.Schema) + } + want := map[string]string{"user": "dev", "port": "2375", "tls": "true", "channel": "stable", "authkey": ""} + for name, def := range want { + p, ok := m.Params[name] + if !ok { + t.Fatalf("no param %q", name) + } + if p.Default != def { + t.Errorf("%s default = %q, want %q", name, p.Default, def) + } + if p.Name != name { + t.Errorf("%s Name = %q, want the map key", name, p.Name) + } + } + if !m.Params["authkey"].Required { + t.Error("authkey is not required") + } + if m.Outputs["socket"] == "" { + t.Error("outputs.socket has no help text") + } + if m.Health.Check != "docker info" || m.Health.Duration() != 45*time.Second { + t.Errorf("health = %+v", m.Health) + } + if got := m.SecretNames(); len(got) != 1 || got[0] != "authkey" { + t.Errorf("SecretNames = %v, want [authkey]", got) + } + params := m.SortedParams() + paramNames := make([]string, len(params)) + for i, p := range params { + paramNames[i] = p.Name + } + if want := []string{"authkey", "channel", "port", "tls", "user"}; !slices.Equal(paramNames, want) { + t.Errorf("SortedParams names = %v, want %v", paramNames, want) + } + outputs := m.SortedOutputs() + outputNames := make([]string, len(outputs)) + for i, output := range outputs { + outputNames[i] = output.Name + } + if want := []string{"socket", "zsocket"}; !slices.Equal(outputNames, want) { + t.Errorf("SortedOutputs names = %v, want %v", outputNames, want) + } +} + +// A recipe without [health] declares no check, and 30s is what a recipe that +// declares one but no timeout gets. +func TestHealthTimeoutDefaults(t *testing.T) { + if (Health{}).Duration() != 0 { + t.Error("an undeclared health check has no timeout") + } + if (Health{Check: "true"}).Duration() != 30*time.Second { + t.Error("a declared check defaults to 30s") + } + body := "schema = 3\nname = \"x\"\nscript = \"i.sh\"\n[health]\ncheck = \"true\"\n" + m, err := ParseManifest(writeManifestFile(t, t.TempDir(), body)) + if err != nil { + t.Fatal(err) + } + if got := m.Health.Duration(); got != 30*time.Second { + t.Errorf("parsed health duration = %s, want 30s", got) + } +} + +func TestParseManifestV3Errors(t *testing.T) { + tests := []struct{ name, body, want string }{ + { + name: "no default and not required", + body: "schema = 3\nname = \"x\"\nscript = \"i.sh\"\n[params.a]\ntype = \"string\"\n", + want: `x.a: needs a default or required = true`, + }, + { + name: "bad type", + body: "schema = 3\nname = \"x\"\nscript = \"i.sh\"\n[params.a]\ntype = \"float\"\ndefault = \"1\"\n", + want: `x.a: type "float" is not one of string, int, bool, enum, secret`, + }, + { + name: "bad param name", + body: "schema = 3\nname = \"x\"\nscript = \"i.sh\"\n[params.Auth-Key]\ntype = \"string\"\ndefault = \"a\"\n", + want: `x: param "Auth-Key" must match [a-z][a-z0-9_]*`, + }, + { + name: "enum default not in values", + body: "schema = 3\nname = \"x\"\nscript = \"i.sh\"\n[params.a]\ntype = \"enum\"\nvalues = [\"p\", \"q\"]\ndefault = \"r\"\n", + want: `x.a: "r" is not one of p, q`, + }, + { + name: "secret with a default", + body: "schema = 3\nname = \"x\"\nscript = \"i.sh\"\n[params.a]\ntype = \"secret\"\ndefault = \"hunter2\"\n", + want: `x.a: a secret has no default`, + }, + { + name: "unsupported schema", + body: "schema = 4\nname = \"x\"\nscript = \"i.sh\"\n", + want: `schema 4 is newer than this stoat (3)`, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + _, err := ParseManifest(writeManifestFile(t, t.TempDir(), tt.body)) + if err == nil || !strings.Contains(err.Error(), tt.want) { + t.Fatalf("err = %v, want it to contain %q", err, tt.want) + } + }) + } +} + +func TestParseManifestRejectsInvalidHealthTimeout(t *testing.T) { + for _, timeout := range []string{"soon", "0s", "-1s"} { + t.Run(timeout, func(t *testing.T) { + body := "schema = 3\nname = \"x\"\nscript = \"i.sh\"\n[health]\ncheck = \"true\"\ntimeout = \"" + timeout + "\"\n" + _, err := ParseManifest(writeManifestFile(t, t.TempDir(), body)) + if err == nil || !strings.Contains(err.Error(), "health.timeout") { + t.Fatalf("err = %v, want a health.timeout validation error", err) + } + }) + } + for _, timeout := range []string{"soon", "30s"} { + t.Run("without-check/"+timeout, func(t *testing.T) { + body := "schema = 3\nname = \"x\"\nscript = \"i.sh\"\n[health]\ntimeout = \"" + timeout + "\"\n" + _, err := ParseManifest(writeManifestFile(t, t.TempDir(), body)) + if err == nil { + t.Fatalf("timeout %q without a check was accepted", timeout) + } + }) + } +} + +func TestParseManifestSchemaBoundaries(t *testing.T) { + base := "name = \"x\"\nscript = \"i.sh\"\n" + for _, tt := range []struct { + name string + body string + want int + }{ + {name: "absent defaults to schema 2", body: base, want: 2}, + {name: "explicit schema 2", body: "schema = 2\n" + base, want: 2}, + {name: "explicit schema 3", body: "schema = 3\n" + base, want: 3}, + } { + t.Run(tt.name, func(t *testing.T) { + m, err := ParseManifest(writeManifestFile(t, t.TempDir(), tt.body)) + if err != nil { + t.Fatal(err) + } + if m.Schema != tt.want { + t.Errorf("Schema = %d, want %d", m.Schema, tt.want) + } + }) + } + for _, schema := range []string{"0", "1", "-1"} { + t.Run("reject-explicit-schema-"+schema, func(t *testing.T) { + _, err := ParseManifest(writeManifestFile(t, t.TempDir(), "schema = "+schema+"\n"+base)) + if err == nil { + t.Fatalf("explicit schema %s was accepted", schema) + } + }) + } + + t.Run("absent schema carries no v3 data", func(t *testing.T) { + m, err := ParseManifest(writeManifestFile(t, t.TempDir(), base+"os = [\"alpine\"]\n")) + if err != nil { + t.Fatal(err) + } + if m.Schema != 2 { + t.Errorf("Schema = %d, want 2 for a manifest with no schema key", m.Schema) + } + if len(m.Params) != 0 || len(m.Outputs) != 0 || m.Health.Check != "" { + t.Errorf("schema 2 manifest carries v3 data: %+v", m) + } + }) + + for _, tt := range []struct { + name string + block string + }{ + {name: "params", block: "[params.user]\ntype = \"string\"\ndefault = \"dev\"\n"}, + {name: "outputs", block: "[outputs]\nsocket = \"path\"\n"}, + {name: "health", block: "[health]\ncheck = \"true\"\n"}, + } { + t.Run(tt.name, func(t *testing.T) { + body := "schema = 2\nname = \"x\"\nscript = \"i.sh\"\n" + tt.block + if _, err := ParseManifest(writeManifestFile(t, t.TempDir(), body)); err == nil { + t.Fatalf("schema 2 %s block was accepted", tt.name) + } + }) + } +} diff --git a/internal/recipes/params.go b/internal/recipes/params.go new file mode 100644 index 00000000..4b815999 --- /dev/null +++ b/internal/recipes/params.go @@ -0,0 +1,127 @@ +package recipes + +import ( + "fmt" + "sort" + "strconv" + "strings" +) + +// OutputDir is the guest directory used for per-recipe output files. +const OutputDir = "/tmp/.stoat-out" + +// Resolve merges manifest defaults with values stored for one recipe. Secret +// values come from the separate secrets map and never become VM parameters. +func Resolve(m Manifest, set map[string]string, secrets map[string]string) (map[string]string, error) { + params := m.SortedParams() + names := make([]string, len(params)) + for i, p := range params { + names[i] = p.Name + } + for name := range set { + if _, ok := m.Params[name]; !ok { + return nil, fmt.Errorf("%s: no param %q; has %s", m.Name, name, strings.Join(names, ", ")) + } + } + + out := make(map[string]string, len(params)) + for _, p := range params { + value, present := "", false + if p.Type == "secret" { + value, present = secrets[p.Name] + present = present && value != "" + } else if value, present = set[p.Name]; !present { + value = p.Default + present = value != "" + } + if !present { + if p.Required { + return nil, requiredUnset(m.Name, p) + } + out[p.Name] = "" + continue + } + if err := Validate(m, p.Name, value); err != nil { + return nil, err + } + out[p.Name] = value + } + return out, nil +} + +func requiredUnset(recipe string, p Param) error { + if p.Type == "secret" { + return fmt.Errorf("%s.%s: required secret is unset; run stoat update --secret %s.%s", recipe, p.Name, recipe, p.Name) + } + return fmt.Errorf("%s.%s: required param is unset; run stoat update --set %s.%s=VALUE", recipe, p.Name, recipe, p.Name) +} + +// Validate checks one declared parameter value against its manifest type. +func Validate(m Manifest, name, value string) error { + p, ok := m.Params[name] + if !ok { + names := make([]string, 0, len(m.Params)) + for _, declared := range m.SortedParams() { + names = append(names, declared.Name) + } + return fmt.Errorf("%s: no param %q; has %s", m.Name, name, strings.Join(names, ", ")) + } + switch p.Type { + case "int": + if _, err := strconv.Atoi(value); err != nil { + return fmt.Errorf("%s.%s: %q is not an integer", m.Name, name, value) + } + case "bool": + if value != "true" && value != "false" { + return fmt.Errorf("%s.%s: %q is not true or false", m.Name, name, value) + } + case "enum": + if !containsString(p.Values, value) { + return fmt.Errorf("%s.%s: %q is not one of %s", m.Name, name, value, strings.Join(p.Values, ", ")) + } + } + return nil +} + +// RecipeHash covers the script, resolved non-secret parameters, and names of +// set secret parameters. Secret values never enter this digest. +func RecipeHash(name, osName string, params map[string]string, secretNames []string) (string, error) { + body, err := ScriptBody(name, osName) + if err != nil { + return "", err + } + if len(params) == 0 && len(secretNames) == 0 { + return sum([]byte(body)), nil + } + var b strings.Builder + b.WriteString(body) + paramNames := make([]string, 0, len(params)) + for name := range params { + paramNames = append(paramNames, name) + } + sort.Strings(paramNames) + for _, name := range paramNames { + fmt.Fprintf(&b, "\n\x00param %s=%s", name, params[name]) + } + secretNames = append([]string(nil), secretNames...) + sort.Strings(secretNames) + for _, name := range secretNames { + fmt.Fprintf(&b, "\n\x00secret %s", name) + } + return sum([]byte(b.String())), nil +} + +// Env renders STOAT_RECIPE and sorted parameter variables for a guest shell. +func Env(recipe string, params map[string]string) []string { + names := make([]string, 0, len(params)) + for name := range params { + names = append(names, name) + } + sort.Strings(names) + env := make([]string, 0, len(names)+1) + env = append(env, "STOAT_RECIPE="+recipe) + for _, name := range names { + env = append(env, "STOAT_PARAM_"+strings.ToUpper(name)+"="+params[name]) + } + return env +} diff --git a/internal/recipes/params_test.go b/internal/recipes/params_test.go new file mode 100644 index 00000000..2ac1a6d7 --- /dev/null +++ b/internal/recipes/params_test.go @@ -0,0 +1,145 @@ +package recipes + +import ( + "os" + "path/filepath" + "strings" + "testing" +) + +func v3ParamsFixture(t *testing.T) Manifest { + t.Helper() + m, err := ParseManifest(writeManifestFile(t, t.TempDir(), v3Manifest)) + if err != nil { + t.Fatal(err) + } + return m +} + +func TestResolveFillsDefaults(t *testing.T) { + m := v3ParamsFixture(t) + got, err := Resolve(m, map[string]string{"user": "alice"}, map[string]string{"authkey": "tskey-abc"}) + if err != nil { + t.Fatal(err) + } + want := map[string]string{ + "user": "alice", "port": "2375", "tls": "true", + "channel": "stable", "authkey": "tskey-abc", + } + for k, v := range want { + if got[k] != v { + t.Errorf("%s = %q, want %q", k, got[k], v) + } + } +} + +func TestResolveErrors(t *testing.T) { + m := v3ParamsFixture(t) + tests := []struct { + name string + set map[string]string + secrets map[string]string + want string + }{ + { + name: "unknown param", set: map[string]string{"usr": "dev"}, + secrets: map[string]string{"authkey": "k"}, + want: `docker: no param "usr"; has authkey, channel, port, tls, user`, + }, + { + name: "enum mismatch", set: map[string]string{"channel": "beta"}, + secrets: map[string]string{"authkey": "k"}, + want: `docker.channel: "beta" is not one of stable, test`, + }, + { + name: "int mismatch", set: map[string]string{"port": "http"}, + secrets: map[string]string{"authkey": "k"}, + want: `docker.port: "http" is not an integer`, + }, + { + name: "bool mismatch", set: map[string]string{"tls": "yes"}, + secrets: map[string]string{"authkey": "k"}, + want: `docker.tls: "yes" is not true or false`, + }, + { + name: "required secret unset", set: map[string]string{}, + want: `docker.authkey: required secret is unset; run stoat update --secret docker.authkey`, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + _, err := Resolve(m, tt.set, tt.secrets) + if err == nil || !strings.Contains(err.Error(), tt.want) { + t.Fatalf("err = %v, want it to contain %q", err, tt.want) + } + }) + } +} + +func TestEnvIsSortedAndUpperCased(t *testing.T) { + got := Env("docker", map[string]string{"user": "dev", "channel": "stable"}) + want := []string{"STOAT_RECIPE=docker", "STOAT_PARAM_CHANNEL=stable", "STOAT_PARAM_USER=dev"} + if len(got) != len(want) { + t.Fatalf("got %v, want %v", got, want) + } + for i := range want { + if got[i] != want[i] { + t.Errorf("[%d] = %q, want %q", i, got[i], want[i]) + } + } +} + +func TestRecipeHashCoversParamsAndSecretNames(t *testing.T) { + t.Setenv("STOAT_HOME", t.TempDir()) + installParamFixtureRecipe(t, "docker", v3Manifest, "#!/bin/sh\nset -e\n") + + base, err := RecipeHash("docker", "alpine", map[string]string{"user": "dev"}, []string{"authkey"}) + if err != nil { + t.Fatal(err) + } + changedParam, err := RecipeHash("docker", "alpine", map[string]string{"user": "bob"}, []string{"authkey"}) + if err != nil { + t.Fatal(err) + } + if changedParam == base { + t.Error("a changed param left the hash unchanged") + } + addedSecret, err := RecipeHash("docker", "alpine", map[string]string{"user": "dev"}, []string{"authkey", "extra"}) + if err != nil { + t.Fatal(err) + } + if addedSecret == base { + t.Error("a newly set secret left the hash unchanged") + } +} + +func TestRecipeHashMatchesScriptHashWithoutParams(t *testing.T) { + t.Setenv("STOAT_HOME", t.TempDir()) + installParamFixtureRecipe(t, "xfce", "name = \"xfce\"\nscript = \"install.sh\"\n", "#!/bin/sh\nset -e\n") + + want, err := ScriptHash("xfce", "alpine") + if err != nil { + t.Fatal(err) + } + got, err := RecipeHash("xfce", "alpine", nil, nil) + if err != nil { + t.Fatal(err) + } + if got != want { + t.Errorf("RecipeHash = %s, ScriptHash = %s", got, want) + } +} + +func installParamFixtureRecipe(t *testing.T, name, manifest, script string) { + t.Helper() + d := filepath.Join(dir(), name) + if err := os.MkdirAll(d, 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(d, "recipe.toml"), []byte(manifest), 0o644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(d, "install.sh"), []byte(script), 0o755); err != nil { + t.Fatal(err) + } +} diff --git a/internal/recipes/samples.go b/internal/recipes/samples.go new file mode 100644 index 00000000..ed4fa19b --- /dev/null +++ b/internal/recipes/samples.go @@ -0,0 +1,10 @@ +package recipes + +import _ "embed" + +// SampleManifest is the annotated recipe.toml copied by `stoat recipe new`. +// Keeping the source beside the embedded asset makes the docs and scaffold +// share one contract; docs/reference/samples/recipe.toml links to this file. +// +//go:embed samples/recipe.toml +var SampleManifest string diff --git a/internal/recipes/samples/recipe.toml b/internal/recipes/samples/recipe.toml new file mode 100644 index 00000000..578613a0 --- /dev/null +++ b/internal/recipes/samples/recipe.toml @@ -0,0 +1,50 @@ +# Every field a recipe.toml can carry. `stoat recipe new` copies this file and +# fills name and os; delete what a recipe does not need. + +schema = 3 # int; default 2; author writes, scaffold preserves. Enables v3 fields. +name = "example" # string; required; author writes, scaffold replaces with the directory name. +description = "one line" # string; default empty; author writes. Shown by `stoat recipes` and `recipe show`. +version = "1.0.0" # string; default empty; author writes. Changes re-run `run = "once"` recipes. +os = ["alpine"] # string[]; default []; author writes. Empty means every guest. +requires = ["apk"] # string[]; default []; author writes. Capabilities from the guest file. +stage = "provision" # string; default "provision"; author writes: "install" or "provision". +script = "install.sh" # string; required; author writes. Relative to this directory. +auto = false # bool; default false; author writes. Offered pre-checked in the TUI picker. +run = "once" # string; default "once"; author writes: "once", "always", or "manual". +reboot = false # bool; default false; author writes. Reboot a disk VM after this recipe. +runtime = "sh" # string; default "sh"; author writes: "sh" or "python3". +depends = [] # string[]; default []; author writes. Recipe names that run first. + +[scripts] # table; default {}; author writes. Per-guest overrides of `script`. +alpine = "install-alpine.sh" # string path; author writes; scaffold creates the declared file. + +[params.user] # table; author writes; scaffold preserves. +type = "string" # string; required; author writes: string, int, bool, enum, or secret. +default = "dev" # string; default "dev"; author writes; guest receives STOAT_PARAM_USER. +required = false # bool; default false; author writes. +help = "account to create" # string; default empty; author writes; shown by the TUI and `recipe show`. + +[params.port] +type = "int" # string; required; author writes. +default = 2375 # int; default 2375; author writes; guest receives "2375". + +[params.tls] +type = "bool" # string; required; author writes. +default = true # bool; default true; author writes; guest receives "true". + +[params.channel] +type = "enum" # string; required; author writes. +values = ["stable", "test"] # string[]; default []; author writes allowed values. +default = "stable" # string; default first choice is not implicit; author writes. + +[params.authkey] +type = "secret" # string; required; author writes; value is stored only in secrets.toml. +required = true # bool; default false; author writes. +help = "auth key" # string; default empty; author writes; shown by the TUI. + +[outputs] # table; default {}; author writes. Script writes name=value to $STOAT_OUTPUT. +socket = "path of the socket" # string help text; author writes; value is recorded after apply. + +[health] +check = "docker info" # string; default empty (no health check); author writes; exit 0 means healthy. +timeout = "30s" # duration string; default "30s" when check is set; author writes. diff --git a/internal/recipes/samples_test.go b/internal/recipes/samples_test.go new file mode 100644 index 00000000..5e19c2bc --- /dev/null +++ b/internal/recipes/samples_test.go @@ -0,0 +1,325 @@ +package recipes_test + +import ( + "os" + "os/exec" + "path/filepath" + "strings" + "testing" + + "github.com/novusedge/stoat/internal/config" + "github.com/novusedge/stoat/internal/guest" + "github.com/novusedge/stoat/internal/recipes" + "github.com/novusedge/stoat/internal/tomlx" +) + +// The samples are the format documentation. Decoding each one in Reject mode +// against its real struct means a field renamed in Go breaks this test rather +// than quietly leaving the docs wrong. +func TestSamplesDecodeInRejectMode(t *testing.T) { + root := "../../docs/reference/samples" + t.Run("recipe.toml", func(t *testing.T) { + var m recipes.Manifest + if err := tomlx.Decode(filepath.Join(root, "recipe.toml"), &m, tomlx.Reject); err != nil { + t.Fatal(err) + } + }) + t.Run("vm.toml", func(t *testing.T) { + var v config.VM + if err := tomlx.Decode(filepath.Join(root, "vm.toml"), &v, tomlx.Reject); err != nil { + t.Fatal(err) + } + }) + t.Run("guest.toml", func(t *testing.T) { + var o guest.OS + if err := tomlx.Decode(filepath.Join(root, "guest.toml"), &o, tomlx.Reject); err != nil { + t.Fatal(err) + } + }) +} + +// The canonical VM sample documents both stoat-owned applied tables. The +// nested outputs table is independently rewritten, so it needs the same +// ownership marker as its parent. +func TestVMSampleAppliedOutputsHasOwnershipComment(t *testing.T) { + body, err := os.ReadFile("../../docs/reference/samples/vm.toml") + if err != nil { + t.Fatal(err) + } + lines := strings.Split(string(body), "\n") + for _, target := range []string{"[applied.docker]", "[applied.docker.outputs]"} { + found := false + for i, line := range lines { + if strings.TrimSpace(line) != target { + continue + } + found = true + j := i - 1 + for j >= 0 && strings.TrimSpace(lines[j]) == "" { + j-- + } + if j < 0 || !strings.Contains(strings.ToLower(lines[j]), "written by stoat; do not edit") { + t.Errorf("sample table %q lacks an ownership comment", target) + } + break + } + if !found { + t.Errorf("sample missing table %q", target) + } + } +} + +// The sample must also survive the manifest's own validation, not only the +// decoder: a sample that documents an illegal param teaches the wrong thing. +func TestSampleRecipeParses(t *testing.T) { + m, err := recipes.ParseManifest("../../docs/reference/samples/recipe.toml") + if err != nil { + t.Fatal(err) + } + if len(m.Params) != 5 || len(m.Outputs) != 1 || m.Health.Check == "" { + t.Errorf("sample lost coverage: %+v", m) + } +} + +// recipe new must create every script path declared by the canonical sample; +// a manifest that names an OS override without its file is not a usable +// scaffold. +func TestNewScaffoldsEveryDeclaredScriptPath(t *testing.T) { + canonical := filepath.Join(t.TempDir(), "recipe.toml") + if err := os.WriteFile(canonical, []byte(recipes.SampleManifest), 0o644); err != nil { + t.Fatal(err) + } + m, err := recipes.ParseManifest(canonical) + if err != nil { + t.Fatal(err) + } + + t.Setenv("STOAT_HOME", t.TempDir()) + dir, err := recipes.New("mine", "alpine", "") + if err != nil { + t.Fatal(err) + } + paths := map[string]bool{m.Script: true} + for _, script := range m.Scripts { + paths[script] = true + } + for script := range paths { + if _, err := os.Stat(filepath.Join(dir, script)); err != nil { + t.Errorf("declared script %q was not scaffolded: %v", script, err) + } + } +} + +func TestBundledTailscaleAuthkeyIsRequiredSecret(t *testing.T) { + t.Setenv("STOAT_HOME", t.TempDir()) + if err := recipes.Install(); err != nil { + t.Fatal(err) + } + m, ok, err := recipes.ManifestFor("tailscale") + if err != nil { + t.Fatal(err) + } + if !ok { + t.Fatal("bundled tailscale manifest missing") + } + authkey, ok := m.Params["authkey"] + if !ok { + t.Fatal("tailscale manifest has no authkey parameter") + } + if authkey.Type != "secret" || !authkey.Required || authkey.Default != "" { + t.Fatalf("tailscale authkey = %+v, want required secret with no default", authkey) + } +} + +func TestBundledRecipeScriptsUseChangedParameterVerbs(t *testing.T) { + t.Setenv("STOAT_HOME", t.TempDir()) + if err := recipes.Install(); err != nil { + t.Fatal(err) + } + cases := []struct { + name string + want []string + }{ + {name: "docker", want: []string{"STOAT_PARAM_USER", "STOAT_OUTPUT", "socket=/var/run/docker.sock"}}, + {name: "tailscale", want: []string{"STOAT_PARAM_AUTHKEY", "tailscale up --authkey"}}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + m, ok, err := recipes.ManifestFor(tc.name) + if err != nil || !ok { + t.Fatalf("ManifestFor(%q) = ok %v, err %v", tc.name, ok, err) + } + body, err := m.ScriptContent("alpine") + if err != nil { + t.Fatal(err) + } + for _, want := range tc.want { + if !strings.Contains(body, want) { + t.Errorf("%s script missing %q", tc.name, want) + } + } + }) + } +} + +// The Debian override and the default Docker script are real shell programs, +// so run each twice against command fakes. The fakes redirect every intended +// write below a temporary root and make curl non-networking; the second run +// proves the keyring is deliberately overwritten rather than prompting. +func TestBundledDockerScriptsRerunWithNonInteractiveKeyring(t *testing.T) { + t.Setenv("STOAT_HOME", t.TempDir()) + if err := recipes.Install(); err != nil { + t.Fatal(err) + } + m, ok, err := recipes.ManifestFor("docker") + if err != nil || !ok { + t.Fatalf("ManifestFor(docker) = ok %v, err %v", ok, err) + } + for _, tc := range []struct { + name string + os string + }{ + {name: "debian-override", os: "debian"}, + {name: "default", os: "unknown-os"}, + } { + t.Run(tc.name, func(t *testing.T) { + body, err := m.ScriptContent(tc.os) + if err != nil { + t.Fatal(err) + } + fakeRoot := t.TempDir() + bin := filepath.Join(fakeRoot, "bin") + if err := os.MkdirAll(bin, 0o755); err != nil { + t.Fatal(err) + } + writeBundledFakeCommands(t, bin) + env := append(os.Environ(), + "PATH="+bin+string(os.PathListSeparator)+os.Getenv("PATH"), + "FAKE_ROOT="+fakeRoot, + "STOAT_PARAM_USER=dev", + "STOAT_OUTPUT="+filepath.Join(fakeRoot, "output"), + ) + for run := 0; run < 2; run++ { + cmd := exec.Command("sh", "-eu", "-c", body) + cmd.Env = env + if output, err := cmd.CombinedOutput(); err != nil { + t.Fatalf("%s run %d failed: %v\n%s", tc.name, run+1, err, output) + } + keyring := filepath.Join(fakeRoot, "etc", "apt", "keyrings", "docker.gpg") + got, err := os.ReadFile(keyring) + if err != nil { + t.Fatalf("run %d keyring missing: %v", run+1, err) + } + if string(got) != "fake-key\n" { + t.Fatalf("run %d keyring = %q, want overwritten fake key", run+1, got) + } + } + calls, err := os.ReadFile(filepath.Join(fakeRoot, "calls")) + if err != nil { + t.Fatal(err) + } + curlCalls := 0 + for _, line := range strings.Split(strings.TrimSpace(string(calls)), "\n") { + if strings.HasPrefix(line, "curl ") { + curlCalls++ + } + } + if curlCalls != 2 { + t.Fatalf("curl calls = %q, want one per run", calls) + } + if !strings.Contains(string(calls), "gpg-existing=true") { + t.Fatalf("gpg did not exercise an existing-keyring overwrite: %q", calls) + } + if output, err := os.ReadFile(filepath.Join(fakeRoot, "output")); err != nil || strings.Count(string(output), "socket=/var/run/docker.sock\n") != 2 { + t.Fatalf("STOAT_OUTPUT = %q, err %v, want one output per run", output, err) + } + }) + } +} + +func writeBundledFakeCommands(t *testing.T, bin string) { + t.Helper() + commands := map[string]string{ + "apt-get": `#!/bin/sh +printf 'apt-get %s\n' "$*" >> "$FAKE_ROOT/calls" +`, + "curl": `#!/bin/sh +printf 'curl %s\n' "$*" >> "$FAKE_ROOT/calls" +printf 'fake-key\n' +`, + "dpkg": `#!/bin/sh +printf 'amd64\n' +`, + "docker": `#!/bin/sh +if [ "${1:-}" = info ]; then exit 0; fi +if [ "${1:-}" = version ]; then printf '24.0.0\n'; fi +`, + "gpg": `#!/bin/sh +out= +batch=false +yes=false +no_tty=false +while [ "$#" -gt 0 ]; do + case "$1" in + --batch) batch=true;; + --yes) yes=true;; + --no-tty) no_tty=true;; + -o) out=$2; shift;; + esac + shift +done +target=$FAKE_ROOT$out +existing=false +if [ -e "$target" ]; then existing=true; fi +printf 'gpg-existing=%s\n' "$existing" >> "$FAKE_ROOT/calls" +if [ "$batch" != true ] || [ "$yes" != true ] || [ "$no_tty" != true ]; then + printf 'gpg missing noninteractive overwrite flags\n' >&2 + exit 43 +fi +mkdir -p "$(dirname "$target")" +cat > "$FAKE_ROOT$out" +`, + "id": `#!/bin/sh +exit 1 +`, + "install": `#!/bin/sh +dir=false +out= +while [ "$#" -gt 0 ]; do + case "$1" in + -d) dir=true; shift;; + -m) shift 2;; + -*) shift;; + *) out=$1; shift;; + esac +done +target=$FAKE_ROOT$out +if $dir; then mkdir -p "$target"; else mkdir -p "$(dirname "$target")"; : > "$target"; fi +`, + "systemctl": `#!/bin/sh +printf 'systemctl %s\n' "$*" >> "$FAKE_ROOT/calls" +`, + "tee": `#!/bin/sh +target=$1 +mkdir -p "$FAKE_ROOT$(dirname "$target")" +cat > "$FAKE_ROOT$target" +`, + "useradd": `#!/bin/sh +printf 'useradd %s\n' "$*" >> "$FAKE_ROOT/calls" +`, + "usermod": `#!/bin/sh +printf 'usermod %s\n' "$*" >> "$FAKE_ROOT/calls" +`, + "chmod": `#!/bin/sh +exit 0 +`, + "sleep": `#!/bin/sh +exit 0 +`, + } + for name, body := range commands { + if err := os.WriteFile(filepath.Join(bin, name), []byte(body), 0o755); err != nil { + t.Fatal(err) + } + } +} diff --git a/internal/recipes/scaffold.go b/internal/recipes/scaffold.go index f8a21938..3e9535c5 100644 --- a/internal/recipes/scaffold.go +++ b/internal/recipes/scaffold.go @@ -14,14 +14,6 @@ import ( // with a recipe.toml", and the only real problem was that nobody could tell. func Dir() string { return dir() } -// manifestTemplate is the recipe.toml skeleton `stoat recipe new` writes. -const manifestTemplate = `name = "%s" -description = "TODO: describe what this recipe does" -os = ["%s"] -stage = "provision" -script = "install.sh" -` - // shellTemplate is the install.sh skeleton. It carries two things every // bundled shell recipe needs, that a first-timer would not think to add. // @@ -63,6 +55,14 @@ func osSetup(osName string) (setup, install string) { return "", "# install: " } +// scaffoldManifest fills the sample's identity fields for a new recipe. The +// rest of the annotated sample is deliberately kept intact so new authors see +// every supported field and the declared override scripts can be scaffolded. +func scaffoldManifest(name, osName string) string { + out := strings.Replace(SampleManifest, `name = "example"`, fmt.Sprintf(`name = %q`, name), 1) + return strings.Replace(out, `os = ["alpine"]`, fmt.Sprintf(`os = [%q]`, osName), 1) +} + // New writes a skeleton recipe directory and returns its path. It refuses to // overwrite. Install() already promises never to clobber a user's edits. // A scaffold command that destroys the recipe you were working on is a @@ -90,14 +90,27 @@ func New(name, osName, _ string) (string, error) { return "", err } - manifest := fmt.Sprintf(manifestTemplate, name, osName) + manifest := scaffoldManifest(name, osName) if err := os.WriteFile(filepath.Join(recipeDir, "recipe.toml"), []byte(manifest), 0o644); err != nil { return "", err } setup, install := osSetup(osName) script := fmt.Sprintf(shellTemplate, name, osName, setup, install) - if err := os.WriteFile(filepath.Join(recipeDir, "install.sh"), []byte(script), 0o755); err != nil { + m, err := ParseManifest(filepath.Join(recipeDir, "recipe.toml")) + if err != nil { + return "", err + } + paths := map[string]struct{}{m.Script: {}} + for _, path := range m.Scripts { + paths[path] = struct{}{} + } + for path := range paths { + if err := os.WriteFile(filepath.Join(recipeDir, path), []byte(script), 0o755); err != nil { + return "", err + } + } + if _, err := os.Stat(filepath.Join(recipeDir, m.Script)); err != nil { return "", err } diff --git a/internal/sshx/outputs.go b/internal/sshx/outputs.go new file mode 100644 index 00000000..1faa0dc1 --- /dev/null +++ b/internal/sshx/outputs.go @@ -0,0 +1,70 @@ +package sshx + +import ( + "context" + "errors" + "fmt" + "io" + "os/exec" + "sort" + "strings" + + "github.com/novusedge/stoat/internal/config" + "github.com/novusedge/stoat/internal/guest" + "github.com/novusedge/stoat/internal/recipes" +) + +// OutputDir is the guest directory used for per-recipe output files. +const OutputDir = recipes.OutputDir + +// ParseOutputs parses name=value lines and keeps undeclared names so a recipe +// result is not lost when its manifest was incomplete. Empty values count. +func ParseOutputs(declared map[string]string, body string) (map[string]string, []string) { + values := map[string]string{} + var undeclared []string + for _, line := range strings.Split(body, "\n") { + line = strings.TrimSpace(line) + if line == "" || strings.HasPrefix(line, "#") { + continue + } + name, value, ok := strings.Cut(line, "=") + if !ok || name == "" { + continue + } + values[name] = value + if _, ok := declared[name]; !ok { + undeclared = append(undeclared, name) + } + } + sort.Strings(undeclared) + return values, undeclared +} + +func collectOutputs(ctx context.Context, v *config.VM, name string, m recipes.Manifest, secrets []string, log io.Writer) error { + path := OutputDir + "/" + name + quoted := shellPath(path) + script := fmt.Sprintf("cat %s 2>/dev/null; rm -f %s", quoted, quoted) + // ssh joins its remote argv with spaces and the login shell re-splits + // it, so a multi-word script must travel as one already-quoted argv + // element or only its first word ends up under the escalation prefix. + remote := []string{"sh -c " + guest.ShQuote(script)} + out, err := exec.CommandContext(ctx, "ssh", Args(v, escalate(v, remote)...)...).Output() + if err != nil { + var ee *exec.ExitError + if errors.As(err, &ee) { + return fmt.Errorf("%w: %s", err, strings.TrimSpace(string(ee.Stderr))) + } + return err + } + values, undeclared := ParseOutputs(m.Outputs, redactString(string(out), secrets)) + for _, output := range undeclared { + fmt.Fprintf(log, "%s: output %q is not declared\n", m.Name, output) + } + if v.Applied == nil { + v.Applied = map[string]config.AppliedRecipe{} + } + a := v.Applied[name] + a.Outputs = values + v.Applied[name] = a + return nil +} diff --git a/internal/sshx/outputs_test.go b/internal/sshx/outputs_test.go new file mode 100644 index 00000000..ee9fabae --- /dev/null +++ b/internal/sshx/outputs_test.go @@ -0,0 +1,293 @@ +package sshx + +import ( + "context" + "fmt" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/novusedge/stoat/internal/config" +) + +func TestParseOutputs(t *testing.T) { + declared := map[string]string{"socket": "path of the docker socket"} + body := "socket=/var/run/docker.sock\nsock=/nope\n\n# a comment\nempty=\n" + vals, undeclared := ParseOutputs(declared, body) + + if vals["socket"] != "/var/run/docker.sock" { + t.Errorf("socket = %q", vals["socket"]) + } + if vals["sock"] != "/nope" { + t.Errorf("an undeclared output was dropped: %v", vals) + } + if len(undeclared) != 2 || !containsOutputName(undeclared, "sock") || !containsOutputName(undeclared, "empty") { + t.Errorf("undeclared = %v, want sock and empty", undeclared) + } + if v, ok := vals["empty"]; !ok || v != "" { + t.Errorf("an empty value was dropped: %v", vals) + } + if _, ok := vals["# a comment"]; ok { + t.Error("a comment line became an output") + } +} + +func TestParseOutputsSplitsAtTheFirstEquals(t *testing.T) { + vals, _ := ParseOutputs(map[string]string{"dsn": ""}, "dsn=postgres://u:p@h/db?a=b\n") + if vals["dsn"] != "postgres://u:p@h/db?a=b" { + t.Errorf("dsn = %q", vals["dsn"]) + } +} + +func TestParseOutputsStoresUndeclaredWithEmptyDeclarationMap(t *testing.T) { + vals, undeclared := ParseOutputs(map[string]string{}, "socket=/var/run/docker.sock\n") + if vals["socket"] != "/var/run/docker.sock" { + t.Fatalf("socket = %q, want the emitted value", vals["socket"]) + } + if len(undeclared) != 1 || undeclared[0] != "socket" { + t.Fatalf("undeclared = %v, want [socket]", undeclared) + } +} + +func TestProvisionKeepsSecretOutOfSSHArgvAndLogs(t *testing.T) { + root := t.TempDir() + t.Setenv("STOAT_HOME", root) + secret := "value with spaces 'quotes'; printf hacked" + valueFile := filepath.Join(root, "work", "guest-param") + installSSHRecipe(t, root, "docker", "schema = 3\nname = \"docker\"\nscript = \"install.sh\"\n[params.authkey]\ntype = \"secret\"\nrequired = true\n", "#!/bin/sh\nprintf '%s' \"$STOAT_PARAM_AUTHKEY\" > "+shellQuoteForTest(valueFile)+"\n") + + vmDir := filepath.Join(root, "work") + if err := os.MkdirAll(vmDir, 0o755); err != nil { + t.Fatal(err) + } + capture := filepath.Join(vmDir, "ssh-capture") + installSecretCheckingSSH(t, capture, valueFile, secret, true) + port := acceptOnly(t, "SSH-2.0-fake\r\n") + v := &config.VM{ + Name: "work", Dir: vmDir, OS: "alpine", SSHPort: port, + Recipes: []string{"docker"}, Applied: map[string]config.AppliedRecipe{}, + } + if err := config.SaveSecrets(vmDir, config.Secrets{"docker": {"authkey": secret}}); err != nil { + t.Fatal(err) + } + + provisionErr := Provision(context.Background(), v) + if provisionErr == nil { + t.Fatal("Provision succeeded, want the fake recipe failure") + } + if strings.Contains(provisionErr.Error(), secret) { + t.Fatalf("secret leaked in Provision error: %v", provisionErr) + } + log, err := os.ReadFile(v.ProvisionLogPath()) + if err != nil { + t.Fatal(err) + } + if strings.Contains(string(log), secret) { + t.Fatalf("secret leaked in provision log:\n%s", log) + } + captured, err := os.ReadFile(capture) + if err != nil { + t.Fatal(err) + } + text := string(captured) + if !strings.Contains(text, secret) { + t.Fatalf("fake ssh did not observe the secret in stdin:\n%s", text) + } + value, err := os.ReadFile(valueFile) + if err != nil { + t.Fatal(err) + } + if string(value) != secret { + t.Fatalf("secret changed while crossing the shell boundary: got %q, want %q", value, secret) + } + for _, line := range strings.Split(text, "\n") { + if strings.HasPrefix(line, "ARGV:") && strings.Contains(line, secret) { + t.Fatalf("secret appeared in ssh argv: %q", line) + } + } + split := len(secret) / 2 + for _, public := range []string{provisionErr.Error(), string(log)} { + for _, fragment := range []string{secret, secret[:split], secret[split:]} { + if strings.Contains(public, fragment) { + t.Fatalf("secret fragment %q leaked into public Provision sink %q", fragment, public) + } + } + } +} + +func TestProvisionExportsOutputToChildProcess(t *testing.T) { + root := t.TempDir() + t.Setenv("STOAT_HOME", root) + childOutput := filepath.Join(root, "child-output") + installSSHRecipe(t, root, "child", "schema = 3\nname = \"child\"\nscript = \"install.sh\"\n", "#!/bin/sh\nsh -c 'printf \"%s\" \"$STOAT_OUTPUT\" > "+shellQuoteForTest(childOutput)+"'\n") + guestOutputRoot := t.TempDir() + installExecutingSSH(t, guestOutputRoot) + port := acceptOnly(t, "SSH-2.0-fake\r\n") + v := &config.VM{Name: "work", Dir: filepath.Join(root, "work"), OS: "alpine", SSHPort: port, Recipes: []string{"child"}} + if err := os.MkdirAll(v.Dir, 0o755); err != nil { + t.Fatal(err) + } + if err := Provision(context.Background(), v); err != nil { + t.Fatal(err) + } + got, err := os.ReadFile(childOutput) + if err != nil { + t.Fatalf("child did not write STOAT_OUTPUT: %v", err) + } + want := filepath.Join(guestOutputRoot, "child") + if string(got) != want { + t.Fatalf("child STOAT_OUTPUT = %q, want %q", got, want) + } +} + +func TestProvisionStoresUndeclaredOutputsEvenWhenManifestDeclaresNone(t *testing.T) { + root := t.TempDir() + t.Setenv("STOAT_HOME", root) + const secret = "output-secret-value" + installSSHRecipe(t, root, "docker", "schema = 3\nname = \"docker\"\nscript = \"install.sh\"\n[params.authkey]\ntype = \"secret\"\nrequired = true\n", "#!/bin/sh\necho provision\n") + + vmDir := filepath.Join(root, "work") + if err := os.MkdirAll(vmDir, 0o755); err != nil { + t.Fatal(err) + } + installOutputSSH(t, "rogue="+secret+"\n") + port := acceptOnly(t, "SSH-2.0-fake\r\n") + v := &config.VM{ + Name: "work", Dir: vmDir, OS: "alpine", SSHPort: port, SSHUser: "stoat", + Recipes: []string{"docker"}, Applied: map[string]config.AppliedRecipe{}, + } + if err := config.SaveSecrets(vmDir, config.Secrets{"docker": {"authkey": secret}}); err != nil { + t.Fatal(err) + } + + if err := Provision(context.Background(), v); err != nil { + t.Fatal(err) + } + got, ok := v.Applied["docker"].Outputs["rogue"] + if !ok { + t.Fatalf("undeclared output was discarded: %v", v.Applied["docker"].Outputs) + } + if got == secret || strings.Contains(got, secret) { + t.Fatalf("secret leaked into captured output: %q", got) + } + if err := v.Save(); err != nil { + t.Fatal(err) + } + persisted, err := os.ReadFile(filepath.Join(vmDir, "vm.toml")) + if err != nil { + t.Fatal(err) + } + if strings.Contains(string(persisted), secret) { + t.Fatal("secret leaked into vm.toml through captured output") + } + log, err := os.ReadFile(v.ProvisionLogPath()) + if err != nil { + t.Fatal(err) + } + if !strings.Contains(string(log), `docker: output "rogue" is not declared`) { + t.Fatalf("missing undeclared-output warning:\n%s", log) + } +} + +func installSSHRecipe(t *testing.T, root, name, manifest, body string) { + t.Helper() + dir := filepath.Join(root, "recipes", name) + if err := os.MkdirAll(dir, 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(dir, "recipe.toml"), []byte(manifest), 0o644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(dir, "install.sh"), []byte(body), 0o755); err != nil { + t.Fatal(err) + } +} + +func installSecretCheckingSSH(t *testing.T, capture, valueFile, secret string, failRecipe bool) { + t.Helper() + bin := t.TempDir() + fail := "" + if failRecipe { + split := len(secret) / 2 + fail = fmt.Sprintf("secret=$(cat %s)\nprintf '%%s' \"$(printf '%%s' \"$secret\" | cut -c1-%d)\"\nsleep 0.05\nprintf '%%s\\n' \"$(printf '%%s' \"$secret\" | cut -c%d-)\"\nexit 1\n", shellQuoteForTest(valueFile), split, split+1) + } + script := "#!/bin/sh\n" + + "input=$(cat)\n" + "printf 'ARGV:%s\\n' \"$*\" >> " + shellQuoteForTest(capture) + "\n" + + "printf 'STDIN:%s\\n' \"$input\" >> " + shellQuoteForTest(capture) + "\n" + + "case \"$input\" in *STOAT_RECIPE=docker*)\n" + + "printf '%s' \"$input\" | sh -s\n" + fail + ";; esac\n" + if err := os.WriteFile(filepath.Join(bin, "ssh"), []byte(script), 0o755); err != nil { + t.Fatal(err) + } + t.Setenv("PATH", bin+string(os.PathListSeparator)+os.Getenv("PATH")) +} + +func installExecutingSSH(t *testing.T, outputRoot string) { + t.Helper() + bin := t.TempDir() + root := strings.ReplaceAll(outputRoot, "#", "\\#") + script := "#!/bin/sh\ninput=$(cat)\ncase \"$input\" in *'STOAT_RECIPE=child'*) safe=$(printf '%s' \"$input\" | sed 's#/tmp/.stoat-out#" + root + "#g'); printf '%s' \"$safe\" | sh -s;; esac\nexit 0\n" + if err := os.WriteFile(filepath.Join(bin, "ssh"), []byte(script), 0o755); err != nil { + t.Fatal(err) + } + t.Setenv("PATH", bin+string(os.PathListSeparator)+os.Getenv("PATH")) +} + +func containsOutputName(names []string, want string) bool { + for _, name := range names { + if name == want { + return true + } + } + return false +} + +// installOutputSSH models real ssh: it joins the argv after the user@host +// spec into one space-separated string and, only for the output read-back +// command (the "/tmp/.stoat-out/docker" path), hands that string to +// /bin/sh -c the way a real remote login shell re-splits it. A script built +// from several unquoted argv elements (the bug this guards against) falls +// apart here exactly as it would over a real connection: only the first word +// after "-c" reaches the inner "sh -c", and any escalation prefix covers only +// that word, not the rest of the line. Every other ssh call Provision makes +// (bootstrap, package refresh, recipe run) still no-ops, since this dev host +// has no real guest to run them against. +// +// The fake sudo strips leading flags and sets FAKE_SUDO before exec'ing the +// rest of argv; the fake rm refuses to run unless FAKE_SUDO is set, standing +// in for a root-owned output directory an unprivileged rm cannot touch. +func installOutputSSH(t *testing.T, output string) { + t.Helper() + bin := t.TempDir() + sshScript := "#!/bin/sh\n" + + "found=0\nremote=\"\"\n" + + "for a in \"$@\"; do\n" + + "\tif [ \"$found\" = 0 ]; then\n" + + "\t\tcase \"$a\" in *@*) found=1 ;; esac\n" + + "\t\tcontinue\n" + + "\tfi\n" + + "\tif [ -z \"$remote\" ]; then remote=$a; else remote=\"$remote $a\"; fi\n" + + "done\n" + + "case \"$remote\" in\n" + + "\t*/tmp/.stoat-out/docker*) exec sh -c \"$remote\" ;;\n" + + "esac\n" + + "exit 0\n" + sudoScript := "#!/bin/sh\n" + + "while [ $# -gt 0 ]; do\n" + + "\tcase \"$1\" in -*) shift ;; *) break ;; esac\n" + + "done\n" + + "FAKE_SUDO=1 exec \"$@\"\n" + rmScript := "#!/bin/sh\n" + + "if [ -z \"$FAKE_SUDO\" ]; then\n" + + "\tprintf \"rm: cannot remove '%s': Permission denied\\n\" \"$2\" >&2\n" + + "\texit 1\n" + + "fi\n" + catScript := "#!/bin/sh\nprintf '%s' " + shellQuoteForTest(output) + "\n" + for name, script := range map[string]string{"ssh": sshScript, "sudo": sudoScript, "rm": rmScript, "cat": catScript} { + if err := os.WriteFile(filepath.Join(bin, name), []byte(script), 0o755); err != nil { + t.Fatal(err) + } + } + t.Setenv("PATH", bin+string(os.PathListSeparator)+os.Getenv("PATH")) +} diff --git a/internal/sshx/sshx.go b/internal/sshx/sshx.go index 19dde014..7551d35e 100644 --- a/internal/sshx/sshx.go +++ b/internal/sshx/sshx.go @@ -8,6 +8,7 @@ import ( "net" "os" "os/exec" + "sort" "strings" "syscall" "time" @@ -224,6 +225,31 @@ func bannerReady(c net.Conn, budget time.Duration) bool { // react. This bounds that grace period rather than waiting on it forever. const recipeShutdownGrace = 5 * time.Second +// healthShutdownGrace is deliberately short: a health probe has no recipe +// state to preserve after its context expires, and a TERM-ignoring probe must +// not extend Wait's single health budget by the normal recipe grace period. +const healthShutdownGrace = 100 * time.Millisecond + +// RunCheck runs one command inside v's guest through the guest prelude, as +// the recipe's ssh user and under the guest's escalation. The command is +// sent over stdin so it does not become a local ssh argv element. +func RunCheck(ctx context.Context, v *config.VM, command string, timeout time.Duration) (string, error) { + ctx, cancel := context.WithTimeout(ctx, timeout) + defer cancel() + + var prelude string + if o, ok := guest.Lookup(v.OS); ok { + prelude = guest.Prelude(o, "sh") + } + body := prelude + "\n" + command + "\n" + cmd := exec.CommandContext(ctx, "ssh", Args(v, escalate(v, []string{"sh", "-s"})...)...) + cmd.Cancel = func() error { return cmd.Process.Signal(syscall.SIGTERM) } + cmd.WaitDelay = healthShutdownGrace + cmd.Stdin = strings.NewReader(body) + out, err := cmd.CombinedOutput() + return string(out), err +} + // Provision runs each of v's recipes over ssh, streaming output to // last-provision.log. The detail view tails that file on a ticker, so there // is no channel plumbing between this and the UI. @@ -315,6 +341,18 @@ func Provision(ctx context.Context, v *config.VM) (err error) { return err } fmt.Fprintf(log, "\n%s\n", RecipeMarker(name)) + m, haveManifest, err := recipes.ManifestFor(name) + if err != nil { + fmt.Fprintf(log, "FAILED: recipe %s: %v\n", name, err) + return err + } + input, secrets, err := recipeInput(v, name, m, haveManifest, runtime, body) + if err != nil { + redacted := redactString(err.Error(), secrets) + fmt.Fprintf(log, "FAILED: recipe %s: %s\n", name, redacted) + return fmt.Errorf("recipe %s: %s", name, redacted) + } + redactor := newRedactingWriter(log, secrets) if bootstrap := recipes.BootstrapScript(runtime, v.OS); bootstrap != "" { fmt.Fprintf(log, "ensuring %s is installed...\n", runtime) @@ -337,10 +375,11 @@ func Provision(ctx context.Context, v *config.VM) (err error) { cmd := exec.CommandContext(ctx, "ssh", Args(v, escalate(v, recipes.InterpreterArgs(runtime))...)...) cmd.Cancel = func() error { return cmd.Process.Signal(syscall.SIGTERM) } cmd.WaitDelay = recipeShutdownGrace - cmd.Stdin = strings.NewReader(guest.WithPrelude(body, preludeFor(v, runtime))) - cmd.Stdout = log - cmd.Stderr = log + cmd.Stdin = strings.NewReader(input) + cmd.Stdout = redactor + cmd.Stderr = redactor if err := cmd.Run(); err != nil { + _ = redactor.Flush() // ctx being the cause is reported as CANCELLED, not a plain recipe // FAILED. waitApplied (internal/core/wait.go) treats only a final // "done" line as success, so either wording leaves the recipe @@ -348,13 +387,196 @@ func Provision(ctx context.Context, v *config.VM) (err error) { // caller sniffing Logs' text, must not read a cancellation as if // the recipe itself had failed. if ctxErr := ctx.Err(); ctxErr != nil { - fmt.Fprintf(log, "CANCELLED: recipe %s: %v\n", name, ctxErr) + fmt.Fprintf(log, "CANCELLED: recipe %s: %s\n", name, redactString(ctxErr.Error(), secrets)) return ctxErr } - fmt.Fprintf(log, "FAILED: recipe %s: %v\n", name, err) - return fmt.Errorf("recipe %s: %w", name, err) + _ = redactor.Flush() + redacted := redactString(err.Error(), secrets) + fmt.Fprintf(log, "FAILED: recipe %s: %s\n", name, redacted) + return fmt.Errorf("recipe %s: %s", name, redacted) + } + if err := redactor.Flush(); err != nil { + return err + } + if haveManifest && m.Schema >= 3 { + if err := collectOutputs(ctx, v, name, m, secrets, log); err != nil { + redacted := redactString(err.Error(), secrets) + fmt.Fprintf(log, "FAILED: recipe %s: reading outputs: %s\n", name, redacted) + return fmt.Errorf("recipe %s: reading outputs: %s", name, redacted) + } } } fmt.Fprintln(log, "\ndone") return nil } + +// recipeInput wraps a recipe with its resolved environment and per-recipe +// output file setup. Secret values travel through stdin and never through ssh +// argv. The Python form sets os.environ before the shared prelude runs. +func recipeInput(v *config.VM, name string, m recipes.Manifest, haveManifest bool, runtime, body string) (string, []string, error) { + params := map[string]string{} + secrets := []string{} + if haveManifest && len(m.Params) > 0 { + stored, err := config.LoadSecrets(v.Dir) + if err != nil { + return "", nil, err + } + resolved, err := recipes.Resolve(m, v.Params[name], stored[name]) + if err != nil { + return "", nil, err + } + for param, value := range resolved { + params[param] = value + if m.Params[param].Type == "secret" { + if value != "" { + secrets = append(secrets, value) + } + continue + } + } + } + env := recipes.Env(name, params) + path := OutputDir + "/" + name + prelude := preludeFor(v, runtime) + if runtime == "python3" { + var b strings.Builder + b.WriteString("import os\n") + for _, kv := range env { + key, value, _ := strings.Cut(kv, "=") + fmt.Fprintf(&b, "os.environ[%q] = %q\n", key, value) + } + if !haveManifest || m.Schema < 3 { + b.WriteString(guest.WithPrelude(body, prelude)) + return b.String(), secrets, nil + } + fmt.Fprintf(&b, "os.environ[\"STOAT_OUTPUT\"] = %q\n", path) + fmt.Fprintf(&b, "os.makedirs(%q, mode=0o700, exist_ok=True)\n", OutputDir) + b.WriteString("open(os.environ[\"STOAT_OUTPUT\"], \"w\").close()\n") + b.WriteString(guest.WithPrelude(body, prelude)) + return b.String(), secrets, nil + } + var b strings.Builder + for _, kv := range env { + key, value, _ := strings.Cut(kv, "=") + fmt.Fprintf(&b, "export %s=%s\n", key, shellValue(value)) + } + if !haveManifest || m.Schema < 3 { + b.WriteString(guest.WithPrelude(body, prelude)) + return b.String(), secrets, nil + } + fmt.Fprintf(&b, "export STOAT_OUTPUT=%s\n", shellPath(path)) + fmt.Fprintf(&b, "mkdir -p %s && chmod 700 %s && : > \"$STOAT_OUTPUT\"\n", shellPath(OutputDir), shellPath(OutputDir)) + b.WriteString(guest.WithPrelude(body, prelude)) + return b.String(), secrets, nil +} + +func shellValue(value string) string { + for _, r := range value { + if !(r == '_' || r == '-' || r == '.' || r >= 'a' && r <= 'z' || r >= 'A' && r <= 'Z' || r >= '0' && r <= '9') { + if !strings.ContainsRune(value, '"') { + var b strings.Builder + b.WriteByte('"') + for _, r := range value { + if r == '\\' || r == '$' || r == '`' { + b.WriteByte('\\') + } + b.WriteRune(r) + } + b.WriteByte('"') + return b.String() + } + return guest.ShQuote(value) + } + } + return value +} + +// shellPath leaves simple paths readable for command diagnostics and quotes +// every path containing shell syntax. +func shellPath(path string) string { + for _, r := range path { + if !(r == '/' || r == '.' || r == '-' || r == '_' || r >= 'a' && r <= 'z' || r >= 'A' && r <= 'Z' || r >= '0' && r <= '9') { + return guest.ShQuote(path) + } + } + return path +} + +// redactingWriter keeps a suffix between writes so a secret split across two +// process writes is removed before either part reaches the apply log. +type redactingWriter struct { + dst io.Writer + secrets []string + pending string + keep int +} + +func newRedactingWriter(dst io.Writer, secrets []string) *redactingWriter { + max := 0 + for _, secret := range secrets { + if len(secret) > max { + max = len(secret) + } + } + if max > 0 { + max-- + } + return &redactingWriter{dst: dst, secrets: sortedSecrets(secrets), keep: max} +} + +func (w *redactingWriter) Write(p []byte) (int, error) { + data := w.pending + string(p) + cut := len(data) - w.keep + if cut > 0 { + for _, secret := range w.secrets { + if secret == "" { + continue + } + for start := strings.Index(data, secret); start >= 0; { + end := start + len(secret) + if start < cut && end > cut { + cut = start + } + next := strings.Index(data[start+1:], secret) + if next < 0 { + break + } + start += next + 1 + } + } + } + if cut < 0 { + cut = 0 + } + if _, err := io.WriteString(w.dst, redactString(data[:cut], w.secrets)); err != nil { + return 0, err + } + w.pending = data[cut:] + return len(p), nil +} + +func (w *redactingWriter) Flush() error { + if w.pending == "" { + return nil + } + _, err := io.WriteString(w.dst, redactString(w.pending, w.secrets)) + w.pending = "" + return err +} + +func redactString(value string, secrets []string) string { + for _, secret := range secrets { + if secret != "" { + value = strings.ReplaceAll(value, secret, "") + } + } + return value +} + +// sortedSecrets provides stable redaction replacement order when secrets +// overlap, replacing the longest value first. +func sortedSecrets(secrets []string) []string { + out := append([]string(nil), secrets...) + sort.Slice(out, func(i, j int) bool { return len(out[i]) > len(out[j]) }) + return out +} diff --git a/internal/tomlx/tomlx.go b/internal/tomlx/tomlx.go index 9c4ff8f7..c0fcc443 100644 --- a/internal/tomlx/tomlx.go +++ b/internal/tomlx/tomlx.go @@ -4,11 +4,14 @@ package tomlx import ( + "bytes" "fmt" "io" + "os" "strings" "github.com/BurntSushi/toml" + gotoml "github.com/pelletier/go-toml/v2" ) type options struct { @@ -61,3 +64,99 @@ func Decode(path string, v any, opts ...Option) error { } return nil } + +// Encode is the single TOML writer for files owned by stoat. +func Encode(path string, v any) error { + var buf bytes.Buffer + if err := gotoml.NewEncoder(&buf).Encode(v); err != nil { + return fmt.Errorf("%s: %w", path, err) + } + data := ownedTableComments(normalizeQuotes(buf.Bytes())) + f, err := os.Create(path) + if err != nil { + return fmt.Errorf("%s: %w", path, err) + } + if _, err := f.Write(data); err != nil { + _ = f.Close() + return fmt.Errorf("%s: %w", path, err) + } + if err := f.Close(); err != nil { + return fmt.Errorf("%s: %w", path, err) + } + return nil +} + +// normalizeQuotes keeps files written by stoat compatible with its existing +// examples, which use basic double-quoted strings. go-toml prefers literal +// single-quoted strings when the value does not need escaping. +func normalizeQuotes(data []byte) []byte { + var out bytes.Buffer + for i := 0; i < len(data); { + if data[i] == '"' { + out.WriteByte(data[i]) + i++ + for i < len(data) { + out.WriteByte(data[i]) + if data[i] == '\\' && i+1 < len(data) { + i++ + out.WriteByte(data[i]) + } else if data[i] == '"' { + i++ + break + } + i++ + } + continue + } + if data[i] != '\'' || i+1 >= len(data) || data[i+1] == '\'' { + out.WriteByte(data[i]) + i++ + continue + } + end := i + 1 + for end < len(data) && data[end] != '\'' && data[end] != '\n' { + end++ + } + if end == len(data) || data[end] == '\n' { + out.WriteByte(data[i]) + i++ + continue + } + out.WriteByte('"') + for _, c := range data[i+1 : end] { + if c == '\\' || c == '"' { + out.WriteByte('\\') + } + out.WriteByte(c) + } + out.WriteByte('"') + i = end + 1 + } + return out.Bytes() +} + +// ownedTableComments repeats a struct field's ownership marker for nested +// tables. go-toml emits a field comment once for a map, while vm.toml has one +// owned table for each recipe and each recipe's outputs. +func ownedTableComments(data []byte) []byte { + const marker = "# written by stoat; do not edit" + if !bytes.Contains(data, []byte(marker)) { + return data + } + lines := strings.Split(string(data), "\n") + out := make([]string, 0, len(lines)+4) + for _, line := range lines { + trimmed := strings.TrimSpace(line) + if strings.HasPrefix(trimmed, "[params") || strings.HasPrefix(trimmed, "[applied") { + j := len(out) - 1 + for j >= 0 && strings.TrimSpace(out[j]) == "" { + j-- + } + if j < 0 || !strings.Contains(out[j], marker) { + out = append(out, marker) + } + } + out = append(out, line) + } + return []byte(strings.Join(out, "\n")) +} diff --git a/internal/tomlx/tomlx_test.go b/internal/tomlx/tomlx_test.go index 2b47c352..8c0a5335 100644 --- a/internal/tomlx/tomlx_test.go +++ b/internal/tomlx/tomlx_test.go @@ -82,3 +82,17 @@ func TestDecodeSchemaAbsentIsFine(t *testing.T) { t.Errorf("schema = %d, want 0 (absent)", d.Schema) } } + +func TestEncodeThenDecodeRoundTrips(t *testing.T) { + p := filepath.Join(t.TempDir(), "x.toml") + if err := Encode(p, doc{Schema: 1, Name: "x"}); err != nil { + t.Fatal(err) + } + var got doc + if err := Decode(p, &got, Reject); err != nil { + t.Fatal(err) + } + if got.Name != "x" || got.Schema != 1 { + t.Errorf("got %+v", got) + } +} diff --git a/internal/tui/detail.go b/internal/tui/detail.go index 7ef95600..4fce94bd 100644 --- a/internal/tui/detail.go +++ b/internal/tui/detail.go @@ -441,11 +441,27 @@ func (m model) viewDetail() string { for _, r := range v.Recipes { inConfig[r] = true status := dimStyle.Render("pending") - if a, ok := v.Applied[r]; ok { + if state, ok := recipeStateByName(v.RecipeStates, r); ok { + if state.Applied { + status = upStyle.Render("applied " + state.Health) + } + } else if a, ok := v.Applied[r]; ok { status = upStyle.Render("applied " + a.At.Format("2006-01-02")) } line(label, recipeLabel(r)+" ("+status+")") label = "" + if state, ok := recipeStateByName(v.RecipeStates, r); ok { + for _, name := range sortedKeys(state.Params) { + value := state.Params[name] + if isSecretParam(state, name) && value != core.SecretUnset { + value = core.SecretSet + } + line("", "param "+name+": "+value) + } + for _, name := range sortedKeys(state.Outputs) { + line("", "out "+name+": "+state.Outputs[name]) + } + } } // Stale means applied but since removed from v.Recipes: the user // edited it out of vm.toml. The applied record survives that edit. @@ -488,3 +504,30 @@ func (m model) viewDetail() string { }, m.width, m.showHelp)) return column(appContentWidth, parts...) } + +func recipeStateByName(states []core.RecipeState, name string) (core.RecipeState, bool) { + for _, state := range states { + if state.Name == name { + return state, true + } + } + return core.RecipeState{}, false +} + +func sortedKeys(values map[string]string) []string { + keys := make([]string, 0, len(values)) + for key := range values { + keys = append(keys, key) + } + sort.Strings(keys) + return keys +} + +func isSecretParam(state core.RecipeState, name string) bool { + for _, secret := range state.SecretNames { + if secret == name { + return true + } + } + return false +} diff --git a/internal/tui/detail_test.go b/internal/tui/detail_test.go index b64afb15..c3bf8411 100644 --- a/internal/tui/detail_test.go +++ b/internal/tui/detail_test.go @@ -232,6 +232,37 @@ func TestTypeConsolePasswordKeyRefusesWhenUnavailable(t *testing.T) { } } +// The TUI detail pane is a sink in its own right. It must render stored +// recipe state through the redacted core projection, even when a lower layer +// accidentally hands it a raw value. +func TestDetailRendersRecipeSecretsAsMarkers(t *testing.T) { + const sentinel = "synthetic-secret-sentinel" + m := model{ + screen: screenDetail, + width: 100, + height: 40, + detail: detailModel{vm: core.VM{ + Name: "work", + Mode: "live", + State: core.StateStopped, + Recipes: []string{"redaction"}, + RecipeStates: []core.RecipeState{{ + Name: "redaction", + Applied: true, + Params: map[string]string{"token": sentinel}, + SecretNames: []string{"token"}, + }}, + }}, + } + out := ansi.Strip(m.viewDetail()) + if strings.Contains(out, sentinel) { + t.Fatalf("detail pane leaked secret value: %s", out) + } + if !strings.Contains(out, "") { + t.Fatalf("detail pane lacks redacted secret marker: %s", out) + } +} + // This VM runs with no graphical session (the test sets no Display and no // WAYLAND_DISPLAY/DISPLAY), so qemu.DisplayKind falls back to vnc regardless // of mode. The detail screen must surface the VNC socket as the actual way diff --git a/internal/tui/form.go b/internal/tui/form.go index 895d95ad..89a32ee5 100644 --- a/internal/tui/form.go +++ b/internal/tui/form.go @@ -11,6 +11,7 @@ import ( "charm.land/bubbles/v2/textinput" tea "charm.land/bubbletea/v2" + "charm.land/huh/v2" "github.com/charmbracelet/x/ansi" "github.com/novusedge/stoat/internal/cloudinit" @@ -228,20 +229,25 @@ func byoOSNames() []string { const formContentWidth = appContentWidth type formModel struct { - inputs []textinput.Model // name, ram, cpus, disk, share - focus int - images []imageOption - imgIdx int - byoBackend string // override for the selected BYO image's backend; "" means "use iso.Infer's guess" - byoOS string // override for the selected BYO image's OS; "" means "use iso.Infer's guess" - mode string // "live" | "disk"; meaningful only while the selected image's backend is apkovl - display string // one of displayChoices; "auto" by default - err string - fetching bool - fetchingOS string - recipeNames []string // installed recipes matching the selected image's OS/backend - recipeIdx int // sub-cursor within the recipes row, moved by left/right - recipeSel map[string]bool // names currently checked + inputs []textinput.Model // name, ram, cpus, disk, share + focus int + images []imageOption + imgIdx int + byoBackend string // override for the selected BYO image's backend; "" means "use iso.Infer's guess" + byoOS string // override for the selected BYO image's OS; "" means "use iso.Infer's guess" + mode string // "live" | "disk"; meaningful only while the selected image's backend is apkovl + display string // one of displayChoices; "auto" by default + err string + fetching bool + fetchingOS string + recipeNames []string // installed recipes matching the selected image's OS/backend + recipeIdx int // sub-cursor within the recipes row, moved by left/right + recipeSel map[string]bool // names currently checked + recipeExplicit map[string]bool + paramValues map[string]map[string]string + paramForm *paramForm + paramQueue []*paramForm + paramRoot string // randomPassword swaps the fixed, documented console password for a // generated one. Cloud images only, see build(). randomPassword bool @@ -416,6 +422,9 @@ func (f formModel) effectiveMode() string { func (f *formModel) refreshRecipes() { f.recipeNames, _ = recipes.List(f.resolvedOS(), f.resolvedBackend()) f.recipeSel = map[string]bool{} + f.recipeExplicit = map[string]bool{} + f.paramValues = map[string]map[string]string{} + f.paramForm = nil f.recipeIdx = 0 } @@ -439,7 +448,7 @@ func (f *formModel) selectImage(idx int) { } func newForm() formModel { - f := formModel{mode: "live", display: "auto", recipeSel: map[string]bool{}} + f := formModel{mode: "live", display: "auto", recipeSel: map[string]bool{}, recipeExplicit: map[string]bool{}, paramValues: map[string]map[string]string{}} labels := []string{"work", "4096", "4", "8G", "~/vms"} for i := 0; i < fieldCount; i++ { ti := theme.TextInput() @@ -510,6 +519,53 @@ func fetchImage(ctx context.Context, id string, gen int) tea.Cmd { } func (m model) updateForm(msg tea.Msg) (tea.Model, tea.Cmd) { + if m.form.paramForm != nil { + param := m.form.paramForm + if key, ok := msg.(tea.KeyPressMsg); ok && key.String() == "esc" { + m.form.cancelParamForms() + return m, nil + } + _, cmd := param.form.Update(msg) + if key, ok := msg.(tea.KeyPressMsg); ok { + switch key.String() { + case "tab": + if len(param.form.Errors()) == 0 { + param.form.NextField() + } + cmd = nil + case "shift+tab": + param.form.PrevField() + cmd = nil + case "enter": + if len(param.form.Errors()) == 0 { + if next := param.form.NextField(); next != nil { + if followUp := next(); followUp != nil { + param.form.Update(followUp) + } + } + } + cmd = nil + } + } + switch param.form.State { + case huh.StateCompleted: + if m.form.paramValues == nil { + m.form.paramValues = map[string]map[string]string{} + } + m.form.paramValues[param.recipe] = param.valuesSnapshot() + if len(m.form.paramQueue) > 0 { + m.form.paramForm = m.form.paramQueue[0] + m.form.paramQueue = m.form.paramQueue[1:] + return m, m.form.paramForm.init() + } + m.form.paramForm = nil + m.form.paramRoot = "" + case huh.StateAborted: + m.form.cancelParamForms() + } + return m, cmd + } + switch msg := msg.(type) { case dlTickMsg: // Anchored to m.form.fetching, not dlGen: the tick chain only needs @@ -677,6 +733,10 @@ func (m model) updateForm(msg tea.Msg) (tea.Model, tea.Cmd) { } if m.form.recipeSel[name] { m.form.recipeSel[name] = false + m.form.recipeExplicit[name] = false + delete(m.form.paramValues, name) + m.form.recomputeRecipeSelection() + m.form.cleanupParamValues() return m, nil } // Checking a box can pull in a recipe it depends on. A @@ -689,9 +749,20 @@ func (m model) updateForm(msg tea.Msg) (tea.Model, tea.Cmd) { return m, m.showToast(err.Error(), true) } m.form.recipeSel[name] = true + m.form.recipeExplicit[name] = true for _, a := range added { m.form.recipeSel[a.Recipe] = true } + forms, err := m.form.parameterForms(name, added) + if err != nil { + return m, m.showToast(err.Error(), true) + } + if len(forms) > 0 { + m.form.paramRoot = name + m.form.paramForm = forms[0] + m.form.paramQueue = forms[1:] + return m, tea.Batch(m.form.paramForm.init(), m.showToast(depMessage(added), false)) + } return m, m.showToast(depMessage(added), false) } // space on the image row downloads the selected catalog entry. @@ -794,6 +865,40 @@ func (f formModel) spec() (core.Spec, error) { selected = append(selected, r) } } + params := map[string]map[string]string{} + secrets := config.Secrets{} + for _, name := range selected { + recipe, err := core.RecipeShow(name) + if err != nil { + return core.Spec{}, err + } + values := f.paramValues[name] + if f.paramForm != nil && f.paramForm.recipe == name { + values = f.paramForm.valuesSnapshot() + } + for _, param := range recipe.Params { + value := param.Default + if values != nil { + if given, ok := values[param.Name]; ok { + value = given + } + } + if value == "" || param.Type != "secret" && value == param.Default { + continue + } + if param.Type == "secret" { + if secrets[name] == nil { + secrets[name] = map[string]string{} + } + secrets[name][param.Name] = value + } else { + if params[name] == nil { + params[name] = map[string]string{} + } + params[name][param.Name] = value + } + } + } // The form's numeric fields are free text, so a non-number has to become // something core will reject rather than silently reading as 0, which core // treats as "use the default". @@ -817,6 +922,8 @@ func (f formModel) spec() (core.Spec, error) { Disk: strings.TrimSpace(f.inputs[fDisk].Value()), Share: strings.TrimSpace(f.inputs[fShare].Value()), Recipes: selected, + Params: params, + Secrets: secrets, } if f.display != "auto" { s.Display = f.display @@ -839,6 +946,9 @@ func createVM(s core.Spec) tea.Cmd { } func (m model) viewForm() string { + if m.form.paramForm != nil { + return m.viewParamForm() + } f := m.form b := fields{width: formContentWidth} diff --git a/internal/tui/paramform.go b/internal/tui/paramform.go new file mode 100644 index 00000000..4aaceb78 --- /dev/null +++ b/internal/tui/paramform.go @@ -0,0 +1,211 @@ +package tui + +import ( + "fmt" + "strconv" + "strings" + + tea "charm.land/bubbletea/v2" + "charm.land/huh/v2" + + "github.com/novusedge/stoat/internal/core" + "github.com/novusedge/stoat/internal/recipes" +) + +// paramForm is the schema-driven form shown after a recipe with parameters is +// selected. Pointer-backed values let huh fields and Spec projection share one +// source without reaching into huh's private selector state. +type paramForm struct { + recipe string + form *huh.Form + values map[string]*string + bools map[string]*bool + defaults map[string]string + params []core.RecipeParam +} + +func newParamForm(recipe core.Recipe) *paramForm { + p := ¶mForm{ + recipe: recipe.Name, + values: map[string]*string{}, + bools: map[string]*bool{}, + defaults: map[string]string{}, + params: append([]core.RecipeParam{}, recipe.Params...), + } + fields := make([]huh.Field, 0, len(recipe.Params)) + for _, param := range recipe.Params { + p.defaults[param.Name] = param.Default + switch param.Type { + case "bool": + value := new(bool) + *value = param.Default == "true" + p.bools[param.Name] = value + fields = append(fields, huh.NewConfirm().Title(param.Name).Description(param.Help).Value(value).Affirmative("true").Negative("false")) + case "enum": + value := new(string) + *value = param.Default + p.values[param.Name] = value + fields = append(fields, huh.NewSelect[string]().Title(param.Name).Description(param.Help).Options(huh.NewOptions(param.Values...)...).Value(value)) + default: + value := new(string) + *value = param.Default + p.values[param.Name] = value + input := huh.NewInput().Title(param.Name).Description(param.Help).Value(value).Validate(paramValidator(param)) + if param.Type == "secret" { + input.EchoMode(huh.EchoModePassword) + } + fields = append(fields, input) + } + } + p.form = huh.NewForm(huh.NewGroup(fields...)).WithAccessible(false).WithWidth(formContentWidth - 2) + return p +} + +func paramValidator(param core.RecipeParam) func(string) error { + return func(value string) error { + if strings.TrimSpace(value) == "" { + if param.Required { + return fmt.Errorf("%s is required", param.Name) + } + return nil + } + switch param.Type { + case "int": + if _, err := strconv.Atoi(value); err != nil { + return fmt.Errorf("%s must be an integer", param.Name) + } + case "bool": + if value != "true" && value != "false" { + return fmt.Errorf("%s must be true or false", param.Name) + } + case "enum": + for _, choice := range param.Values { + if choice == value { + return nil + } + } + return fmt.Errorf("%s must be one of %s", param.Name, strings.Join(param.Values, ", ")) + } + return nil + } +} + +func (p *paramForm) init() tea.Cmd { + return p.form.Init() +} + +func (p *paramForm) valuesSnapshot() map[string]string { + out := make(map[string]string, len(p.params)) + for _, param := range p.params { + if param.Type == "bool" { + out[param.Name] = strconv.FormatBool(*p.bools[param.Name]) + } else { + out[param.Name] = *p.values[param.Name] + } + } + return out +} + +// parameterForms queues dependency forms before the explicitly selected +// recipe. Dependencies are real selections, so their required inputs must be +// completed through the same wizard rather than being left for core.Plan to +// reject after the user submits the VM form. +func (f *formModel) parameterForms(root string, added []core.DepAddition) ([]*paramForm, error) { + names := make([]string, 0, len(added)+1) + seen := make(map[string]bool, len(added)+1) + for _, addition := range added { + if !seen[addition.Recipe] { + names = append(names, addition.Recipe) + seen[addition.Recipe] = true + } + } + if !seen[root] { + names = append(names, root) + } + forms := make([]*paramForm, 0, len(names)) + for _, name := range names { + recipe, err := core.RecipeShow(name) + if err != nil { + return nil, err + } + if manifest, ok, err := recipes.ManifestFor(name); err != nil { + return nil, err + } else if ok { + byName := make(map[string]core.RecipeParam, len(recipe.Params)) + for _, param := range recipe.Params { + byName[param.Name] = param + } + ordered := make([]core.RecipeParam, 0, len(recipe.Params)) + for _, param := range manifest.OrderedParams() { + if projected, exists := byName[param.Name]; exists { + ordered = append(ordered, projected) + delete(byName, param.Name) + } + } + for _, param := range recipe.Params { + if _, exists := byName[param.Name]; exists { + ordered = append(ordered, param) + delete(byName, param.Name) + } + } + recipe.Params = ordered + } + if len(recipe.Params) > 0 { + forms = append(forms, newParamForm(recipe)) + } + } + return forms, nil +} + +func (f *formModel) cancelParamForms() { + root := f.paramRoot + f.paramForm = nil + f.paramQueue = nil + f.paramRoot = "" + if root != "" { + f.recipeExplicit[root] = false + delete(f.paramValues, root) + } + f.recomputeRecipeSelection() + f.cleanupParamValues() +} + +func (f *formModel) cleanupParamValues() { + for name := range f.paramValues { + if !f.recipeSel[name] { + delete(f.paramValues, name) + } + } +} + +func (f *formModel) recomputeRecipeSelection() { + if f.recipeSel == nil { + f.recipeSel = map[string]bool{} + } + for name := range f.recipeSel { + f.recipeSel[name] = false + } + roots := make([]string, 0) + for _, name := range f.recipeNames { + if f.recipeExplicit[name] { + roots = append(roots, name) + f.recipeSel[name] = true + } + } + added, err := resolveDeps(f.resolvedOS(), roots) + if err != nil { + return + } + for _, dep := range added { + f.recipeSel[dep.Recipe] = true + } + f.cleanupParamValues() +} + +func (m model) viewParamForm() string { + body := m.form.paramForm.form.View() + box := paneAt("recipe params", body, formContentWidth, m.width) + parts := []string{box, "", warnStyle.Render(m.status)} + parts = append(parts, renderFooter(formHelp{}, m.width, m.showHelp)) + return column(appContentWidth, parts...) +} diff --git a/internal/tui/paramform_test.go b/internal/tui/paramform_test.go new file mode 100644 index 00000000..92bb2661 --- /dev/null +++ b/internal/tui/paramform_test.go @@ -0,0 +1,421 @@ +package tui + +import ( + "os" + "path/filepath" + "strings" + "testing" + + "github.com/charmbracelet/x/ansi" + + "github.com/novusedge/stoat/internal/config" +) + +func writeParamRecipe(t *testing.T) { + t.Helper() + baseDir := filepath.Join(config.Root(), "recipes", "param-base") + if err := os.MkdirAll(baseDir, 0o755); err != nil { + t.Fatal(err) + } + baseManifest := `schema = 2 +name = "param-base" +description = "parameter dependency" +os = ["alpine"] +script = "install.sh" +` + if err := os.WriteFile(filepath.Join(baseDir, "recipe.toml"), []byte(baseManifest), 0o644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(baseDir, "install.sh"), []byte("#!/bin/sh\n"), 0o755); err != nil { + t.Fatal(err) + } + + dir := filepath.Join(config.Root(), "recipes", "param-docker") + if err := os.MkdirAll(dir, 0o755); err != nil { + t.Fatal(err) + } + manifest := `schema = 3 +name = "param-docker" +description = "parameterized recipe" +os = ["alpine"] +script = "install.sh" +depends = ["param-base"] + +[params.authkey] +type = "secret" +required = true +help = "tailnet auth key" + +[params.channel] +type = "enum" +values = ["stable", "test"] +default = "stable" + +[params.port] +type = "int" +default = 2375 + +[params.tls] +type = "bool" +default = true + +[params.user] +type = "string" +required = true +` + if err := os.WriteFile(filepath.Join(dir, "recipe.toml"), []byte(manifest), 0o644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(dir, "install.sh"), []byte("#!/bin/sh\n"), 0o755); err != nil { + t.Fatal(err) + } +} + +func parameterizedForm(t *testing.T) model { + t.Helper() + t.Setenv("STOAT_HOME", t.TempDir()) + writeParamRecipe(t) + f := newForm() + f.images = []imageOption{stubImage(t, "alpine-standard-3.20.0-x86_64.iso")} + f.imgIdx = 0 + f.refreshRecipes() + for i, name := range f.recipeNames { + if name == "param-docker" { + f.recipeIdx = i + break + } + } + f.focus = fRecipes + f.inputs[fName].SetValue("param-vm") + return model{screen: screenForm, width: 100, height: 40, form: f} +} + +func sendParamKeys(m model, keys ...string) model { + for _, key := range keys { + out, _ := m.Update(keyMsg(key)) + m = out.(model) + } + return m +} + +func typeParamText(m model, text string) model { + for _, r := range text { + m = sendParamKeys(m, string(r)) + } + return m +} + +// Selecting a parameterized recipe enters the existing wizard's form +// lifecycle. The rendered boundary must show every declared field while a +// secret remains masked and absent from the screen. +func TestParameterizedRecipeSelectionOpensMaskedForm(t *testing.T) { + m := parameterizedForm(t) + out, _ := m.Update(keyMsg(keySpace)) + m = out.(model) + rendered := ansi.Strip(m.View().Content) + for _, field := range []string{"authkey", "channel", "port", "tls", "user"} { + if !strings.Contains(rendered, field) { + t.Errorf("parameter form omitted %q:\n%s", field, rendered) + } + } + for _, defaultValue := range []string{"stable", "2375", "true"} { + if !strings.Contains(rendered, defaultValue) { + t.Errorf("parameter form omitted manifest default %q:\n%s", defaultValue, rendered) + } + } + if strings.Contains(rendered, "dev") { + t.Errorf("parameter form invented a user default not declared by the manifest:\n%s", rendered) + } + const sentinel = "synthetic-secret-sentinel" + m = typeParamText(m, sentinel) + spec, err := m.form.spec() + if err != nil { + t.Fatalf("form spec after typing secret: %v", err) + } + if got := spec.Secrets["param-docker"]["authkey"]; got != sentinel { + t.Fatalf("wizard did not carry typed secret through spec: got %q, want %q", got, sentinel) + } + rendered = ansi.Strip(m.View().Content) + if strings.Contains(rendered, sentinel) { + t.Fatal("parameter form rendered the typed secret value") + } +} + +// Confirming with an empty required secret must keep the wizard in the +// parameter form and expose validation feedback; it must not create a VM with +// an absent required credential. +func TestParameterizedRecipeRequiredValueBlocksConfirm(t *testing.T) { + m := parameterizedForm(t) + out, _ := m.Update(keyMsg(keySpace)) + m = out.(model) + out, _ = m.Update(keyMsg("enter")) + m = out.(model) + if m.screen != screenForm { + t.Fatalf("required validation left screen %v", m.screen) + } + rendered := ansi.Strip(m.View().Content) + if !strings.Contains(strings.ToLower(rendered), "required") { + t.Fatalf("required validation feedback missing:\n%s", rendered) + } +} + +// Typed fields validate at the wizard boundary: a non-numeric port cannot be +// confirmed even though the surrounding VM form itself accepts free text. +func TestParameterizedRecipeIntValidationBlocksConfirm(t *testing.T) { + m := parameterizedForm(t) + m = sendParamKeys(m, keySpace) + m = typeParamText(m, "tskey-secret") + // authkey -> channel -> port + m = sendParamKeys(m, "tab", "tab") + m = typeParamText(m, "not-an-int") + m = sendParamKeys(m, "enter") + if m.screen != screenForm { + t.Fatalf("invalid integer left screen %v", m.screen) + } + rendered := strings.ToLower(ansi.Strip(m.View().Content)) + if !strings.Contains(rendered, "integer") && !strings.Contains(rendered, "number") { + t.Fatalf("integer validation feedback missing:\n%s", rendered) + } +} + +func TestParameterizedRecipeEnumOffersOnlyDeclaredChoices(t *testing.T) { + m := parameterizedForm(t) + m = sendParamKeys(m, keySpace) + rendered := ansi.Strip(m.View().Content) + if !strings.Contains(rendered, "stable") || !strings.Contains(rendered, "test") { + t.Fatalf("enum choices are not rendered by the wizard:\n%s", rendered) + } +} + +// After a normal wizard submission, non-secret edits belong in Spec.Params, +// secret edits belong in Spec.Secrets, and untouched defaults are omitted so +// future manifest changes can still take effect. +func TestParameterizedRecipeBuildSplitsSecretsAndOmitsDefaults(t *testing.T) { + m := parameterizedForm(t) + m = sendParamKeys(m, keySpace) + m = typeParamText(m, "tskey-secret") + // authkey -> channel -> port -> tls -> user + m = sendParamKeys(m, "tab", "tab", "tab", "tab") + m = typeParamText(m, "alice") + m = sendParamKeys(m, "enter") + spec, err := m.form.spec() + if err != nil { + t.Fatalf("form spec after parameter submission: %v", err) + } + if got := spec.Secrets["param-docker"]["authkey"]; got != "tskey-secret" { + t.Errorf("secret authkey = %q, want secret storage", got) + } + if got := spec.Params["param-docker"]["user"]; got != "alice" { + t.Errorf("non-secret user = %q, want Params storage", got) + } + for _, name := range []string{"channel", "port", "tls"} { + if _, ok := spec.Params["param-docker"][name]; ok { + t.Errorf("unchanged default %q was stored in Params", name) + } + } +} + +// A completed parameter form returns to the recipe picker with its values +// attached to the selected recipe. Deselecting that recipe must remove both +// its parameter values and dependencies that are no longer needed. +func TestParameterizedRecipeSubmitAndDeselectCleansSelection(t *testing.T) { + m := parameterizedForm(t) + m = sendParamKeys(m, keySpace) + m = typeParamText(m, "submitted-secret") + // authkey -> channel -> port -> tls -> user + m = sendParamKeys(m, "tab", "tab", "tab", "tab") + m = typeParamText(m, "submitted-user") + m = sendParamKeys(m, "enter") + + spec, err := m.form.spec() + if err != nil { + t.Fatalf("form spec after parameter submission: %v", err) + } + if !containsString(spec.Recipes, "param-docker") || !containsString(spec.Recipes, "param-base") { + t.Fatalf("submitted spec = %+v, want recipe and dependency selected", spec.Recipes) + } + if got := spec.Secrets["param-docker"]["authkey"]; got != "submitted-secret" { + t.Fatalf("submitted secret = %q, want typed value", got) + } + if got := spec.Params["param-docker"]["user"]; got != "submitted-user" { + t.Fatalf("submitted user = %q, want typed value", got) + } + + m = sendParamKeys(m, keySpace) + cleaned, err := m.form.spec() + if err != nil { + t.Fatalf("form spec after recipe deselection: %v", err) + } + if containsString(cleaned.Recipes, "param-docker") || containsString(cleaned.Recipes, "param-base") { + t.Fatalf("deselected spec = %+v, retained recipe or dependency", cleaned.Recipes) + } + if _, ok := cleaned.Secrets["param-docker"]; ok { + t.Errorf("deselected spec retained recipe secrets: %#v", cleaned.Secrets) + } + if _, ok := cleaned.Params["param-docker"]; ok { + t.Errorf("deselected spec retained recipe params: %#v", cleaned.Params) + } +} + +func containsString(values []string, want string) bool { + for _, value := range values { + if value == want { + return true + } + } + return false +} + +// Escaping the parameter form returns to the recipe picker and discards the +// transient selection. A later deselection must not retain stale parameters. +func TestParameterizedRecipeCancelCleansSelection(t *testing.T) { + m := parameterizedForm(t) + out, _ := m.Update(keyMsg(keySpace)) + m = out.(model) + out, _ = m.Update(keyMsg("esc")) + m = out.(model) + if m.screen != screenForm { + t.Fatalf("cancel left screen %v, want recipe picker", m.screen) + } + if m.form.recipeSel["param-docker"] { + t.Fatal("cancel retained a recipe selection") + } + if strings.Contains(ansi.Strip(m.View().Content), "authkey") { + t.Fatal("cancel left parameter fields visible") + } +} + +// The existing view lifecycle owns narrow-terminal behavior too: opening a +// parameter form must not bypass the established minimum-size message. +func TestParameterizedRecipeNarrowTerminalUsesExistingFloor(t *testing.T) { + m := parameterizedForm(t) + out, _ := m.Update(keyMsg(keySpace)) + m = out.(model) + m.width, m.height = 59, 20 + if got := ansi.Strip(m.View().Content); !strings.Contains(got, "terminal too small") { + t.Fatalf("narrow parameter form did not use terminal floor:\n%s", got) + } +} + +func writeDependencyParamRecipes(t *testing.T) { + t.Helper() + baseDir := filepath.Join(config.Root(), "recipes", "dependency-base") + if err := os.MkdirAll(baseDir, 0o755); err != nil { + t.Fatal(err) + } + baseManifest := `schema = 3 +name = "dependency-base" +description = "required dependency parameters" +os = ["alpine"] +script = "install.sh" + +[params.channel] +type = "enum" +values = ["stable", "canary"] +default = "stable" + +[params.token] +type = "secret" +required = true + +[params.owner] +type = "string" +required = true +` + if err := os.WriteFile(filepath.Join(baseDir, "recipe.toml"), []byte(baseManifest), 0o644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(baseDir, "install.sh"), []byte("#!/bin/sh\n"), 0o755); err != nil { + t.Fatal(err) + } + + parentDir := filepath.Join(config.Root(), "recipes", "dependency-parent") + if err := os.MkdirAll(parentDir, 0o755); err != nil { + t.Fatal(err) + } + parentManifest := `schema = 2 +name = "dependency-parent" +description = "parameterized dependency parent" +os = ["alpine"] +script = "install.sh" +depends = ["dependency-base"] +` + if err := os.WriteFile(filepath.Join(parentDir, "recipe.toml"), []byte(parentManifest), 0o644); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(parentDir, "install.sh"), []byte("#!/bin/sh\n"), 0o755); err != nil { + t.Fatal(err) + } +} + +func dependencyForm(t *testing.T) model { + t.Helper() + t.Setenv("STOAT_HOME", t.TempDir()) + writeDependencyParamRecipes(t) + f := newForm() + f.images = []imageOption{stubImage(t, "alpine-standard-3.20.0-x86_64.iso")} + f.imgIdx = 0 + f.refreshRecipes() + for i, name := range f.recipeNames { + if name == "dependency-parent" { + f.recipeIdx = i + break + } + } + f.focus = fRecipes + f.inputs[fName].SetValue("dependency-vm") + return model{screen: screenForm, width: 100, height: 40, form: f} +} + +// A selected parent must make a required secret and non-secret field from its +// dependency usable through the same wizard boundary. The enum is changed +// from its manifest default so the key path, not only the rendered options, +// is exercised; submitting then records values on the dependency and parent +// deselection removes both selections and their values. +func TestParameterizedDependencyFieldsSubmitAndDeselectThroughWizard(t *testing.T) { + m := dependencyForm(t) + m = sendParamKeys(m, keySpace) + rendered := ansi.Strip(m.View().Content) + for _, field := range []string{"channel", "token", "owner"} { + if !strings.Contains(rendered, field) { + t.Fatalf("dependency form omitted %q:\n%s", field, rendered) + } + } + m = sendParamKeys(m, "down", "tab") + m = typeParamText(m, "dependency-secret") + m = sendParamKeys(m, "tab") + m = typeParamText(m, "alice") + m = sendParamKeys(m, "enter") + + spec, err := m.form.spec() + if err != nil { + t.Fatalf("dependency wizard spec = %v", err) + } + if !containsString(spec.Recipes, "dependency-parent") || !containsString(spec.Recipes, "dependency-base") { + t.Fatalf("dependency recipes = %v, want parent and dependency", spec.Recipes) + } + if got := spec.Secrets["dependency-base"]["token"]; got != "dependency-secret" { + t.Fatalf("dependency secret = %q, want typed value", got) + } + if got := spec.Params["dependency-base"]["owner"]; got != "alice" { + t.Fatalf("dependency owner = %q, want typed value", got) + } + if got := spec.Params["dependency-base"]["channel"]; got != "canary" { + t.Fatalf("dependency enum = %q, want non-default key selection", got) + } + + m = sendParamKeys(m, keySpace) + cleaned, err := m.form.spec() + if err != nil { + t.Fatalf("dependency spec after deselection = %v", err) + } + if containsString(cleaned.Recipes, "dependency-parent") || containsString(cleaned.Recipes, "dependency-base") { + t.Fatalf("deselected dependency recipes = %v", cleaned.Recipes) + } + if _, ok := cleaned.Secrets["dependency-base"]; ok { + t.Fatalf("deselected dependency retained secret: %#v", cleaned.Secrets) + } + if _, ok := cleaned.Params["dependency-base"]; ok { + t.Fatalf("deselected dependency retained params: %#v", cleaned.Params) + } +} diff --git a/scripts/e2e.sh b/scripts/e2e.sh index d5a19710..e4d57f87 100755 --- a/scripts/e2e.sh +++ b/scripts/e2e.sh @@ -7,9 +7,10 @@ # code that has regressed before, so this asserts the OUTCOME a user sees: # udev is the device manager and Xorg drives input through libinput. # -# Runs against the real data root by default, under a unique VM name it deletes -# on exit. Set STOAT_HOME to isolate it from your VMs. Needs KVM and network; -# the xfce apk pull is ~1.4GB, so budget ~15 minutes. +# Runs against a temporary data root by default, under a unique VM name it +# deletes on exit. STOAT_HOME selects the isolated data root; failure evidence +# is retained separately via STOAT_E2E_EVIDENCE_DIR. +# Needs KVM and network; the xfce apk pull is ~1.4GB, so budget ~15 minutes. set -eu IMAGE=alpine-standard @@ -17,6 +18,9 @@ RECIPE=xfce VM="e2e-$$" UP_TIMEOUT=1200 # install + xfce pull + reboot-once X_TIMEOUT=90 # Xorg restart after the reboot-once +E2E_RAM=${STOAT_E2E_RAM:-2048} +E2E_CPUS=${STOAT_E2E_CPUS:-2} +E2E_DISPLAY=${STOAT_E2E_DISPLAY:-vnc} # Prefer the just-built binary over whatever is on PATH. root=$(CDPATH='' cd -- "$(dirname -- "$0")/.." && pwd) @@ -26,12 +30,69 @@ STOAT="$root/stoat" say() { printf '\n=== %s ===\n' "$*"; } fail() { printf '\nFAIL: %s\n' "$*" >&2; exit 1; } +capture_failure_evidence() { + evidence=${STOAT_E2E_EVIDENCE_DIR:-${TMPDIR:-/tmp}/stoat-e2e-evidence-$VM} + if ! mkdir -p "$evidence"; then + printf 'could not create failure evidence directory: %s\n' "$evidence" >&2 + return 1 + fi + if ! "$STOAT" screenshot "$VM" -o "$evidence/screenshot.png" >"$evidence/screenshot.txt" 2>&1; then + printf 'screenshot unavailable; see %s/screenshot.txt\n' "$evidence" >&2 + fi + if ! "$STOAT" logs "$VM" --which console >"$evidence/console.log" 2>&1; then + printf 'console log capture failed; see %s/console.log\n' "$evidence" >&2 + fi + if ! "$STOAT" logs "$VM" --which apply >"$evidence/provision.log" 2>&1; then + printf 'provision log capture failed; see %s/provision.log\n' "$evidence" >&2 + fi + if [ -f "$STOAT_HOME/$VM/e2e-apply.log" ]; then + cp "$STOAT_HOME/$VM/e2e-apply.log" "$evidence/apply-output.log" \ + || printf 'apply output copy failed; see %s/%s/e2e-apply.log\n' "$STOAT_HOME" "$VM" >&2 + fi + printf 'failure evidence retained at %s\n' "$evidence" >&2 +} + cleanup() { - "$STOAT" down "$VM" >/dev/null 2>&1 || true - "$STOAT" rm "$VM" -y >/dev/null 2>&1 || true + status=$? + if [ -z "${STOAT_HOME:-}" ] || [ ! -f "$STOAT_HOME/$VM/vm.toml" ]; then + exit "$status" + fi + trap - EXIT INT TERM + if [ "$status" -ne 0 ]; then + capture_failure_evidence || true + fi + cleanup_status=0 + if [ -f "$STOAT_HOME/$VM/qemu.pid" ] && ! "$STOAT" down "$VM"; then + printf 'cleanup failed: %s is still running; preserving %s for evidence\n' "$VM" "$STOAT_HOME/$VM" >&2 + cleanup_status=1 + fi + if [ "$cleanup_status" -eq 0 ] && ! "$STOAT" rm "$VM" -y; then + printf 'cleanup failed: could not delete %s; preserving %s for evidence\n' "$VM" "$STOAT_HOME/$VM" >&2 + cleanup_status=1 + fi + if [ "$cleanup_status" -ne 0 ] && [ "$status" -eq 0 ]; then + status=$cleanup_status + fi + exit "$status" } trap cleanup EXIT INT TERM +if [ -z "${STOAT_HOME:-}" ]; then + STOAT_HOME=$(mktemp -d "${TMPDIR:-/tmp}/stoat-e2e.XXXXXX") +fi +export STOAT_HOME + +case "$E2E_RAM" in + ''|*[!0-9]*) fail "STOAT_E2E_RAM must be an integer no greater than 2048";; +esac +case "$E2E_CPUS" in + ''|*[!0-9]*) fail "STOAT_E2E_CPUS must be an integer no greater than 2";; +esac +[ "$E2E_RAM" -ge 256 ] && [ "$E2E_RAM" -le 2048 ] || fail "STOAT_E2E_RAM must be between 256 and 2048 MiB" +[ "$E2E_CPUS" -ge 1 ] && [ "$E2E_CPUS" -le 2 ] || fail "STOAT_E2E_CPUS must be between 1 and 2" +[ "$E2E_DISPLAY" = vnc ] || fail "STOAT_E2E_DISPLAY must be vnc" +export STOAT_GRAPHICAL=0 + say "build" ( cd "$root" && go build -o stoat ./cmd/stoat ) @@ -39,11 +100,21 @@ say "image $IMAGE" "$STOAT" images 2>/dev/null | grep -q "^$IMAGE .*downloaded" || "$STOAT" pull "$IMAGE" say "create $VM (alpine disk, recipe $RECIPE)" -"$STOAT" create "$VM" --image "$IMAGE" --mode disk --recipes "$RECIPE" +"$STOAT" create "$VM" --image "$IMAGE" --mode disk --ram "$E2E_RAM" --cpus "$E2E_CPUS" --recipes "$RECIPE" say "up $VM (install -> restart -> apply -> reboot-once)" timeout "$UP_TIMEOUT" "$STOAT" up "$VM" || fail "up did not finish in ${UP_TIMEOUT}s" +say "assert: disk installer completed and wrote an ext4 root" +"$STOAT" exec "$VM" -- sh -c 'test -s /mnt/work/.installed' \ + || fail "disk installer completion marker is missing" + +grep -Fq 'stoat: install complete, powering off' "$STOAT_HOME/$VM/console.log" \ + || fail "console.log has no install completion message" + +"$STOAT" exec "$VM" -- sh -c "[ \"\$(awk '\$2 == \"/\" { print \$3 }' /proc/mounts)\" = ext4 ]" \ + || fail "installed guest root is not ext4" + say "assert: recipe applied (xfce present)" "$STOAT" exec "$VM" -- sh -c 'command -v xfce4-session' \ || fail "xfce4-session missing: the recipe did not apply" @@ -57,13 +128,69 @@ say "assert: udev is the device manager (not mdev)" say "assert: Xorg drives input through libinput (mouse clickable)" # X restarts on the reboot-once; poll until its log records a libinput device. end=0 -while [ "$end" -lt "$X_TIMEOUT" ]; do +found=0 +while [ "$found" -eq 0 ] && [ "$end" -lt "$X_TIMEOUT" ]; do if "$STOAT" exec "$VM" -- sh -c \ "grep -q \"Using input driver 'libinput'\" /var/log/Xorg.0.log 2>/dev/null"; then + found=1 printf '\nPASS: %s reached a clickable xfce desktop with no manual steps\n' "$VM" - exit 0 + break fi end=$((end + 5)) sleep 5 done -fail "no libinput device in Xorg.0.log after ${X_TIMEOUT}s" +[ "$found" -eq 1 ] || fail "no libinput device in Xorg.0.log after ${X_TIMEOUT}s" + +E2E_SECRET="stoat-e2e-secret-$$" +export STOAT_SECRET_REDACTION_TOKEN=$E2E_SECRET +redaction_src="$root/scripts/testdata/e2e-redaction" +redaction_dst="$STOAT_HOME/recipes/redaction" +if [ -e "$redaction_dst" ] || [ -L "$redaction_dst" ]; then + fail "refusing to overwrite existing recipe: $redaction_dst" +fi +mkdir -p "$redaction_dst" +cp "$redaction_src/recipe.toml" "$redaction_src/install.sh" "$redaction_dst/" +chmod 755 "$redaction_dst/install.sh" + +say "assert: docker recipe contract" +"$STOAT" update "$VM" --recipes xfce,docker,redaction --set docker.user=dev --secret redaction.token +apply_output="$STOAT_HOME/$VM/e2e-apply.log" +if ! "$STOAT" apply "$VM" >"$apply_output" 2>&1; then + fail "recipe apply failed; see $apply_output" +fi +grep -Fq '' "$apply_output" \ + || fail "apply output omitted the redaction marker" +if grep -Fq "$E2E_SECRET" "$apply_output"; then + fail "secret sentinel reached apply output" +fi + +console_logs=$("$STOAT" logs "$VM" --which console 2>&1) \ + || fail "console log reader failed" +apply_logs=$("$STOAT" logs "$VM" --which apply 2>&1) \ + || fail "apply log reader failed" +if printf '%s\n%s\n' "$console_logs" "$apply_logs" | grep -Fq "$E2E_SECRET"; then + fail "secret sentinel reached CLI log readers" +fi +printf '%s\n' "$apply_logs" | grep -Fq '' \ + || fail "apply log reader omitted the redaction marker" +"$STOAT" wait "$VM" --healthy --timeout 90s + +status=$( + "$STOAT" get "$VM" --json +) +printf '%s\n' "$status" | grep -q '"socket":"/var/run/docker.sock"' \ + || fail "docker did not report its socket output" +printf '%s\n' "$status" | grep -q '"socket":"/var/run/redaction.sock"' \ + || fail "redaction fixture did not report its output" +printf '%s\n' "$status" | grep -q '"health":"ok"' \ + || fail "healthy wait did not record an ok status" +printf '%s\n' "$status" | grep -q '"token":""' \ + || fail "redaction recipe secret was not represented as " +if printf '%s\n' "$status" | grep -Fq "$E2E_SECRET"; then + fail "secret sentinel reached stoat get output" +fi + +say "assert: non-secret param reruns" +"$STOAT" update "$VM" --set docker.user=e2e-rerun +"$STOAT" apply "$VM" --dry-run | grep -q 'params changed' \ + || fail "docker param change did not trigger a params-changed rerun" diff --git a/scripts/testdata/e2e-redaction/install.sh b/scripts/testdata/e2e-redaction/install.sh new file mode 100644 index 00000000..cb2d74f4 --- /dev/null +++ b/scripts/testdata/e2e-redaction/install.sh @@ -0,0 +1,6 @@ +#!/bin/sh +set -eu + +test -n "${STOAT_PARAM_TOKEN:?missing redaction token}" +printf 'redaction-secret=%s\n' "$STOAT_PARAM_TOKEN" +printf '%s\n' 'socket=/var/run/redaction.sock' >> "$STOAT_OUTPUT" diff --git a/scripts/testdata/e2e-redaction/recipe.toml b/scripts/testdata/e2e-redaction/recipe.toml new file mode 100644 index 00000000..4670c6d9 --- /dev/null +++ b/scripts/testdata/e2e-redaction/recipe.toml @@ -0,0 +1,18 @@ +schema = 3 +name = "redaction" +description = "e2e secret redaction probe" +os = ["alpine"] +script = "install.sh" +run = "once" + +[params.token] +type = "secret" +required = true +help = "synthetic e2e sentinel" + +[outputs] +socket = "synthetic output" + +[health] +check = "true" +timeout = "5s"