diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index d4d65e73..0a66fee8 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -86,11 +86,11 @@ reviewer rejects a PR that ignores them. Bundled recipes are `xfce`, `docker`, `devtools`, `tailscale`, and the set is closed. Write a new recipe in `~/.stoat/recipes//` from `stoat -recipe new`; see `docs/recipes/writing-your-own.md`. A recipe index for -sharing them is planned and not built yet. Every recipe script -starts with `set -e` and carries the live-vs-disk block that `stoat recipe -new` scaffolds; `internal/recipes/recipes_test.go` checks both for the -bundled set. +recipe new`; see `docs/recipes/writing-your-own.md` and +`docs/recipes/sharing.md` for installing and pinning remote recipes. Every +recipe script starts with `set -e` and carries the live-vs-disk block that +`stoat recipe new` scaffolds; `internal/recipes/recipes_test.go` checks both +for the bundled set. ## Reporting a security issue diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md index 9a261643..5f849512 100644 --- a/docs/SUMMARY.md +++ b/docs/SUMMARY.md @@ -28,6 +28,7 @@ * [Overview](recipes/overview.md) * [Writing your own](recipes/writing-your-own.md) +* [Sharing recipes](recipes/sharing.md) ## Troubleshooting diff --git a/docs/design/mcp-server.md b/docs/design/mcp-server.md index 8074d5f3..458253c9 100644 --- a/docs/design/mcp-server.md +++ b/docs/design/mcp-server.md @@ -114,6 +114,7 @@ is enforced regardless of what the client does. | `check_image_id` | Catalog IDs only. An absolute or relative path is rejected: §7.1 #4. | | `rate_limit` | A token bucket per tool, and a second one shared by every tool. Per-tool alone let a caller burst `capacity` times across each of ~20 tools. The MCP spec makes rate limiting a server `MUST`. | | `check_flag_free` | Values splatted into argv as positionals (`forward` pairs, `check_recipes` names) must not start with `-`. `forward(pairs=["--clear"])` otherwise reached kong as the clear flag. | +| `check_index_name` | `add_recipe` takes an index name with an optional `@ref`; a URL, path separator in the plain name, dot segment, leading dash, or malformed ref is refused before it reaches the CLI. A valid slash-containing branch ref after `@` is allowed. `update_recipe` and `remove_recipe` take plain names. | **Never exposed as tools at all** (§7.1): `share` as any parameter, BYO image paths, `recipe new`, `ssh-command`, and the global (no VM) `logs`. These are @@ -129,9 +130,9 @@ may ignore them. | Class | Tools | `readOnlyHint` | `destructiveHint` | |---|---|---|---| -| Read-only | `list_vms`, `vm_status`, `list_images`, `list_recipes`, `check_recipes`, `logs`, `doctor`, `plan_recipes` | true | false | -| Mutating | `create`, `start`, `stop`, `apply_recipes`, `update`, `clone`, `snapshot`, `forward`, `wait` | false | false | -| Destructive | `destroy`, `prune`, `restore` | false | true | +| Read-only | `list_vms`, `vm_status`, `list_images`, `list_recipes`, `check_recipes`, `logs`, `search_recipes`, `doctor`, `plan_recipes` | true | false | +| Mutating | `create`, `start`, `stop`, `apply_recipes`, `update`, `add_recipe`, `update_recipe`, `clone`, `snapshot`, `forward`, `wait` | false | false | +| Destructive | `destroy`, `prune`, `remove_recipe`, `restore` | false | true | | Execution | `exec`, `copy_to`, `copy_from` | false | true, `openWorldHint` true | `plan_recipes` is `apply --dry-run`. It exists so an agent can read what an @@ -146,6 +147,13 @@ and only another snapshot taken later undoes that. `allow_exec` exactly as `exec` does. A VM created with `--allow-exec=false` refuses both. +`search_recipes` and `add_recipe` use the curated index. `add_recipe` accepts an +index name and an optional tag or branch ref, including slash-containing branch +refs. It never accepts a repository URL. `update_recipe` and `remove_recipe` +address an existing remote pin by plain name, including one originally added +from a URL. `remove_recipe` has no `force` argument and therefore refuses while +a VM still lists the recipe. + Every schema sets `additionalProperties: false`, so an unexpected parameter is rejected rather than silently ignored (OWASP MCP guidance). diff --git a/docs/recipes/sharing.md b/docs/recipes/sharing.md new file mode 100644 index 00000000..ffa0f684 --- /dev/null +++ b/docs/recipes/sharing.md @@ -0,0 +1,120 @@ +# Sharing recipes + +Remote recipes come from a Git repository with a `recipe.toml` at its root. +Stoat validates the manifest, pins the resolved commit in `stoat.lock`, and +keeps the checkout in `.stoat/recipes/` for a project or `~/.stoat/recipes/` +for the global scope. Git must be installed on the host. + +## Add and search + +Search the configured index by name or description: + +```sh +stoat recipe search my-tools +``` + +Add an index entry by name. An index name does not prompt for confirmation: + +```sh +stoat recipe add my-tools +``` + +Add directly from a repository URL when the source is not in the index. A TTY +shows the manifest name, target OSes, requirements, and parameters before it +asks for confirmation. `-y` skips that prompt: + +```sh +stoat recipe add https://github.com/example/stoat-my-tools@main -y +``` + +The default index is configured as +`https://github.com/novusedge/stoat-recipes`, but that repository currently +returns 404 and is not operational. Set `STOAT_INDEX` to a reachable Git +repository containing `index.toml` until a published index is available: + +```sh +export STOAT_INDEX=/path/to/stoat-recipes +stoat recipe search my-tools +``` + +Stoat does not create or publish that repository. Index refreshes are cached +for 24 hours; `--refresh` forces a new fetch. + +List installed recipes and their scope and short commit pin: + +```sh +stoat recipe list +``` + +Search reads the configured index. `update` addresses an existing remote pin +by its plain name and does not search the index again. This also applies to a +recipe added from a URL: + +```sh +stoat recipe update my-tools +``` + +## Project and global scopes + +If the current directory contains `stoat.toml`, recipe commands use project +scope. The declaration lives in its `[recipes]` table: + +```toml +[recipes] +my-tools = "v1.2" +other-tools = { source = "https://github.com/example/stoat-other-tools", ref = "main" } +``` + +Project scope writes `./stoat.lock` and caches checkouts under +`./.stoat/recipes/`. Stoat adds `.stoat/` to `.gitignore` when the directory +is a Git checkout. Commit both `stoat.toml` and `stoat.lock` so another +checkout can reproduce the same recipe commits. + +Without `stoat.toml`, commands use the global lock at `~/.stoat/stoat.lock` +and cache at `~/.stoat/recipes/`. Pass `--global` to force global scope from a +project directory. Stoat does not search parent directories for a project +file. + +## Lock, sync, update, and remove + +Resolve every project declaration to a commit without changing the cache: + +```sh +stoat recipe lock +``` + +Populate the cache from the lock, removing project cache entries no longer in +the lock: + +```sh +stoat recipe sync +``` + +Fetch refs again and repin one recipe, or every remote recipe when no name is +given: + +```sh +stoat recipe update my-tools +stoat recipe update +``` + +Remove a remote recipe after checking that no VM uses it: + +```sh +stoat recipe rm my-tools -y +``` + +If an add would replace a bundled, local, or same-scope recipe, it refuses +unless the replacement is intentional: + +```sh +stoat recipe add my-tools --global --force +``` + +For removal, use `--force` only when intentionally removing a recipe listed by +a VM. A recipe checkout with local changes is never overwritten by update or +sync; copy it to a local recipe first. + +`apply` in project scope checks that declarations, lock entries, and cache +checkouts agree. A stale declaration reports a repair instruction to run +`stoat recipe lock`; a missing or changed cache is synchronized before apply. diff --git a/docs/reference/json.md b/docs/reference/json.md index 392a0204..ab6b509c 100644 --- a/docs/reference/json.md +++ b/docs/reference/json.md @@ -10,7 +10,7 @@ This document is the contract. The human-facing CLI is documented in ``` stoat --json ls -{"v":2,"type":"result","cmd":"ls","ok":true,"data":{"vms":[...]}} +{"v":3,"type":"result","cmd":"ls","ok":true,"data":{"vms":[...]}} ``` ## The consumer contract, in one page @@ -106,6 +106,8 @@ bump the contract version. Do not write code that requires them. | `invalid_spec` | the request itself is malformed | | `image_not_downloaded` | the image exists in the catalog but not on disk | | `recipe_not_applicable` | a named recipe cannot run on this VM | +| `in_use` | a recipe is still listed by one or more VMs | +| `git_required` | a recipe operation needs Git on `PATH` | | `not_running` | the operation needs a running VM | | `already_running` | the operation needs a stopped VM | | `no_disk` | the VM has no qcow2 (a live VM has none) | @@ -131,6 +133,7 @@ bump the contract version. Do not write code that requires them. | `canceled` | the context was cancelled | | `usage` | a bad flag, a missing argument, an unknown subcommand | | `confirmation_required` | a destructive command was run without `-y` | +| `lock_out_of_date` | a project declaration is not pinned in `stoat.lock` | | `internal` | anything unanticipated; the escape hatch | **Codes are only ever added.** Never renamed, never repurposed, never removed. @@ -202,7 +205,7 @@ Snapshot {"tag":"clean","vm_state":true,"size_display":"203 MiB", "created_display":"2026-08-04 12:00:00"} Check {"name":"qemu-img","ok":false,"detail":"not found", - "fix":["sudo","pacman","-S","qemu-img"]} + "fix":["sudo","pacman","-S","qemu-img"],"optional":false} PruneItem {"class":"orphaned_image","path":"/home/u/.stoat/isos/old.iso"} @@ -219,6 +222,21 @@ RecipeParam {"name":"channel","type":"enum","required":false, RecipeOutput {"name":"socket","help":"path of the socket"} RecipeHealth {"check":"docker info","timeout":"30s"} +RecipeEntry {"name":"tailscale","description":"join a tailnet on boot", + "scope":"global","source":"https://github.com/x/stoat-tailscale", + "ref":"v1.2","commit":"9f3c1e2"} + +RecipeRoot {"path":"/home/u/.stoat/recipes","scope":"global"} + +RecipeAdded {"name":"tailscale","source":"https://github.com/x/stoat-tailscale", + "ref":"v1.2","commit":"9f3c1e2d4a5b6c7d8e9f0a1b2c3d4e5f6a7b8c9d", + "scope":"global"} + +RecipeRemoved {"name":"tailscale","scope":"global"} + +IndexEntry {"name":"tailscale","source":"https://github.com/x/stoat-tailscale", + "description":"join a tailnet on boot","os":["alpine"]} + RecipeIssue {"name":"docker","reason":"docker is not offered to debian/cloudinit"} ApplyPlan {"name":"xfce","action":"run","reason":"never applied", @@ -236,6 +254,14 @@ Guest {"name":"fedora","init":"systemd","shell":"/bin/bash", "source":"bundled"} ``` +`RecipeEntry` has `name`, `description`, `scope`, `source`, `ref`, and +`commit`. `scope` is one of `bundled`, `local`, `global`, or `project`; only +`global` and `project` entries carry `source`, `ref`, and the seven-character +commit prefix. `RecipeRoot` identifies each search root with `path` and +`scope`. `RecipeAdded` uses the same remote pin fields for add, lock, sync, and +update results, with the full resolved commit. `RecipeRemoved` contains only +the name and scope. + `state` is one of `stopped`, `running`, `broken`. `error` appears only on a broken VM. @@ -350,27 +376,23 @@ so a leak fails the build rather than shipping. | `check-recipes` | `{"applicable":false,"issues":[RecipeIssue,...]}` | | `guest ls` | `{"guests":[Guest,...]}` | | `guest show` | `{"guest":Guest}` | -| `recipe list` | `{"dir":"...","recipes":["xfce"]}`, see note below | +| `recipe list` | `{"roots":[RecipeRoot,...],"recipes":[RecipeEntry,...]}` | | `recipe show` | `{"recipe":RecipeSchema}` | -| `recipe new` | `{"path":"/home/u/.stoat/recipes/foo"}` | +| `recipe new` | `{"path":"/home/u/.stoat/recipes/foo/"}` | +| `recipe add` | `{"name":"tailscale","source":"...","ref":"v1.2","commit":"9f3c1e2d4a5b6c7d8e9f0a1b2c3d4e5f6a7b8c9d","scope":"global"}` | +| `recipe lock` | `{"recipes":[RecipeAdded,...]}` | +| `recipe sync` | `{"recipes":[RecipeAdded,...]}` | +| `recipe update` | `{"recipes":[RecipeAdded,...]}` | +| `recipe rm` | `{"name":"tailscale","scope":"global"}` | +| `recipe search` | `{"recipes":[IndexEntry,...]}` | | `screenshot` | `{"vm":"work","path":"/home/u/.stoat/work/screenshots/2026-09-05T140302Z.png","bytes":48213,"width":1280,"height":800}` | | `logs` (no VM) | `{"lines":[...]}` (stoat's own log) | | `logs ` | `{"vm":"work","which":"console","lines":[...]}` | | `doctor` | `{"healthy":false,"checks":[Check,...]}` | -| `version` | `{"version":"1.2.3","contract":2}` | +| `version` | `{"version":"1.2.3","contract":3}` | | `help` | `{"usage":"..."}` | | `ssh` | **refused**, see below | -Both `recipe` subcommands report `"cmd":"recipe"`, not `"cmd":"recipe list"`, -and both `guest` subcommands report `"cmd":"guest"`. Distinguish them by which -fields `data` carries. - -`recipe list` is "every file in the recipes directory", which is not the same -as "every recipe you can use": it currently includes the `.bak` files the -one-time manifest upgrade left behind, and those are not applicable to any VM. -Use `recipes` (which filters by OS and backend) to find something a VM can -actually run; use `recipe list` only to find a recipe directory to inspect. - `get` returns `{"vm":VMStatus}`: `VMStatus` embeds the VM fields directly; only the outer get result has the `vm` member. `recipes` remains the compatible string list, while `recipes_detail` adds stored per-recipe state. `health` is the stored aggregate @@ -382,6 +404,15 @@ string list, while `recipes_detail` adds stored per-recipe state. `health` is th and outputs are named arrays sorted by name. A recipe without a health check has `health:null`; all list fields are `[]`, never `null`. +All `recipe` subcommands report `"cmd":"recipe"`, not the full subcommand +path, and all `guest` subcommands report `"cmd":"guest"`. Distinguish them by +which fields `data` carries. + +`recipe list` reports every valid manifest in shadow order. Each row names its +scope (`bundled`, `local`, `global`, or `project`); only remote `global` and +`project` rows carry source, ref, and the seven-character commit prefix. The +`roots` list gives the search order and the scope label for each root. + Fields worth knowing about: - **`update.changed`** names the fields that actually changed, in wire naming @@ -432,9 +463,9 @@ faking one would break the exactly-one-result guarantee everywhere. Use --dry-run` emits none: it computes the plan host-side and runs nothing. ``` -{"v":2,"type":"progress","cmd":"pull","data":{"id":"alpine-virt","done":41943040,"total":62914560,"percent":66}} -{"v":2,"type":"progress","cmd":"pull","data":{"id":"alpine-virt","done":62914560,"total":62914560,"percent":100}} -{"v":2,"type":"result","cmd":"pull","ok":true,"data":{"id":"alpine-virt","downloaded":true,"verified":true,"checksum_available":true}} +{"v":3,"type":"progress","cmd":"pull","data":{"id":"alpine-virt","done":41943040,"total":62914560,"percent":66}} +{"v":3,"type":"progress","cmd":"pull","data":{"id":"alpine-virt","done":62914560,"total":62914560,"percent":100}} +{"v":3,"type":"result","cmd":"pull","ok":true,"data":{"id":"alpine-virt","downloaded":true,"verified":true,"checksum_available":true}} ``` `progress` fires only when the percentage changes, not per read. @@ -443,8 +474,8 @@ faking one would break the exactly-one-result guarantee everywhere. Use a `stage` event at each recipe boundary: ``` -{"v":2,"type":"stage","cmd":"apply","data":{"recipe":"xfce"}} -{"v":2,"type":"log","cmd":"apply","data":{"line":"+ apk add xfce4"}} +{"v":3,"type":"stage","cmd":"apply","data":{"recipe":"xfce"}} +{"v":3,"type":"log","cmd":"apply","data":{"line":"+ apk add xfce4"}} ``` The stage boundaries are real, read out of the markers the provisioner already @@ -455,7 +486,7 @@ streaming into "silent until exit". ## Versioning -`"v"` is an integer **contract** version, not the build version. It is `2`. +`"v"` is an integer **contract** version, not the build version. It is `3`. It bumps only for a removal or a meaning change: a field deleted, a unit changed, an error code split or repurposed, or `result` ceasing to be last. @@ -490,3 +521,8 @@ A consumer may rely on: A consumer may **not** rely on: field order, the exact text of `message`, the contents of any `*_display` field, or the absence of fields it does not know. + +**v3.** `recipe list` changed shape for remote recipes. `dir` became `roots`, +a list of `{path, scope}` in search order, and `recipes` became a list of +`RecipeEntry` objects rather than names. A consumer that read +`data.recipes[]` as strings reads `data.recipes[].name` instead. diff --git a/go.mod b/go.mod index 744ca0a1..e616c356 100644 --- a/go.mod +++ b/go.mod @@ -11,6 +11,7 @@ require ( github.com/BurntSushi/toml v1.6.0 github.com/alecthomas/kong v1.16.0 github.com/charmbracelet/x/ansi v0.11.7 + github.com/charmbracelet/x/term v0.2.2 github.com/pelletier/go-toml/v2 v2.4.3 golang.org/x/sys v0.47.0 gopkg.in/yaml.v3 v3.0.1 @@ -24,7 +25,6 @@ require ( github.com/charmbracelet/ultraviolet v0.0.0-20260703014108-f5a850f9c2b7 // indirect github.com/charmbracelet/x/exp/ordered v0.1.0 // indirect github.com/charmbracelet/x/exp/strings v0.0.0-20240722160745-212f7b056ed0 // indirect - github.com/charmbracelet/x/term v0.2.2 // indirect github.com/charmbracelet/x/termios v0.1.1 // indirect github.com/charmbracelet/x/windows v0.2.2 // indirect github.com/clipperhouse/displaywidth v0.11.0 // indirect diff --git a/internal/cli/cli.go b/internal/cli/cli.go index 171d7a80..b5865fe8 100644 --- a/internal/cli/cli.go +++ b/internal/cli/cli.go @@ -77,6 +77,11 @@ type Args struct { // OS and Backend belong to "recipe new"; VM carries the recipe name // there, since it is the same "one positional argument" slot. Sub string + Global bool + Force bool + Refresh bool + Ref string + Names []string OS string Backend string @@ -195,6 +200,7 @@ func Parse(args []string) (*Args, error) { if len(args) > 0 && args[0] == "exec" { return parseExec(args[1:]) } + args = preserveRecipeSearchTerm(args) var g grammar var help bytes.Buffer @@ -237,6 +243,31 @@ func Parse(args []string) (*Args, error) { return a, nil } +// preserveRecipeSearchTerm inserts Kong's argument terminator before a search +// term that starts with a dash. Search terms are data, so Kong must not parse +// their leading characters as short or long flags. +func preserveRecipeSearchTerm(args []string) []string { + for i := 0; i+2 < len(args); i++ { + if args[i] != "recipe" || args[i+1] != "search" || args[i+2] == "--" { + continue + } + for j := i + 2; j < len(args); j++ { + if args[j] == "--refresh" { + continue + } + if !strings.HasPrefix(args[j], "-") || args[j] == "--" { + break + } + out := make([]string, 0, len(args)+1) + out = append(out, args[:j]...) + out = append(out, "--") + out = append(out, args[j:]...) + return out + } + } + return args +} + // parseExec handles `exec ...` without kong. Kong's passthrough is // not verbatim enough for this command: // @@ -437,7 +468,7 @@ func Main(args []string, version string, stdin io.Reader, stdout, stderr io.Writ case "rm": return runRM(a, stdin, stdout, stderr) case "recipe": - return runRecipe(a, stdout, stderr) + return runRecipe(a, stdin, stdout, stderr) case "guest": return runGuest(a, stdout, stderr) case "logs": diff --git a/internal/cli/confirm.go b/internal/cli/confirm.go new file mode 100644 index 00000000..f793b83f --- /dev/null +++ b/internal/cli/confirm.go @@ -0,0 +1,42 @@ +package cli + +import ( + "bufio" + "fmt" + "io" + "os" + "strings" + + "github.com/charmbracelet/x/term" + + "github.com/novusedge/stoat/internal/cli/wire" +) + +// confirm is the one gate every destructive command uses. It only reads a +// response from a terminal; pipes and JSON callers must opt in with -y. +func confirm(a *Args, stdin io.Reader, stdout, stderr io.Writer, prompt string) (bool, int) { + if a.Yes { + return true, ExitOK + } + if a.JSON { + return false, a.failMsg(stdout, stderr, wire.ErrConfirmationRequired, prompt+"; pass -y to confirm") + } + if a.Quiet || !terminal(stdin) || !terminal(stdout) { + return false, a.failMsg(stdout, stderr, wire.ErrConfirmationRequired, prompt+"; pass -y to confirm") + } + fmt.Fprintf(stdout, "%s [y/N] ", prompt) + line, _ := bufio.NewReader(stdin).ReadString('\n') + if strings.ToLower(strings.TrimSpace(line)) != "y" { + fmt.Fprintln(stdout, "aborted") + return false, ExitFail + } + return true, ExitOK +} + +func terminal(v any) bool { + f, ok := v.(*os.File) + if !ok || f == nil { + return false + } + return term.IsTerminal(f.Fd()) +} diff --git a/internal/cli/confirm_linux_test.go b/internal/cli/confirm_linux_test.go new file mode 100644 index 00000000..7ab14248 --- /dev/null +++ b/internal/cli/confirm_linux_test.go @@ -0,0 +1,197 @@ +//go:build linux + +package cli + +import ( + "bytes" + "encoding/json" + "fmt" + "os" + "path/filepath" + "strings" + "testing" + "time" + + "github.com/novusedge/stoat/internal/cli/wire" + "github.com/novusedge/stoat/internal/recipes" + "github.com/novusedge/stoat/internal/testutil" + "golang.org/x/sys/unix" +) + +func openTestPTY(t *testing.T) (master, slave *os.File) { + t.Helper() + master, err := os.OpenFile("/dev/ptmx", os.O_RDWR|unix.O_NOCTTY, 0) + if err != nil { + t.Skipf("pseudo-terminal unavailable: %v", err) + } + t.Cleanup(func() { _ = master.Close() }) + n, err := unix.IoctlGetInt(int(master.Fd()), unix.TIOCGPTN) + if err != nil { + t.Skipf("cannot query pseudo-terminal: %v", err) + } + if err := unix.IoctlSetPointerInt(int(master.Fd()), unix.TIOCSPTLCK, 0); err != nil { + t.Skipf("cannot unlock pseudo-terminal: %v", err) + } + slave, err = os.OpenFile(fmt.Sprintf("/dev/pts/%d", n), os.O_RDWR|unix.O_NOCTTY, 0) + if err != nil { + t.Skipf("cannot open pseudo-terminal slave: %v", err) + } + t.Cleanup(func() { _ = slave.Close() }) + return master, slave +} + +func TestRecipeURLPreviewUsesManifestFieldsOnATTY(t *testing.T) { + cliRoot(t) + t.Chdir(t.TempDir()) + src := testutil.GitRepo(t, map[string]string{ + "recipe.toml": "schema = 3\nname = \"demo\"\ndescription = \"demo recipe\"\nos = [\"alpine\"]\nrequires = [\"git\"]\nscript = \"install.sh\"\n\n[params.channel]\ntype = \"enum\"\nvalues = [\"stable\", \"test\"]\ndefault = \"stable\"\n", + "install.sh": "#!/bin/sh\necho demo\n", + }) + bare := filepath.Join(filepath.Dir(src), "demo.git") + if err := os.Rename(src, bare); err != nil { + t.Fatal(err) + } + master, tty := openTestPTY(t) + if _, err := master.Write([]byte("y\n")); err != nil { + t.Fatal(err) + } + var errOut bytes.Buffer + code := Main([]string{"recipe", "add", bare, "--global"}, "test", tty, tty, &errOut) + if code != ExitOK { + t.Fatalf("TTY URL add exit = %d, stderr = %q", code, errOut.String()) + } + if err := master.SetReadDeadline(time.Now().Add(time.Second)); err != nil { + t.Fatal(err) + } + buf := make([]byte, 4096) + n, err := master.Read(buf) + if err != nil { + t.Fatal(err) + } + output := string(buf[:n]) + normalized := strings.ReplaceAll(output, "\r\n", "\n") + start := strings.Index(normalized, "name: demo") + if start < 0 { + t.Fatalf("TTY output has no manifest preview: %q", output) + } + preview, _, ok := strings.Cut(normalized[start:], "install demo from ") + if !ok { + t.Fatalf("TTY output has no confirmation prompt: %q", output) + } + if want := "name: demo\nos: alpine\nrequires: git\nparam: channel (enum)\n"; preview != want { + t.Fatalf("TTY preview = %q, want exactly %q", preview, want) + } + if strings.Contains(preview, "description:") { + t.Fatalf("TTY preview exposed unapproved description field: %q", preview) + } + for _, want := range []string{"demo", "alpine", "git", "channel"} { + if !strings.Contains(output, want) { + t.Errorf("TTY preview missing %q: %q", want, output) + } + } +} + +func TestRecipeURLNonTTYFilesRefuseBeforePreviewAndMutation(t *testing.T) { + cliRoot(t) + t.Chdir(t.TempDir()) + src := cliRecipeRepo(t, "demo", "#!/bin/sh\necho demo\n") + trace := filepath.Join(t.TempDir(), "git-trace.json") + t.Setenv("GIT_TRACE2_EVENT", trace) + + cases := []struct { + name string + stdin func(t *testing.T) *os.File + stdoutNull bool + args []string + }{ + {name: "dev-null", stdin: func(t *testing.T) *os.File { + f, err := os.Open("/dev/null") + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = f.Close() }) + return f + }, stdoutNull: true}, + {name: "pipe", stdin: func(t *testing.T) *os.File { + r, w, err := os.Pipe() + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = r.Close(); _ = w.Close() }) + return r + }}, + {name: "json-dev-null", stdin: func(t *testing.T) *os.File { + f, err := os.Open("/dev/null") + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = f.Close() }) + return f + }, args: []string{"--json"}}, + } + + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + _ = os.Remove(trace) + root := cliRoot(t) + t.Chdir(t.TempDir()) + stdin := tc.stdin(t) + stdoutPath := filepath.Join(t.TempDir(), "stdout") + var err error + var stdout *os.File + if tc.stdoutNull { + stdout, err = os.OpenFile("/dev/null", os.O_WRONLY, 0) + } else { + stdout, err = os.Create(stdoutPath) + } + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = stdout.Close() }) + args := append(append([]string{}, tc.args...), "recipe", "add", src, "--global") + if tc.name == "pipe" { + args = append([]string{"--quiet"}, args...) + } + var errOut bytes.Buffer + code := Main(args, "test", stdin, stdout, &errOut) + if err := stdout.Close(); err != nil { + t.Fatal(err) + } + var body []byte + if !tc.stdoutNull { + body, err = os.ReadFile(stdoutPath) + if err != nil { + t.Fatal(err) + } + } + if code != ExitFail { + t.Fatalf("non-TTY URL add exit = %d, want ExitFail; stdout=%q stderr=%q", code, body, errOut.String()) + } + if tc.name == "json-dev-null" { + var envelope map[string]any + if err := json.Unmarshal(bytes.TrimSpace(body), &envelope); err != nil { + t.Fatalf("JSON refusal is not one envelope: %v; output=%q", err, body) + } + errObj, _ := envelope["error"].(map[string]any) + if errObj["code"] != string(wire.CodeConfirmationRequired) { + t.Fatalf("JSON refusal code = %v, want %q", errObj["code"], wire.CodeConfirmationRequired) + } + } else if !tc.stdoutNull && len(body) != 0 { + t.Fatalf("quiet non-TTY refusal wrote preview/prompt prose: %q", body) + } + scope, err := recipes.ScopeFor(true) + if err != nil { + t.Fatal(err) + } + if _, err := os.Stat(scope.LockPath); !os.IsNotExist(err) { + t.Fatalf("non-TTY refusal mutated global lock under %s: %v", root, err) + } + if _, err := os.Stat(trace); err == nil { + traceBody, readErr := os.ReadFile(trace) + if readErr != nil || len(traceBody) > 0 { + t.Fatalf("non-TTY refusal triggered Git preview trace: %q", traceBody) + } + } + }) + } +} diff --git a/internal/cli/confirm_noninteractive_test.go b/internal/cli/confirm_noninteractive_test.go new file mode 100644 index 00000000..9dfd3747 --- /dev/null +++ b/internal/cli/confirm_noninteractive_test.go @@ -0,0 +1,30 @@ +package cli + +import ( + "bytes" + "strings" + "testing" + + "github.com/novusedge/stoat/internal/config" +) + +func TestQuietHumanVMRemoveRefusalExplainsHowToConfirm(t *testing.T) { + cliRoot(t) + if err := (&config.VM{Name: "work", Mode: "live", RAM: 1024, CPUs: 1, SSHPort: 2200}).Save(); err != nil { + t.Fatal(err) + } + var out, errOut bytes.Buffer + code := Main([]string{"--quiet", "rm", "work"}, "test", nil, &out, &errOut) + if code != ExitFail { + t.Fatalf("quiet rm exit = %d, want ExitFail", code) + } + if out.Len() != 0 { + t.Fatalf("quiet rm wrote stdout: %q", out.String()) + } + if !strings.Contains(errOut.String(), "pass -y") { + t.Fatalf("quiet rm refusal is not actionable: %q", errOut.String()) + } + if _, err := config.Load("work"); err != nil { + t.Fatalf("quiet refusal removed VM: %v", err) + } +} diff --git a/internal/cli/doctor_test.go b/internal/cli/doctor_test.go new file mode 100644 index 00000000..028e8107 --- /dev/null +++ b/internal/cli/doctor_test.go @@ -0,0 +1,100 @@ +package cli + +import ( + "bytes" + "encoding/json" + "os" + "os/exec" + "path/filepath" + "strings" + "testing" + + "github.com/novusedge/stoat/internal/core" +) + +func TestDoctorShowsOptionalGitRepairWithoutUnhealthy(t *testing.T) { + cliRoot(t) + bin := t.TempDir() + for _, name := range []string{"qemu-system-x86_64", "qemu-img", "ssh", "xorriso"} { + path := filepath.Join(bin, name) + body := "#!/bin/sh\nexit 0\n" + if err := os.WriteFile(path, []byte(body), 0o755); err != nil { + t.Fatal(err) + } + } + sshKeygen, err := exec.LookPath("ssh-keygen") + if err != nil { + t.Skipf("ssh-keygen unavailable for CLI setup: %v", err) + } + if err := os.Symlink(sshKeygen, filepath.Join(bin, "ssh-keygen")); err != nil { + t.Fatal(err) + } + t.Setenv("PATH", bin) + + checks := core.Doctor() + var gitFix string + for _, c := range checks { + switch c.Name { + case "git": + if c.OK { + t.Fatal("git unexpectedly passed with a PATH containing only required stubs") + } + if !c.Optional { + t.Fatal("Doctor marked missing git as required") + } + gitFix = strings.Join(c.Fix, " && ") + case "/dev/kvm": + if !c.OK { + t.Skipf("host does not provide a usable /dev/kvm: %s", c.Detail) + } + default: + if !c.OK { + t.Fatalf("required check %s failed in isolated PATH: %s", c.Name, c.Detail) + } + } + } + if gitFix == "" { + t.Fatal("Doctor omitted the optional git fix") + } + + var out, errOut bytes.Buffer + if code := Main([]string{"doctor"}, "test", nil, &out, &errOut); code != ExitOK { + t.Fatalf("human doctor exit = %d, want ExitOK; stdout=%q stderr=%q", code, out.String(), errOut.String()) + } + if !strings.Contains(strings.ToLower(out.String()), "optional") || !strings.Contains(out.String(), "git") { + t.Fatalf("human doctor did not identify optional Git: %q", out.String()) + } + if !strings.Contains(out.String(), gitFix) { + t.Fatalf("human doctor omitted Git repair command %q: %q", gitFix, out.String()) + } + if strings.Contains(out.String(), "FAIL:") { + t.Fatalf("optional Git made a healthy host look failed: %q", out.String()) + } + if errOut.Len() != 0 { + t.Fatalf("human doctor stderr = %q", errOut.String()) + } + + var jsonOut, jsonErr bytes.Buffer + if code := Main([]string{"--json", "doctor"}, "test", nil, &jsonOut, &jsonErr); code != ExitOK { + t.Fatalf("JSON doctor exit = %d, want ExitOK; stdout=%q stderr=%q", code, jsonOut.String(), jsonErr.String()) + } + var envelope map[string]any + if err := json.Unmarshal(bytes.TrimSpace(jsonOut.Bytes()), &envelope); err != nil { + t.Fatalf("JSON doctor output is not one envelope: %v; output=%q", err, jsonOut.String()) + } + data, _ := envelope["data"].(map[string]any) + if data["healthy"] != true { + t.Fatalf("JSON doctor healthy = %v, want true with only optional Git missing", data["healthy"]) + } + rows, _ := data["checks"].([]any) + for _, raw := range rows { + row, _ := raw.(map[string]any) + if row["name"] == "git" { + if row["optional"] != true { + t.Fatalf("JSON Git row optional = %v, want true", row["optional"]) + } + return + } + } + t.Fatal("JSON doctor omitted the Git check") +} diff --git a/internal/cli/grammar.go b/internal/cli/grammar.go index 99136382..32bbbaff 100644 --- a/internal/cli/grammar.go +++ b/internal/cli/grammar.go @@ -222,9 +222,15 @@ type checkRecipesCmd struct { } type recipeCmd struct { - List recipeListCmd `cmd:"" help:"list installed recipes and where they live"` - New recipeNewCmd `cmd:"" help:"scaffold a recipe in the recipes directory"` - Show recipeShowCmd `cmd:"" help:"print one recipe's params, outputs and health check"` + List recipeListCmd `cmd:"" aliases:"ls" help:"list recipes with their scope and pinned commit"` + New recipeNewCmd `cmd:"" help:"scaffold a recipe in the recipes directory"` + Show recipeShowCmd `cmd:"" help:"print one recipe's params, outputs and health check"` + Add recipeAddCmd `cmd:"" help:"install a recipe from the index or a git URL"` + Lock recipeLockCmd `cmd:"" help:"resolve every declaration to a commit"` + Sync recipeSyncCmd `cmd:"" help:"make the recipe cache match the lock"` + Update recipeUpdateCmd `cmd:"" help:"fetch a recipe's ref again and repin it"` + RM recipeRmCmd `cmd:"" name:"rm" help:"remove a remote recipe"` + Search recipeSearchCmd `cmd:"" help:"search the recipe index"` } type recipeListCmd struct{} @@ -239,6 +245,38 @@ type recipeShowCmd struct { Name string `arg:"" help:"recipe name"` } +type recipeAddCmd struct { + Ref string `arg:"" help:"index name, or a git URL, either with an optional @tag"` + Yes bool `short:"y" help:"skip the confirmation for a URL source"` + Global bool `help:"use the home scope even inside a project"` + Force bool `help:"install over a name that already exists"` +} + +type recipeLockCmd struct { + Global bool `help:"use the home scope even inside a project"` +} + +type recipeSyncCmd struct { + Global bool `help:"use the home scope even inside a project"` +} + +type recipeUpdateCmd struct { + Names []string `arg:"" optional:"" help:"recipe names; omit for every remote recipe"` + Global bool `help:"use the home scope even inside a project"` +} + +type recipeRmCmd struct { + Name string `arg:"" help:"recipe name"` + Yes bool `short:"y" help:"skip the delete confirmation"` + Global bool `help:"use the home scope even inside a project"` + Force bool `help:"remove it even while a VM lists it"` +} + +type recipeSearchCmd struct { + Term []string `arg:"" optional:"" passthrough:"all" help:"match against name and description"` + Refresh bool `help:"fetch the index even when it is fresh"` +} + type recipeGuestCmd struct { LS guestLsCmd `cmd:"" name:"ls" help:"one line per guest: name, init, package manager, backend, source"` Show guestShowCmd `cmd:"" help:"the merged definition of one guest"` @@ -456,6 +494,36 @@ func (g *grammar) toArgs(path string) (*Args, error) { a.Cmd, a.Sub = "recipe", "show" a.VM = g.Recipe.Show.Name + case "recipe add": + c := g.Recipe.Add + a.Cmd, a.Sub = "recipe", "add" + a.Ref, a.Yes, a.Global, a.Force = c.Ref, c.Yes, c.Global, c.Force + + case "recipe lock": + a.Cmd, a.Sub = "recipe", "lock" + a.Global = g.Recipe.Lock.Global + + case "recipe sync": + a.Cmd, a.Sub = "recipe", "sync" + a.Global = g.Recipe.Sync.Global + + case "recipe update": + a.Cmd, a.Sub = "recipe", "update" + a.Names, a.Global = trimList(g.Recipe.Update.Names), g.Recipe.Update.Global + + case "recipe rm": + c := g.Recipe.RM + a.Cmd, a.Sub = "recipe", "rm" + a.Names, a.Yes, a.Global, a.Force = []string{c.Name}, c.Yes, c.Global, c.Force + + case "recipe search": + a.Cmd, a.Sub = "recipe", "search" + term := g.Recipe.Search.Term + if len(term) > 0 && term[0] == "--" { + term = term[1:] + } + a.Ref, a.Refresh = strings.Join(term, " "), g.Recipe.Search.Refresh + case "guest ls": a.Cmd, a.Sub = "guest", "ls" diff --git a/internal/cli/kong_test.go b/internal/cli/kong_test.go index 5eae4924..cceeeb13 100644 --- a/internal/cli/kong_test.go +++ b/internal/cli/kong_test.go @@ -56,6 +56,57 @@ func TestKongFlagNamingSurface(t *testing.T) { } } +func TestParseRecipeSubcommands(t *testing.T) { + tests := []struct { + argv []string + sub string + want func(*Args) bool + }{ + {[]string{"recipe", "add", "tailscale@v1.2"}, "add", func(a *Args) bool { return a.Ref == "tailscale@v1.2" && !a.Global }}, + {[]string{"recipe", "add", "tailscale", "--global", "-y"}, "add", func(a *Args) bool { return a.Global && a.Yes }}, + {[]string{"recipe", "lock"}, "lock", func(a *Args) bool { return !a.Global }}, + {[]string{"recipe", "sync", "--global"}, "sync", func(a *Args) bool { return a.Global }}, + {[]string{"recipe", "update"}, "update", func(a *Args) bool { return len(a.Names) == 0 }}, + {[]string{"recipe", "update", "tailscale"}, "update", func(a *Args) bool { return len(a.Names) == 1 && a.Names[0] == "tailscale" }}, + {[]string{"recipe", "rm", "tailscale", "--force"}, "rm", func(a *Args) bool { return a.Force && a.Names[0] == "tailscale" }}, + {[]string{"recipe", "search", "tail"}, "search", func(a *Args) bool { return a.Ref == "tail" }}, + {[]string{"recipe", "search", "-tail"}, "search", func(a *Args) bool { return a.Ref == "-tail" }}, + {[]string{"recipe", "ls"}, "list", func(a *Args) bool { return true }}, + } + for _, tt := range tests { + t.Run(strings.Join(tt.argv, " "), func(t *testing.T) { + a, err := Parse(tt.argv) + if err != nil { + t.Fatal(err) + } + if a.Cmd != "recipe" || a.Sub != tt.sub { + t.Fatalf("cmd/sub = %q/%q, want recipe/%q", a.Cmd, a.Sub, tt.sub) + } + if !tt.want(a) { + t.Errorf("args = %+v", a) + } + }) + } +} + +func TestParseRecipeFlagsDoNotChangeUnrelatedCommands(t *testing.T) { + a, err := Parse([]string{"guest", "ls"}) + if err != nil { + t.Fatal(err) + } + if a.Cmd != "guest" || a.Sub != "ls" { + t.Fatalf("guest ls = %+v, want the existing command", a) + } + + a, err = Parse([]string{"recipe", "new", "demo", "--os", "alpine"}) + if err != nil { + t.Fatal(err) + } + if a.Cmd != "recipe" || a.Sub != "new" || a.VM != "demo" || a.OS != "alpine" { + t.Fatalf("recipe new = %+v, want the existing scaffold command", a) + } +} + // TestKongFlagNamingSurfacePopulatesTheRightField goes one step further than // acceptance: --cpus must land on the CPUs field, not merely parse without // error. A wrong name tag (e.g. pointing --cpus at RAM) would pass the diff --git a/internal/cli/remote_snapshot_final_test.go b/internal/cli/remote_snapshot_final_test.go new file mode 100644 index 00000000..d345fd20 --- /dev/null +++ b/internal/cli/remote_snapshot_final_test.go @@ -0,0 +1,125 @@ +package cli + +import ( + "bytes" + "encoding/json" + "os" + "path/filepath" + "syscall" + "testing" + "time" + + "github.com/novusedge/stoat/internal/recipes" + "github.com/novusedge/stoat/internal/testutil" +) + +func TestRecipeListBlocksAcrossAnIntermediateCacheAndLockPublication(t *testing.T) { + cliRoot(t) + t.Chdir(t.TempDir()) + src := cliRecipeRepo(t, "demo", "#!/bin/sh\necho v1\n") + scope, err := recipes.ScopeFor(true) + if err != nil { + t.Fatal(err) + } + if _, err := recipes.Add(scope, src, false); err != nil { + t.Fatal(err) + } + oldLock, err := scope.Lock() + if err != nil { + t.Fatal(err) + } + oldCommit := oldLock.Recipes["demo"].Commit + newCommit := testutil.GitCommit(t, src, map[string]string{ + "recipe.toml": "schema = 3\nname = \"demo\"\ndescription = \"demo-v2 recipe\"\nos = [\"alpine\"]\nrequires = [\"git\"]\nscript = \"install.sh\"\n\n[params.channel]\ntype = \"enum\"\nvalues = [\"stable\", \"test\"]\ndefault = \"stable\"\n", + "install.sh": "#!/bin/sh\necho v2\n", + }, "") + newCache := testutil.GitClone(t, src) + newLock := recipes.Lock{Schema: oldLock.Schema, Recipes: make(map[string]recipes.LockEntry, len(oldLock.Recipes))} + for name, entry := range oldLock.Recipes { + newLock.Recipes[name] = entry + } + entry := newLock.Recipes["demo"] + entry.Commit = newCommit + newLock.Recipes["demo"] = entry + + coordPath := filepath.Join(scope.Dir, "recipe.lock") + coord, err := os.OpenFile(coordPath, os.O_CREATE|os.O_RDWR, 0o644) + if err != nil { + t.Fatal(err) + } + defer func() { _ = coord.Close() }() + if err := syscall.Flock(int(coord.Fd()), syscall.LOCK_EX); err != nil { + t.Fatal(err) + } + defer func() { _ = syscall.Flock(int(coord.Fd()), syscall.LOCK_UN) }() + cache := filepath.Join(scope.CachePath, "demo") + oldCache := filepath.Join(t.TempDir(), "old-demo") + if err := os.Rename(cache, oldCache); err != nil { + t.Fatal(err) + } + if err := os.Rename(newCache, cache); err != nil { + t.Fatal(err) + } + + type listResult struct { + code int + out string + err string + } + done := make(chan listResult, 1) + go func() { + var out, errOut bytes.Buffer + code := Main([]string{"--json", "recipe", "list"}, "test", nil, &out, &errOut) + done <- listResult{code: code, out: out.String(), err: errOut.String()} + }() + select { + case got := <-done: + t.Fatalf("recipe list returned while publication lock was held: code=%d stdout=%q stderr=%q", got.code, got.out, got.err) + case <-time.After(500 * time.Millisecond): + } + + if err := recipes.SaveLock(scope.LockPath, newLock); err != nil { + t.Fatal(err) + } + if err := syscall.Flock(int(coord.Fd()), syscall.LOCK_UN); err != nil { + t.Fatal(err) + } + select { + case got := <-done: + if got.code != ExitOK { + t.Fatalf("recipe list exit = %d, stdout=%q, stderr=%q", got.code, got.out, got.err) + } + var lines []map[string]any + for _, line := range bytes.Split(bytes.TrimSpace([]byte(got.out)), []byte{'\n'}) { + var obj map[string]any + if err := json.Unmarshal(line, &obj); err != nil { + t.Fatalf("recipe list line = %q: %v", line, err) + } + lines = append(lines, obj) + } + if len(lines) != 1 { + t.Fatalf("recipe list emitted %d result lines: %q", len(lines), got.out) + } + data, _ := lines[0]["data"].(map[string]any) + rows, _ := data["recipes"].([]any) + var row map[string]any + for _, raw := range rows { + candidate, _ := raw.(map[string]any) + if candidate["name"] == "demo" { + row = candidate + break + } + } + if row == nil { + t.Fatalf("recipe list omitted demo: %v", rows) + } + if row["description"] != "demo-v2 recipe" { + t.Fatalf("recipe list saw an incomplete publication: row=%v, old commit=%s, new commit=%s", row, oldCommit, newCommit) + } + if row["commit"] != newCommit[:7] { + t.Fatalf("recipe list paired the new manifest with the wrong pin: row=%v, want commit %s", row, newCommit[:7]) + } + case <-time.After(5 * time.Second): + t.Fatal("recipe list did not finish after the publication lock was released") + } +} diff --git a/internal/cli/run_misc.go b/internal/cli/run_misc.go index 829bf774..e70b95d8 100644 --- a/internal/cli/run_misc.go +++ b/internal/cli/run_misc.go @@ -125,32 +125,55 @@ func splitTail(b []byte, n int) []string { return lines } -// runRecipe implements "recipe list" and "recipe new". Authoring a recipe has +// runRecipe implements the recipe subcommands. Authoring a recipe has // always been "put a correctly named file in the recipes directory": the // only real problem was that nothing told you so, or what the name had to be. -func runRecipe(a *Args, stdout, stderr io.Writer) int { +func runRecipe(a *Args, stdin io.Reader, stdout, stderr io.Writer) int { switch a.Sub { case "list": - names, err := recipes.Installed() + snapshot, err := recipes.ListSnapshot() if err != nil { return a.fail(stdout, stderr, err) } - if a.JSON { - if names == nil { - names = []string{} + out := wire.RecipeList{Roots: make([]wire.RecipeRoot, 0, len(snapshot.Roots)), Recipes: make([]wire.RecipeEntry, 0, len(snapshot.Manifests))} + for _, root := range snapshot.Roots { + out.Roots = append(out.Roots, wire.RecipeRoot{Path: root.Path, Scope: root.Scope}) + } + for _, m := range snapshot.Manifests { + scope := snapshot.Scopes[m.Name] + e := wire.RecipeEntry{Name: m.Name, Description: m.Description, Scope: scope} + if pin, ok := snapshot.Pins[m.Name]; ok { + e.Source, e.Ref, e.Commit = pin.Source, pin.Ref, short(pin.Commit) } - return a.ok(stdout, map[string]any{"dir": recipes.Dir(), "recipes": names}) + out.Recipes = append(out.Recipes, e) } - fmt.Fprintln(stdout, recipes.Dir()) - if len(names) == 0 { + out.Roots, out.Recipes = wire.NonNil(out.Roots), wire.NonNil(out.Recipes) + if a.JSON { + return a.ok(stdout, out) + } + fmt.Fprintf(stdout, "%-20s %-9s %-8s %s\n", "NAME", "SCOPE", "COMMIT", "DESCRIPTION") + if len(out.Recipes) == 0 { fmt.Fprintln(stdout, " (none)") return ExitOK } - for _, n := range names { - fmt.Fprintln(stdout, " "+n) + for _, e := range out.Recipes { + fmt.Fprintf(stdout, "%-20s %-9s %-8s %s\n", e.Name, e.Scope, e.Commit, e.Description) } return ExitOK + case "add": + return runRecipeAdd(a, stdin, stdout, stderr) + case "lock": + return runRecipeLock(a, stdout, stderr) + case "sync": + return runRecipeSync(a, stdout, stderr) + case "update": + return runRecipeUpdate(a, stdout, stderr) + case "rm": + return runRecipeRM(a, stdin, stdout, stderr) + case "search": + return runRecipeSearch(a, stdout, stderr) + case "new": path, err := recipes.New(a.VM, a.OS, a.Backend) if err != nil { @@ -168,7 +191,7 @@ func runRecipe(a *Args, stdout, stderr io.Writer) int { case "show": return runRecipeShow(a, stdout, stderr) } - // Unreachable: Parse rejects any action but list/new. + // Unreachable: Parse rejects any action but the declared recipe commands. if a.JSON { _ = wire.NewEmitter(stdout).ResultErr(a.Cmd, wire.UsageError("recipe: unknown action "+a.Sub)) return ExitUsage diff --git a/internal/cli/run_recipe_remote.go b/internal/cli/run_recipe_remote.go new file mode 100644 index 00000000..207d4ab4 --- /dev/null +++ b/internal/cli/run_recipe_remote.go @@ -0,0 +1,216 @@ +package cli + +import ( + "errors" + "fmt" + "io" + "os" + "sort" + "strings" + + "github.com/novusedge/stoat/internal/cli/wire" + "github.com/novusedge/stoat/internal/core" + "github.com/novusedge/stoat/internal/gitx" + "github.com/novusedge/stoat/internal/hostcheck" + "github.com/novusedge/stoat/internal/recipes" +) + +const shortSHA = 7 + +func short(sha string) string { + if len(sha) > shortSHA { + return sha[:shortSHA] + } + return sha +} + +func runRecipeAdd(a *Args, stdin io.Reader, stdout, stderr io.Writer) int { + s, err := recipes.ScopeFor(a.Global) + if err != nil { + return a.fail(stdout, stderr, err) + } + source, gitRef, isURL := recipes.ParseRef(a.Ref) + if isURL && !a.Yes { + if a.JSON || !terminal(stdin) || !terminal(stdout) { + _, code := confirm(a, stdin, stdout, stderr, "install this recipe; pass -y to confirm") + return code + } + m, tmp, previewErr := recipes.Preview(source, gitRef) + if previewErr != nil { + return a.fail(stdout, stderr, previewErr) + } + if removeErr := os.RemoveAll(tmp); removeErr != nil { + return a.fail(stdout, stderr, removeErr) + } + fmt.Fprintf(stdout, "name: %s\n", m.Name) + fmt.Fprintf(stdout, "os: %s\n", strings.Join(m.OS, ", ")) + fmt.Fprintf(stdout, "requires: %s\n", strings.Join(m.Requires, ", ")) + for _, p := range m.SortedParams() { + fmt.Fprintf(stdout, "param: %s (%s)\n", p.Name, p.Type) + } + if ok, code := confirm(a, stdin, stdout, stderr, "install "+m.Name+" from "+source+"?"); !ok { + return code + } + } + e, err := recipes.Add(s, a.Ref, a.Force) + if err != nil { + return a.fail(stdout, stderr, recipeGitError(err, "add")) + } + if a.JSON { + return a.ok(stdout, wire.RecipeAdded{Name: e.Name, Source: e.Source, Ref: e.Ref, Commit: e.Commit, Scope: s.Name}) + } + fmt.Fprintf(stdout, "%s %s (%s)\n", e.Name, e.Ref, short(e.Commit)) + return ExitOK +} + +func runRecipeLock(a *Args, stdout, stderr io.Writer) int { + s, err := recipes.ScopeFor(a.Global) + if err != nil { + return a.fail(stdout, stderr, err) + } + l, err := recipes.LockAll(s) + if err != nil { + return a.fail(stdout, stderr, recipeGitError(err, "lock")) + } + return reportLock(a, stdout, s, l) +} + +func runRecipeSync(a *Args, stdout, stderr io.Writer) int { + s, err := recipes.ScopeFor(a.Global) + if err != nil { + return a.fail(stdout, stderr, err) + } + if err := recipes.Sync(s); err != nil { + return a.fail(stdout, stderr, recipeGitError(err, "sync")) + } + l, err := recipes.ReadLock(s) + if err != nil { + return a.fail(stdout, stderr, err) + } + return reportLock(a, stdout, s, l) +} + +func reportLock(a *Args, stdout io.Writer, s recipes.Scope, l recipes.Lock) int { + rows := make([]wire.RecipeAdded, 0, len(l.Recipes)) + names := make([]string, 0, len(l.Recipes)) + for name := range l.Recipes { + names = append(names, name) + } + sort.Strings(names) + for _, name := range names { + e := l.Recipes[name] + rows = append(rows, wire.RecipeAdded{Name: name, Source: e.Source, Ref: e.Ref, Commit: e.Commit, Scope: s.Name}) + } + if a.JSON { + return a.ok(stdout, wire.RecipeBatch{Recipes: wire.NonNil(rows)}) + } + for _, row := range rows { + fmt.Fprintf(stdout, "%-20s %-12s %s\n", row.Name, row.Ref, short(row.Commit)) + } + return ExitOK +} + +func runRecipeUpdate(a *Args, stdout, stderr io.Writer) int { + s, err := recipes.ScopeFor(a.Global) + if err != nil { + return a.fail(stdout, stderr, err) + } + entries, err := recipes.Update(s, a.Names) + if err != nil { + return a.fail(stdout, stderr, recipeGitError(err, "update")) + } + rows := make([]wire.RecipeAdded, 0, len(entries)) + for _, e := range entries { + rows = append(rows, wire.RecipeAdded{Name: e.Name, Source: e.Source, Ref: e.Ref, Commit: e.Commit, Scope: s.Name}) + } + if a.JSON { + return a.ok(stdout, wire.RecipeBatch{Recipes: rows}) + } + for _, row := range rows { + fmt.Fprintf(stdout, "%-20s %-12s %s\n", row.Name, row.Ref, short(row.Commit)) + } + return ExitOK +} + +func runRecipeRM(a *Args, stdin io.Reader, stdout, stderr io.Writer) int { + name := a.Names[0] + s, err := recipes.ScopeFor(a.Global) + if err != nil { + return a.fail(stdout, stderr, err) + } + lock, err := recipes.ReadLock(s) + if err != nil { + return a.fail(stdout, stderr, err) + } + if _, ok := lock.Recipes[name]; !ok { + return a.failMsg(stdout, stderr, core.ErrNotFound, + fmt.Sprintf("%s is not a remote recipe in %s scope", name, s.Name)) + } + if !a.Force { + users, err := core.RecipeUsers(name) + if err != nil { + return a.fail(stdout, stderr, err) + } + if len(users) > 0 { + return a.failMsg(stdout, stderr, core.ErrInUse, + fmt.Sprintf("%s is used by %s; pass --force to remove it anyway", name, strings.Join(users, ", "))) + } + } + if ok, code := confirm(a, stdin, stdout, stderr, "remove recipe "+name+"?"); !ok { + return code + } + var users func() ([]string, error) + if !a.Force { + users = func() ([]string, error) { return core.RecipeUsers(name) } + } + if err := recipes.RemoveChecked(s, name, users); err != nil { + var inUse *recipes.RemoveInUse + if errors.As(err, &inUse) { + return a.failMsg(stdout, stderr, core.ErrInUse, + fmt.Sprintf("%s is used by %s; pass --force to remove it anyway", name, strings.Join(inUse.Users, ", "))) + } + return a.fail(stdout, stderr, recipeGitError(err, "remove")) + } + if a.JSON { + return a.ok(stdout, wire.RecipeRemoved{Name: name, Scope: s.Name}) + } + fmt.Fprintln(stdout, "removed", name) + return ExitOK +} + +func runRecipeSearch(a *Args, stdout, stderr io.Writer) int { + if a.Refresh { + if err := recipes.RefreshIndex(true); err != nil { + return a.fail(stdout, stderr, err) + } + } + hits, err := recipes.SearchIndex(a.Ref) + if err != nil { + return a.fail(stdout, stderr, err) + } + if a.JSON { + return a.ok(stdout, wire.RecipeSearch{Recipes: wire.FromIndexEntries(hits)}) + } + if len(hits) == 0 { + fmt.Fprintln(stdout, "no matches") + return ExitOK + } + for _, h := range hits { + fmt.Fprintf(stdout, "%-20s %s\n", h.Name, h.Description) + } + return ExitOK +} + +func recipeGitError(err error, operation string) error { + if !errors.Is(err, gitx.ErrNoGit) { + return err + } + fix := "install git" + for _, check := range hostcheck.RunChecks(hostcheck.DetectDistro()) { + if check.Name == "git" && len(check.Fix) > 0 { + fix = strings.Join(check.Fix, " && ") + break + } + } + return fmt.Errorf("%w: git is required for recipe %s; install it: %s", gitx.ErrNoGit, operation, fix) +} diff --git a/internal/cli/run_recipe_remote_test.go b/internal/cli/run_recipe_remote_test.go new file mode 100644 index 00000000..e1a2957a --- /dev/null +++ b/internal/cli/run_recipe_remote_test.go @@ -0,0 +1,531 @@ +package cli + +import ( + "bytes" + "encoding/json" + "fmt" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/novusedge/stoat/internal/cli/wire" + "github.com/novusedge/stoat/internal/config" + "github.com/novusedge/stoat/internal/recipes" + "github.com/novusedge/stoat/internal/testutil" +) + +func cliRecipeRepo(t *testing.T, name, script string) string { + t.Helper() + src := testutil.GitRepo(t, map[string]string{ + "recipe.toml": fmt.Sprintf("schema = 3\nname = %q\ndescription = %q\nos = [\"alpine\"]\nrequires = [\"git\"]\nscript = \"install.sh\"\n\n[params.channel]\ntype = \"enum\"\nvalues = [\"stable\", \"test\"]\ndefault = \"stable\"\n", name, name+" recipe"), + "install.sh": script, + }) + dst := filepath.Join(filepath.Dir(src), name+".git") + if err := os.Rename(src, dst); err != nil { + t.Fatal(err) + } + return dst +} + +func cliIndex(t *testing.T, entries string) string { + t.Helper() + index := testutil.GitRepo(t, map[string]string{"index.toml": "schema = 1\n\n" + entries}) + t.Setenv("STOAT_INDEX", index) + return index +} + +func writeCLIFile(t *testing.T, path, body string) { + t.Helper() + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(path, []byte(body), 0o644); err != nil { + t.Fatal(err) + } +} + +func TestRecipeAddAndListReportScopeAndCommit(t *testing.T) { + cliRoot(t) + t.Chdir(t.TempDir()) + src := cliRecipeRepo(t, "demo", "#!/bin/sh\nset -e\necho v1\n") + + if code, _ := runJSON(t, "recipe", "add", src, "-y", "--global"); code != ExitOK { + t.Fatalf("add exit = %d", code) + } + code, objs := runJSON(t, "recipe", "list") + if code != ExitOK { + t.Fatalf("list exit = %d", code) + } + data := dataOf(t, objs) + entries, ok := data["recipes"].([]any) + if !ok { + t.Fatalf("recipes = %#v, want an array of named entries", data["recipes"]) + } + var found map[string]any + for _, e := range entries { + m, _ := e.(map[string]any) + if m["name"] == "demo" { + found = m + } + } + if found == nil { + t.Fatalf("demo missing from %v", entries) + } + if found["scope"] != "global" { + t.Errorf("scope = %v, want global", found["scope"]) + } + if s, _ := found["commit"].(string); len(s) != 7 { + t.Errorf("commit = %v, want a 7-character sha", found["commit"]) + } +} + +func TestRecipeLockPersistsResolvedPinAndSyncCreatesCache(t *testing.T) { + cliRoot(t) + project := t.TempDir() + t.Chdir(project) + src := cliRecipeRepo(t, "demo", "#!/bin/sh\necho v1\n") + writeCLIFile(t, filepath.Join(project, "stoat.toml"), fmt.Sprintf("[recipes]\ndemo = { source = %q, ref = \"main\" }\n", src)) + + code, objs := runJSON(t, "recipe", "lock") + if code != ExitOK { + t.Fatalf("lock exit = %d: %v", code, objs) + } + rows, ok := dataOf(t, objs)["recipes"].([]any) + if !ok || len(rows) != 1 { + t.Fatalf("lock data.recipes = %#v, want one named row", dataOf(t, objs)["recipes"]) + } + row, _ := rows[0].(map[string]any) + if row["name"] != "demo" { + t.Errorf("lock row name = %v, want demo", row["name"]) + } + scope, err := recipes.ScopeFor(false) + if err != nil { + t.Fatal(err) + } + lock, err := scope.Lock() + if err != nil { + t.Fatal(err) + } + entry, ok := lock.Recipes["demo"] + if !ok || entry.Commit == "" { + t.Fatalf("persisted lock = %+v, want demo's resolved commit", lock) + } + + code, objs = runJSON(t, "recipe", "sync") + if code != ExitOK { + t.Fatalf("sync exit = %d: %v", code, objs) + } + if _, err := os.Stat(filepath.Join(scope.CachePath, "demo", "recipe.toml")); err != nil { + t.Fatalf("sync did not create the pinned cache: %v", err) + } +} + +func TestRecipeUpdateTargetedAndAllUseReturnedNames(t *testing.T) { + cliRoot(t) + t.Chdir(t.TempDir()) + demo := cliRecipeRepo(t, "demo", "#!/bin/sh\necho demo-v1\n") + other := cliRecipeRepo(t, "other", "#!/bin/sh\necho other-v1\n") + + for _, src := range []string{demo, other} { + if code, _ := runJSON(t, "recipe", "add", src, "-y", "--global"); code != ExitOK { + t.Fatalf("add %s failed", src) + } + } + scope, err := recipes.ScopeFor(true) + if err != nil { + t.Fatal(err) + } + before, err := scope.Lock() + if err != nil { + t.Fatal(err) + } + demoV2 := testutil.GitCommit(t, demo, map[string]string{"install.sh": "#!/bin/sh\necho demo-v2\n"}, "") + if code, _ := runJSON(t, "recipe", "update", "demo", "--global"); code != ExitOK { + t.Fatal("targeted update failed") + } + after, err := scope.Lock() + if err != nil { + t.Fatal(err) + } + if after.Recipes["demo"].Commit != demoV2 { + t.Fatalf("targeted update demo pin = %q, want %q", after.Recipes["demo"].Commit, demoV2) + } + if after.Recipes["other"].Commit != before.Recipes["other"].Commit { + t.Fatal("targeted update changed the unrequested recipe") + } + + otherV2 := testutil.GitCommit(t, other, map[string]string{"install.sh": "#!/bin/sh\necho other-v2\n"}, "") + if code, _ := runJSON(t, "recipe", "update", "--global"); code != ExitOK { + t.Fatal("all-recipes update failed") + } + after, err = scope.Lock() + if err != nil { + t.Fatal(err) + } + if after.Recipes["other"].Commit != otherV2 || after.Recipes["demo"].Commit != demoV2 { + t.Fatalf("all update pins = %+v, want both current commits", after.Recipes) + } +} + +func TestRecipeRMPreflightsVMUseBeforeConfirmationAndPreservesDisk(t *testing.T) { + cliRoot(t) + t.Chdir(t.TempDir()) + src := cliRecipeRepo(t, "demo", "#!/bin/sh\necho demo\n") + if code, _ := runJSON(t, "recipe", "add", src, "-y", "--global"); code != ExitOK { + t.Fatal("add failed") + } + if err := (&config.VM{Name: "work", Mode: "live", RAM: 1024, CPUs: 1, SSHPort: 2200, Recipes: []string{"demo"}}).Save(); err != nil { + t.Fatal(err) + } + if err := (&config.VM{Name: "other", Mode: "live", RAM: 1024, CPUs: 1, SSHPort: 2201, Recipes: []string{"demo"}}).Save(); err != nil { + t.Fatal(err) + } + scope, err := recipes.ScopeFor(true) + if err != nil { + t.Fatal(err) + } + lockBefore, err := os.ReadFile(scope.LockPath) + if err != nil { + t.Fatal(err) + } + vmBefore, err := os.ReadFile(filepath.Join(config.Root(), "work", "vm.toml")) + if err != nil { + t.Fatal(err) + } + cachePath := filepath.Join(scope.CachePath, "demo") + if _, err := os.Stat(filepath.Join(cachePath, "recipe.toml")); err != nil { + t.Fatalf("add did not create recipe cache: %v", err) + } + + assertPreserved := func(label string) { + t.Helper() + lockAfter, readErr := os.ReadFile(scope.LockPath) + if readErr != nil || string(lockAfter) != string(lockBefore) { + t.Fatalf("%s changed lock: %v", label, readErr) + } + vmAfter, readErr := os.ReadFile(filepath.Join(config.Root(), "work", "vm.toml")) + if readErr != nil || string(vmAfter) != string(vmBefore) { + t.Fatalf("%s changed VM declaration: %v", label, readErr) + } + if _, readErr := os.Stat(filepath.Join(cachePath, "recipe.toml")); readErr != nil { + t.Fatalf("%s removed recipe cache: %v", label, readErr) + } + } + + code, objs := runJSON(t, "recipe", "rm", "demo", "--global") + errObj := result(t, objs)["error"].(map[string]any) + if code != ExitFail || errObj["code"] != string(wire.CodeInUse) { + t.Fatalf("in-use rm without -y = %d, %v; want in_use before confirmation", code, objs) + } + if message, _ := errObj["message"].(string); !strings.Contains(message, "work") || !strings.Contains(message, "other") { + t.Fatalf("in-use message = %q, want every VM name", message) + } + assertPreserved("in-use rm without -y") + + code, objs = runJSON(t, "recipe", "rm", "demo", "--global", "-y") + errObj = result(t, objs)["error"].(map[string]any) + if code != ExitFail || errObj["code"] != string(wire.CodeInUse) { + t.Fatalf("in-use rm with -y = %d, %v; want in_use", code, objs) + } + assertPreserved("in-use rm with -y") + + code, objs = runJSON(t, "recipe", "rm", "demo", "--global", "--force") + if code != ExitFail || result(t, objs)["error"].(map[string]any)["code"] != string(wire.CodeConfirmationRequired) { + t.Fatalf("--force without -y = %d, %v; want confirmation_required", code, objs) + } + assertPreserved("forced rm without -y") + if code, _ = runJSON(t, "recipe", "rm", "demo", "--global", "--force", "-y"); code != ExitOK { + t.Fatal("--force -y should remove a recipe used by a VM") + } + lock, err := scope.Lock() + if err != nil { + t.Fatal(err) + } + if _, ok := lock.Recipes["demo"]; ok { + t.Fatal("forced removal left the lock entry") + } + if _, err := os.Stat(cachePath); !os.IsNotExist(err) { + t.Fatalf("forced removal left recipe cache: %v", err) + } + + free := cliRecipeRepo(t, "free", "#!/bin/sh\necho free\n") + if code, _ := runJSON(t, "recipe", "add", free, "-y", "--global"); code != ExitOK { + t.Fatal("free recipe add failed") + } + code, objs = runJSON(t, "recipe", "rm", "free", "--global") + if code != ExitFail || result(t, objs)["error"].(map[string]any)["code"] != string(wire.CodeConfirmationRequired) { + t.Fatalf("removable rm without -y = %d, %v; want confirmation_required", code, objs) + } +} + +func TestRecipeListShowsGlobalPinFromProjectScope(t *testing.T) { + cliRoot(t) + globalDir := t.TempDir() + t.Chdir(globalDir) + src := cliRecipeRepo(t, "demo", "#!/bin/sh\necho demo\n") + if code, _ := runJSON(t, "recipe", "add", src, "-y", "--global"); code != ExitOK { + t.Fatal("global add failed") + } + project := t.TempDir() + writeCLIFile(t, filepath.Join(project, "stoat.toml"), "[recipes]\n") + t.Chdir(project) + code, objs := runJSON(t, "recipe", "list") + if code != ExitOK { + t.Fatalf("project list exit = %d", code) + } + for _, raw := range dataOf(t, objs)["recipes"].([]any) { + row := raw.(map[string]any) + if row["name"] == "demo" { + if row["scope"] != "global" || row["source"] != src || row["commit"] == "" { + t.Fatalf("global row lost pin metadata in project scope: %v", row) + } + return + } + } + t.Fatal("project recipe list omitted the visible global recipe") +} + +func TestRecipeURLJSONRefusalHasOneTypedEnvelopeWithoutPreviewProse(t *testing.T) { + cliRoot(t) + t.Chdir(t.TempDir()) + src := cliRecipeRepo(t, "demo", "#!/bin/sh\necho demo\n") + var out, errOut bytes.Buffer + code := Main([]string{"--json", "recipe", "add", src, "--global"}, "test", strings.NewReader("y\n"), &out, &errOut) + if code != ExitFail { + t.Fatalf("URL add without -y exit = %d, want ExitFail", code) + } + lines := strings.Split(strings.TrimSpace(out.String()), "\n") + if len(lines) != 1 { + t.Fatalf("URL refusal wrote %d lines: %q", len(lines), out.String()) + } + var envelope map[string]any + if err := json.Unmarshal([]byte(lines[0]), &envelope); err != nil { + t.Fatalf("URL refusal line is not JSON: %v", err) + } + errObj := envelope["error"].(map[string]any) + if errObj["code"] != string(wire.CodeConfirmationRequired) { + t.Errorf("error.code = %v, want %q", errObj["code"], wire.CodeConfirmationRequired) + } + if strings.Contains(out.String(), "os:") || strings.Contains(out.String(), "requires:") { + t.Errorf("JSON refusal leaked preview prose: %q", out.String()) + } + if errOut.Len() != 0 { + t.Errorf("stderr = %q, want empty under --json", errOut.String()) + } +} + +func TestRecipeURLNonTTYDoesNotPromptOrMutate(t *testing.T) { + cliRoot(t) + t.Chdir(t.TempDir()) + src := cliRecipeRepo(t, "demo", "#!/bin/sh\necho demo\n") + var out, errOut bytes.Buffer + code := Main([]string{"recipe", "add", src, "--global"}, "test", strings.NewReader("y\n"), &out, &errOut) + if code != ExitFail { + t.Fatalf("URL add on a non-TTY exit = %d, want ExitFail", code) + } + if strings.Contains(out.String(), "demo") || strings.Contains(out.String(), "os:") { + t.Fatalf("non-TTY add printed preview/prompt prose: %q", out.String()) + } + scope, err := recipes.ScopeFor(true) + if err != nil { + t.Fatal(err) + } + lock, err := scope.Lock() + if err != nil { + t.Fatal(err) + } + if len(lock.Recipes) != 0 { + t.Fatalf("non-TTY refusal mutated the lock: %+v", lock.Recipes) + } +} + +func TestRecipeAddIndexNameSkipsConfirmation(t *testing.T) { + cliRoot(t) + t.Chdir(t.TempDir()) + src := cliRecipeRepo(t, "demo", "#!/bin/sh\necho demo\n") + cliIndex(t, fmt.Sprintf("[recipes.demo]\nsource = %q\ndescription = \"demo\"\nos = [\"alpine\"]\n", src)) + if code, objs := runJSON(t, "recipe", "add", "demo", "--global"); code != ExitOK { + t.Fatalf("index-name add without -y exit = %d: %v", code, objs) + } +} + +func TestRecipeVersionAndEmptyRemoteListsUseContractThreeAndArrays(t *testing.T) { + cliRoot(t) + t.Chdir(t.TempDir()) + cliIndex(t, "") + code, objs := runJSON(t, "version") + if code != ExitOK { + t.Fatal("version failed") + } + version := dataOf(t, objs)["contract"] + if version != float64(3) { + t.Fatalf("version contract = %v, want 3", version) + } + for _, argv := range [][]string{ + {"recipe", "search", "no-match"}, + {"recipe", "lock", "--global"}, + {"recipe", "sync", "--global"}, + } { + code, objs = runJSON(t, argv...) + if code != ExitOK { + t.Fatalf("%v exit = %d: %v", argv, code, objs) + } + if recipes, ok := dataOf(t, objs)["recipes"].([]any); !ok || recipes == nil { + t.Fatalf("%v data.recipes = %#v, want non-null array", argv, dataOf(t, objs)["recipes"]) + } + } +} + +func TestRecipeSearchLiteralLeadingOptionsReachTheRealIndexQuery(t *testing.T) { + cases := []struct { + term string + name string + }{ + {term: "--json", name: "match-json"}, + {term: "--refresh", name: "match-refresh"}, + {term: "--", name: "match-terminator"}, + } + for _, tc := range cases { + t.Run(tc.term, func(t *testing.T) { + cliRoot(t) + t.Chdir(t.TempDir()) + cliIndex(t, fmt.Sprintf( + "[recipes.%s]\nsource = \"local\"\ndescription = %q\nos = [\"alpine\"]\n\n[recipes.other]\nsource = \"local\"\ndescription = \"not this term\"\nos = [\"alpine\"]\n", + tc.name, tc.term, + )) + + code, objs := runJSON(t, "recipe", "search", "--", tc.term) + if code != ExitOK { + t.Fatalf("search %q exit = %d: %v", tc.term, code, objs) + } + rows, _ := dataOf(t, objs)["recipes"].([]any) + if len(rows) != 1 { + t.Fatalf("search %q returned %d rows %v, want only its matching fixture", tc.term, len(rows), rows) + } + row, _ := rows[0].(map[string]any) + if row["name"] != tc.name || row["description"] != tc.term { + t.Fatalf("search %q row = %v, want %q/%q", tc.term, row, tc.name, tc.term) + } + }) + } +} + +func TestRecipeJSONRemoteResultShapesUseFullPinsAndMinimalRemoval(t *testing.T) { + t.Run("empty batch", func(t *testing.T) { + cliRoot(t) + t.Chdir(t.TempDir()) + code, objs := runJSON(t, "recipe", "update", "--global") + if code != ExitOK { + t.Fatalf("empty update exit = %d: %v", code, objs) + } + data := dataOf(t, objs) + if len(data) != 1 { + t.Fatalf("empty update data = %v, want exactly recipes", data) + } + rows, ok := data["recipes"].([]any) + if !ok || rows == nil || len(rows) != 0 { + t.Fatalf("empty update recipes = %#v, want []", data["recipes"]) + } + }) + + t.Run("add list update remove", func(t *testing.T) { + cliRoot(t) + t.Chdir(t.TempDir()) + src := cliRecipeRepo(t, "demo", "#!/bin/sh\necho v1\n") + + code, objs := runJSON(t, "recipe", "add", src, "-y", "--global") + if code != ExitOK { + t.Fatalf("add exit = %d: %v", code, objs) + } + add := dataOf(t, objs) + if got := len(add); got != 5 { + t.Fatalf("add data has %d fields (%v), want name/source/ref/commit/scope", got, add) + } + for _, key := range []string{"name", "source", "ref", "commit", "scope"} { + if _, ok := add[key]; !ok { + t.Errorf("add data omitted %q: %v", key, add) + } + } + if add["name"] != "demo" || add["source"] != src || add["scope"] != "global" { + t.Errorf("add metadata = %v", add) + } + addCommit, _ := add["commit"].(string) + if len(addCommit) != 40 { + t.Fatalf("add commit = %q, want full commit", addCommit) + } + + code, objs = runJSON(t, "recipe", "list") + if code != ExitOK { + t.Fatalf("list exit = %d: %v", code, objs) + } + list := dataOf(t, objs) + rows, _ := list["recipes"].([]any) + var row map[string]any + for _, raw := range rows { + candidate, _ := raw.(map[string]any) + if candidate["name"] == "demo" { + row = candidate + break + } + } + if row == nil { + t.Fatalf("list recipes omitted demo: %#v", list["recipes"]) + } + if len(row) != 6 || row["name"] != "demo" || row["description"] != "demo recipe" || row["source"] != src || row["scope"] != "global" { + t.Fatalf("list row = %v, want named pin metadata", row) + } + listCommit, _ := row["commit"].(string) + if len(listCommit) != 7 || listCommit != addCommit[:7] { + t.Fatalf("list commit = %q, want seven-character prefix %q", listCommit, addCommit[:7]) + } + + v2 := testutil.GitCommit(t, src, map[string]string{"install.sh": "#!/bin/sh\necho v2\n"}, "") + code, objs = runJSON(t, "recipe", "update", "demo", "--global") + if code != ExitOK { + t.Fatalf("update exit = %d: %v", code, objs) + } + update := dataOf(t, objs) + updateRows, _ := update["recipes"].([]any) + if len(updateRows) != 1 { + t.Fatalf("update rows = %#v, want one row", update["recipes"]) + } + updated, _ := updateRows[0].(map[string]any) + updatedCommit, _ := updated["commit"].(string) + if len(updatedCommit) != 40 || updatedCommit != v2 { + t.Fatalf("update commit = %q, want full %q", updatedCommit, v2) + } + + code, objs = runJSON(t, "recipe", "rm", "demo", "--global", "-y") + if code != ExitOK { + t.Fatalf("remove exit = %d: %v", code, objs) + } + removed := dataOf(t, objs) + if len(removed) != 2 || removed["name"] != "demo" || removed["scope"] != "global" { + t.Fatalf("remove data = %v, want exactly name and scope", removed) + } + for _, forbidden := range []string{"source", "ref", "commit"} { + if _, present := removed[forbidden]; present { + t.Errorf("remove data exposed empty %q field: %v", forbidden, removed) + } + } + }) +} + +func TestRecipeApplyStaleProjectLockUsesTypedRepairCode(t *testing.T) { + cliRoot(t) + project := t.TempDir() + t.Chdir(project) + writeCLIFile(t, filepath.Join(project, "stoat.toml"), "[recipes]\ndemo = \"main\"\n") + if err := (&config.VM{Name: "work", Mode: "live", OS: "alpine", RAM: 1024, CPUs: 1, SSHPort: 2200, Recipes: []string{"demo"}}).Save(); err != nil { + t.Fatal(err) + } + code, objs := runJSON(t, "apply", "work", "--dry-run") + if code != ExitFail { + t.Fatalf("stale apply exit = %d, want ExitFail: %v", code, objs) + } + errObj := result(t, objs)["error"].(map[string]any) + if errObj["code"] != string(wire.CodeLockOutOfDate) { + t.Fatalf("stale apply error.code = %v, want %q", errObj["code"], wire.CodeLockOutOfDate) + } +} diff --git a/internal/cli/run_state.go b/internal/cli/run_state.go index bb5f3434..78fff5bc 100644 --- a/internal/cli/run_state.go +++ b/internal/cli/run_state.go @@ -206,17 +206,18 @@ func runSnapshot(a *Args, stdout, stderr io.Writer) int { // pre-install checklist runs (qemu-system-x86_64, qemu-img, ssh, xorriso, // /dev/kvm), so `stoat doctor` and `just setup` agree on host readiness. // -// It prints the fix command when there is one, so a failed check tells the -// user how to repair it instead of leaving them to guess. +// It prints every failed check's fix command, including optional dependencies. +// Optional checks are warnings only; required failures remain FAIL and make +// the aggregate doctor result unhealthy. func runDoctor(a *Args, stdout, stderr io.Writer) int { checks := core.Doctor() - var failed []core.HostCheck - for _, c := range checks { - if !c.OK { - failed = append(failed, c) - } - } if a.JSON { + var failed []core.HostCheck + for _, c := range checks { + if !c.OK && !c.Optional { + failed = append(failed, c) + } + } // healthy, not ok: the envelope already owns "ok", and two // differently-scoped ok fields one level apart is a trap. Exit is 0 // even when the host is unhealthy (§5): doctor SUCCEEDED at checking, @@ -226,15 +227,27 @@ func runDoctor(a *Args, stdout, stderr io.Writer) int { "checks": wire.FromHostChecks(checks), }) } - if len(failed) == 0 { - fmt.Fprintln(stdout, "ok") - return ExitOK - } - for _, c := range failed { - fmt.Fprintf(stdout, "FAIL: %s: %s\n", c.Name, c.Detail) + failedRequired := false + for _, c := range checks { + if c.OK { + continue + } + label := c.Name + prefix := "FAIL" + if c.Optional { + label += " (optional)" + prefix = "WARN" + } else { + failedRequired = true + } + fmt.Fprintf(stdout, "%s: %s: %s\n", prefix, label, c.Detail) if len(c.Fix) > 0 { fmt.Fprintf(stdout, " try: %s\n", strings.Join(c.Fix, " ")) } } + if !failedRequired { + fmt.Fprintln(stdout, "ok") + return ExitOK + } return ExitFail } diff --git a/internal/cli/run_vm.go b/internal/cli/run_vm.go index 649c4179..56619d35 100644 --- a/internal/cli/run_vm.go +++ b/internal/cli/run_vm.go @@ -1,7 +1,6 @@ package cli import ( - "bufio" "context" "errors" "fmt" @@ -246,22 +245,8 @@ func runRM(a *Args, stdin io.Reader, stdout, stderr io.Writer) int { if v.State == core.StateRunning { return a.failMsg(stdout, stderr, core.ErrAlreadyRunning, a.VM+" is running; stop it first") } - if !a.Yes { - if a.JSON { - // --json never prompts (§1): an enforcement boundary a process can - // cross by answering a prompt is not a boundary. - return a.fail(stdout, stderr, fmt.Errorf("%w: %s", wire.ErrConfirmationRequired, a.VM)) - } - if a.Quiet { - fmt.Fprintln(stderr, "stoat: rm: refusing to delete without -y in non-interactive mode") - return ExitFail - } - fmt.Fprintf(stdout, "delete VM %s? [y/N] ", a.VM) - line, _ := bufio.NewReader(stdin).ReadString('\n') - if strings.ToLower(strings.TrimSpace(line)) != "y" { - fmt.Fprintln(stdout, "aborted") - return ExitFail - } + if ok, code := confirm(a, stdin, stdout, stderr, "delete VM "+a.VM+"?"); !ok { + return code } if err := core.Destroy(a.VM); err != nil { // Same race as runDown: the State check above refuses a running VM diff --git a/internal/cli/wire/dto.go b/internal/cli/wire/dto.go index 5eb87ba3..10b979bd 100644 --- a/internal/cli/wire/dto.go +++ b/internal/cli/wire/dto.go @@ -8,6 +8,7 @@ import ( "github.com/novusedge/stoat/internal/core" "github.com/novusedge/stoat/internal/guest" + "github.com/novusedge/stoat/internal/recipes" ) // DTOs, not json tags on core types (§3.1). Reasons, in order of weight: @@ -33,6 +34,10 @@ import ( // nonNil turns a nil slice into an empty, non-nil one so it marshals as [] // rather than null (§3.1, §6: "MAY rely on [] for an empty list, never // null"). +// NonNil is nonNil, exported so cli callers building a wire slice by hand +// (not through a From* constructor) route through the same guarantee. +func NonNil[T any](s []T) []T { return nonNil(s) } + func nonNil[T any](s []T) []T { if s == nil { return []T{} @@ -287,14 +292,15 @@ func FromSnapshots(ss []core.Snapshot) []Snapshot { // HostCheck is core.HostCheck for the wire. type HostCheck struct { - Name string `json:"name"` - OK bool `json:"ok"` - Detail string `json:"detail"` - Fix []string `json:"fix"` + Name string `json:"name"` + OK bool `json:"ok"` + Detail string `json:"detail"` + Fix []string `json:"fix"` + Optional bool `json:"optional"` } func FromHostCheck(c core.HostCheck) HostCheck { - return HostCheck{Name: c.Name, OK: c.OK, Detail: c.Detail, Fix: nonNil(c.Fix)} + return HostCheck{Name: c.Name, OK: c.OK, Detail: c.Detail, Fix: nonNil(c.Fix), Optional: c.Optional} } func FromHostChecks(cs []core.HostCheck) []HostCheck { @@ -305,6 +311,75 @@ func FromHostChecks(cs []core.HostCheck) []HostCheck { return nonNil(out) } +// RecipeEntry is one row of `recipe list`: the recipe, its source scope, and +// the commit pinned by that scope. +type RecipeEntry struct { + Name string `json:"name"` + Description string `json:"description"` + Scope string `json:"scope"` + Source string `json:"source"` + Ref string `json:"ref"` + Commit string `json:"commit"` +} + +// IndexEntry is one result of `recipe search`. +type IndexEntry struct { + Name string `json:"name"` + Source string `json:"source"` + Description string `json:"description"` + OS []string `json:"os"` +} + +func FromIndexEntry(e recipes.IndexEntry) IndexEntry { + return IndexEntry{Name: e.Name, Source: e.Source, Description: e.Description, OS: nonNil(e.OS)} +} + +func FromIndexEntries(es []recipes.IndexEntry) []IndexEntry { + out := make([]IndexEntry, len(es)) + for i, e := range es { + out[i] = FromIndexEntry(e) + } + return nonNil(out) +} + +// RecipeList is the data of `recipe list`. +type RecipeList struct { + Roots []RecipeRoot `json:"roots"` + Recipes []RecipeEntry `json:"recipes"` +} + +// RecipeRoot is one recipe search root and its scope label. +type RecipeRoot struct { + Path string `json:"path"` + Scope string `json:"scope"` +} + +// RecipeAdded is one resolved remote recipe pin. +type RecipeAdded struct { + Name string `json:"name"` + Source string `json:"source"` + Ref string `json:"ref"` + Commit string `json:"commit"` + Scope string `json:"scope"` +} + +// RecipeRemoved is the minimal result of `recipe rm`: removal has no pin +// metadata to report after the lock and cache are gone. +type RecipeRemoved struct { + Name string `json:"name"` + Scope string `json:"scope"` +} + +// RecipeBatch is the data returned by lock, sync, and update. +type RecipeBatch struct { + Recipes []RecipeAdded `json:"recipes"` +} + +// RecipeSearch is the data returned by recipe search. +type RecipeSearch struct { + Recipes []IndexEntry `json:"recipes"` +} + // PruneItem is core.PruneItem for the wire. Class is already the wire value // on core.PruneItem itself, so this constructor translates nothing; it // exists so a field added to core.PruneItem does not silently reach the diff --git a/internal/cli/wire/dto_test.go b/internal/cli/wire/dto_test.go index 384915d8..5c65b80d 100644 --- a/internal/cli/wire/dto_test.go +++ b/internal/cli/wire/dto_test.go @@ -112,7 +112,16 @@ func TestPruneItemGolden(t *testing.T) { func TestHostCheckGolden(t *testing.T) { c := core.HostCheck{Name: "qemu-img", OK: false, Detail: "not found", Fix: []string{"sudo", "pacman", "-S", "qemu-img"}} got := marshal(t, FromHostCheck(c)) - want := `{"name":"qemu-img","ok":false,"detail":"not found","fix":["sudo","pacman","-S","qemu-img"]}` + want := `{"name":"qemu-img","ok":false,"detail":"not found","fix":["sudo","pacman","-S","qemu-img"],"optional":false}` + if got != want { + t.Errorf("got %s\nwant %s", got, want) + } +} + +func TestHostCheckOptionalGolden(t *testing.T) { + c := core.HostCheck{Name: "git", Detail: "not found", Fix: []string{"sudo", "pacman", "-S", "git"}, Optional: true} + got := marshal(t, FromHostCheck(c)) + want := `{"name":"git","ok":false,"detail":"not found","fix":["sudo","pacman","-S","git"],"optional":true}` if got != want { t.Errorf("got %s\nwant %s", got, want) } diff --git a/internal/cli/wire/envelope.go b/internal/cli/wire/envelope.go index a4a95d49..62715b84 100644 --- a/internal/cli/wire/envelope.go +++ b/internal/cli/wire/envelope.go @@ -25,7 +25,9 @@ import ( // label, target_os and shared. Three fields deleted is exactly the case this // number exists for. There is no v1 compatibility path anywhere: a clean // break, so nothing has to reason about which shape it is looking at. -const ContractVersion = 2 +// v3: recipe list changed shape. dir became roots, a list of {path, scope}, +// and recipes became a list of RecipeEntry objects rather than names. +const ContractVersion = 3 // Event types (§2, §4). "result" is the one terminal type; every other type // is non-terminal and a consumer MUST ignore any type it does not recognize. @@ -134,10 +136,13 @@ func (e *Emitter) write(env envelope) error { // usage error (an unknown subcommand, a bad flag, before any FlagSet // exists) can still produce an envelope. // -// The one hard case is "exec": it parses no flags of its own (cli.go's -// runExec dispatch), because everything after the VM name is the guest's -// command, verbatim. So "--json" is recognized anywhere in argv EXCEPT -// after exec's VM name: +// The hard cases are "exec" and the conventional positional terminator: +// exec parses no flags of its own (cli.go's runExec dispatch), because +// everything after the VM name is the guest's command, verbatim. The +// terminator likewise makes every following token positional, so a search for +// the literal "--json" must not lose that token while looking for stoat's +// global JSON flag. Thus "--json" is recognized anywhere in argv EXCEPT +// after exec's VM name or after "--": // // stoat --json exec work ls -la -> stoat // stoat exec --json work ls -la -> stoat @@ -149,7 +154,8 @@ func SplitJSONFlag(argv []string) (jsonMode bool, rest []string) { // exec's own two positionals are argv[0] ("exec") and argv[1] (the VM // name); scanning stops consuming --json after that point, so anything // from argv[2] on is the guest's command and passes through untouched, - // --json included. + // --json included. For every command, the first -- also ends this scan: + // everything after it is positional data for the command. stop := len(argv) if len(argv) > 0 && argv[0] == "exec" { stop = 2 @@ -160,6 +166,10 @@ func SplitJSONFlag(argv []string) (jsonMode bool, rest []string) { rest = make([]string, 0, len(argv)) for i, a := range argv { + if i < stop && a == "--" { + rest = append(rest, argv[i:]...) + break + } if i < stop && a == "--json" { jsonMode = true continue diff --git a/internal/cli/wire/errors.go b/internal/cli/wire/errors.go index d3853ece..1c4b9989 100644 --- a/internal/cli/wire/errors.go +++ b/internal/cli/wire/errors.go @@ -6,8 +6,10 @@ import ( "slices" "github.com/novusedge/stoat/internal/core" + "github.com/novusedge/stoat/internal/gitx" "github.com/novusedge/stoat/internal/iso" "github.com/novusedge/stoat/internal/qemu" + "github.com/novusedge/stoat/internal/recipes" ) // Code is a stable, machine-readable error code. Codes are only ever ADDED: @@ -25,6 +27,8 @@ const ( CodeBroken Code = "broken" CodeNameTaken Code = "name_taken" CodeInvalidSpec Code = "invalid_spec" + CodeInUse Code = "in_use" + CodeGitRequired Code = "git_required" CodeImageNotDownloaded Code = "image_not_downloaded" CodeRecipeNotApplicable Code = "recipe_not_applicable" CodeNotRunning Code = "not_running" @@ -55,6 +59,8 @@ const ( CodeNoSuchImage Code = "no_such_image" CodeScreenshotFailed Code = "screenshot_failed" + // CodeLockOutOfDate identifies a repairable project-lock condition. + CodeLockOutOfDate Code = "lock_out_of_date" ) // Codes returns every declared code, sorted. Built from the same string @@ -63,7 +69,7 @@ const ( func Codes() []Code { out := []Code{ CodeNotFound, CodeBroken, CodeNameTaken, CodeInvalidSpec, - CodeImageNotDownloaded, CodeRecipeNotApplicable, CodeNotRunning, + CodeImageNotDownloaded, CodeRecipeNotApplicable, CodeInUse, CodeGitRequired, CodeNotRunning, CodeAlreadyRunning, CodeNoDisk, CodeImmutableField, CodeDiskShrink, CodeCannotReach, CodeUnknownLog, CodeTimeout, CodeCanceled, CodeUsage, CodeConfirmationRequired, CodeInternal, @@ -73,6 +79,7 @@ func Codes() []Code { CodeDownloadFailed, CodeDownloadStalled, CodeChecksumMismatch, CodeNoSuchImage, CodeScreenshotFailed, + CodeLockOutOfDate, } slices.Sort(out) return out @@ -101,6 +108,9 @@ var codeTable = []struct { {CodeBroken, core.ErrBroken}, {CodeNameTaken, core.ErrNameTaken}, {CodeInvalidSpec, core.ErrInvalidSpec}, + {CodeInvalidSpec, recipes.ErrInvalidTree}, + {CodeInUse, core.ErrInUse}, + {CodeGitRequired, gitx.ErrNoGit}, {CodeImageNotDownloaded, core.ErrImageNotDownloaded}, {CodeRecipeNotApplicable, core.ErrRecipeNotApplicable}, {CodeNotRunning, core.ErrNotRunning}, @@ -128,6 +138,7 @@ var codeTable = []struct { {CodeNoSuchImage, iso.ErrNoSuchImage}, {CodeScreenshotFailed, qemu.ErrScreenshotFailed}, {CodeNotRunning, qemu.ErrNotRunning}, + {CodeLockOutOfDate, core.ErrLockOutOfDate}, } // MapError converts a core (or context) error into an ErrorInfo, walking diff --git a/internal/cli/wire/errors_test.go b/internal/cli/wire/errors_test.go index 3dd6e6d0..54b2d4ea 100644 --- a/internal/cli/wire/errors_test.go +++ b/internal/cli/wire/errors_test.go @@ -1,7 +1,9 @@ package wire import ( + "bytes" "context" + "encoding/json" "errors" "fmt" "regexp" @@ -11,6 +13,7 @@ import ( "github.com/novusedge/stoat/internal/core" "github.com/novusedge/stoat/internal/iso" "github.com/novusedge/stoat/internal/qemu" + "github.com/novusedge/stoat/internal/recipes" ) func TestMapErrorEveryCoreSentinel(t *testing.T) { @@ -51,6 +54,76 @@ func TestMapErrorEveryCoreSentinel(t *testing.T) { } } +func TestMapErrorLockOutOfDateUsesADedicatedCode(t *testing.T) { + err := fmt.Errorf("%w; run stoat recipe lock", core.ErrLockOutOfDate) + got := MapError(err) + if got == nil { + t.Fatal("MapError returned nil for ErrLockOutOfDate") + } + if got.Code != CodeLockOutOfDate { + t.Errorf("code = %q, want %q", got.Code, CodeLockOutOfDate) + } + if got.Message != err.Error() { + t.Errorf("message = %q, want %q", got.Message, err.Error()) + } +} + +func TestLockOutOfDateCodeSurvivesPublicJSONResultBoundary(t *testing.T) { + var output bytes.Buffer + err := fmt.Errorf("%w: project recipes are stale", core.ErrLockOutOfDate) + if writeErr := NewEmitter(&output).ResultErr("recipe sync", MapError(err)); writeErr != nil { + t.Fatal(writeErr) + } + var result struct { + Type string `json:"type"` + OK bool `json:"ok"` + Error struct { + Code Code `json:"code"` + Message string `json:"message"` + } `json:"error"` + } + if decodeErr := json.Unmarshal(bytes.TrimSpace(output.Bytes()), &result); decodeErr != nil { + t.Fatal(decodeErr) + } + if result.Type != TypeResult || result.OK { + t.Fatalf("result boundary = type %q, ok %v; want result, false", result.Type, result.OK) + } + if result.Error.Code != CodeLockOutOfDate || result.Error.Message != err.Error() { + t.Fatalf("JSON error = %+v, want code %q and original message", result.Error, CodeLockOutOfDate) + } +} + +// TestRejectedRecipeTreeSurvivesTheJSONResultBoundary pins the envelope a +// user sees for a mismatched recipe.toml: invalid_spec, not the internal +// escape hatch (recipes.ErrInvalidTree previously carried no sentinel). +func TestRejectedRecipeTreeSurvivesTheJSONResultBoundary(t *testing.T) { + err := fmt.Errorf("%w: source.q494eV: recipe.toml is named %q", recipes.ErrInvalidTree, "tailscale") + var output bytes.Buffer + if writeErr := NewEmitter(&output).ResultErr("recipe", MapError(err)); writeErr != nil { + t.Fatal(writeErr) + } + var result struct { + Type string `json:"type"` + OK bool `json:"ok"` + Error struct { + Code Code `json:"code"` + Message string `json:"message"` + } `json:"error"` + } + if decodeErr := json.Unmarshal(bytes.TrimSpace(output.Bytes()), &result); decodeErr != nil { + t.Fatal(decodeErr) + } + if result.OK { + t.Fatal("result.ok = true, want false") + } + if result.Error.Code != CodeInvalidSpec { + t.Fatalf("error.code = %q, want %q", result.Error.Code, CodeInvalidSpec) + } + if result.Error.Message != err.Error() { + t.Fatalf("error.message = %q, want %q", result.Error.Message, err.Error()) + } +} + func TestMapErrorUnrecognizedFallsBackToInternal(t *testing.T) { got := MapError(errors.New("qemu-img: some unrelated failure")) if got.Code != CodeInternal { diff --git a/internal/config/config.go b/internal/config/config.go index 1d33189d..d839c804 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -177,13 +177,16 @@ func EnsureRoot() error { // reserved reports whether a directory in the data root is stoat's own rather // than a VM. Everything not listed here is treated as a VM directory, so a new -// one must be added or it gets scanned as a candidate VM. +// one must be added or it gets scanned as a candidate VM. Index refreshes use +// the two hidden workspace prefixes below; their contents can include vm.toml +// files copied from a previously published index. func reserved(name string) bool { switch name { - case "isos", "recipes", "shared", "logs": + case "isos", "recipes", "shared", "logs", "index": return true } - return false + return strings.HasPrefix(name, ".stoat-index-stage-") || + strings.HasPrefix(name, ".stoat-index-old-") } // Expand resolves a leading ~ against the user's home directory. Exported so diff --git a/internal/core/apply.go b/internal/core/apply.go index dceefd79..ced2a87c 100644 --- a/internal/core/apply.go +++ b/internal/core/apply.go @@ -230,7 +230,7 @@ func applyLocked(ctx context.Context, v *config.VM, opts ApplyOpts) error { if err != nil { return err } - scriptHash, err := recipes.ScriptHash(name, v.OS) + scriptHash, err := m.ScriptHash(v.OS) if err != nil { return err } @@ -371,9 +371,9 @@ func discoverCloudInitApplied(ctx context.Context, v *config.VM) ([]string, erro } hash, hashErr := recipeHashFor(v, m) if hashErr != nil { - hash, _ = recipes.ScriptHash(name, v.OS) + hash, _ = m.ScriptHash(v.OS) } - scriptHash, _ := recipes.ScriptHash(name, v.OS) + scriptHash, _ := m.ScriptHash(v.OS) values, undeclared := sshx.ParseOutputs(m.Outputs, redactCloudSecrets(body, secrets[name])) for _, output := range undeclared { warnings = append(warnings, fmt.Sprintf("%s: output %q is not declared", name, output)) @@ -490,12 +490,20 @@ type recipeDecision struct { // reports every decision). It raises the same errors either way: a missing // recipe.toml, a dependency cycle, or an unsatisfiable dependency. func planRecipes(v *config.VM, targets []string, explicit map[string]bool) ([]recipeDecision, map[string]recipes.Manifest, error) { + snapshot, err := recipes.RepairSnapshot() + if err != nil { + if errors.Is(err, recipes.ErrLockOutOfDate) { + return nil, nil, fmt.Errorf("%w; run stoat recipe lock", ErrLockOutOfDate) + } + return nil, nil, err + } manifests := make(map[string]recipes.Manifest, len(targets)) + byName := make(map[string]recipes.Manifest, len(snapshot.Manifests)) + for _, manifest := range snapshot.Manifests { + byName[manifest.Name] = manifest + } for _, name := range targets { - m, ok, err := recipes.ManifestFor(name) - if err != nil { - return nil, nil, err - } + m, ok := byName[name] if !ok { return nil, nil, fmt.Errorf("%w: recipe %q has no recipe.toml", ErrRecipeNotApplicable, name) } @@ -561,7 +569,7 @@ func planRecipes(v *config.VM, targets []string, explicit map[string]bool) ([]re // stale keys in secrets.toml do not. func recipeHashFor(v *config.VM, m recipes.Manifest) (string, error) { if len(m.Params) == 0 { - return recipes.ScriptHash(m.Name, v.OS) + return m.ScriptHash(v.OS) } secrets, err := config.LoadSecrets(v.Dir) if err != nil { @@ -596,7 +604,7 @@ func scriptUnchanged(v *config.VM, m recipes.Manifest, applied config.AppliedRec if applied.ScriptHash == "" { return false } - body, err := recipes.ScriptHash(m.Name, v.OS) + body, err := m.ScriptHash(v.OS) return err == nil && body == applied.ScriptHash } diff --git a/internal/core/core.go b/internal/core/core.go index bf1b6948..63bbb2fe 100644 --- a/internal/core/core.go +++ b/internal/core/core.go @@ -30,6 +30,7 @@ var ( ErrNameTaken = errors.New("name already taken") ErrImageNotDownloaded = errors.New("image not downloaded") ErrInvalidSpec = errors.New("invalid spec") + ErrInUse = errors.New("in use") // ErrRecipeNotApplicable: a recipe was named that this VM's OS and // backend cannot run. Typed because a caller retrying with a corrected // name needs to tell this apart from a malformed spec. diff --git a/internal/core/doctor.go b/internal/core/doctor.go index 3bf9a5d8..3bb631fc 100644 --- a/internal/core/doctor.go +++ b/internal/core/doctor.go @@ -12,6 +12,8 @@ type HostCheck struct { OK bool Detail string // "/usr/bin", "not found", "permission denied" Fix []string // shell commands, already distro-resolved; empty when OK + // Optional marks a binary some commands need; a missing one is not a broken host. + Optional bool } // Doctor probes every host dependency and reports it as data, printing @@ -25,7 +27,7 @@ func Doctor() []HostCheck { checks := hostcheck.RunChecks(hostcheck.DetectDistro()) out := make([]HostCheck, len(checks)) for i, c := range checks { - out[i] = HostCheck{Name: c.Name, OK: c.OK, Detail: c.Detail, Fix: c.Fix} + out[i] = HostCheck{Name: c.Name, OK: c.OK, Detail: c.Detail, Fix: c.Fix, Optional: c.Optional} } return out } diff --git a/internal/core/doctor_test.go b/internal/core/doctor_test.go index dd57709b..aef28686 100644 --- a/internal/core/doctor_test.go +++ b/internal/core/doctor_test.go @@ -44,10 +44,10 @@ func TestDoctorStructure(t *testing.T) { } } - // The union of both prior doctors' checks must be present: the four + // The union of both prior doctors' checks must be present: the five // installer binChecks plus /dev/kvm. Losing one silently is the failure to // catch. - want := []string{"qemu-system-x86_64", "qemu-img", "ssh", "xorriso", "/dev/kvm"} + want := []string{"qemu-system-x86_64", "qemu-img", "ssh", "xorriso", "git", "/dev/kvm"} for _, name := range want { if !seen[name] { t.Errorf("Doctor() is missing the %q check", name) @@ -79,3 +79,20 @@ func TestDoctorNoSSHKeygenCheck(t *testing.T) { } } } + +func TestDoctorCarriesOptionalGitFailure(t *testing.T) { + t.Setenv("PATH", t.TempDir()) + for _, c := range Doctor() { + if c.Name != "git" { + continue + } + if !c.Optional { + t.Fatal("Doctor() lost git's optional marker") + } + if len(c.Fix) == 0 { + t.Fatal("optional git failure has no install guidance") + } + return + } + t.Fatal("Doctor() has no git check") +} diff --git a/internal/core/remote_recipes.go b/internal/core/remote_recipes.go new file mode 100644 index 00000000..e0ae328d --- /dev/null +++ b/internal/core/remote_recipes.go @@ -0,0 +1,47 @@ +package core + +import ( + "errors" + "fmt" + "sort" + + "github.com/novusedge/stoat/internal/config" + "github.com/novusedge/stoat/internal/recipes" +) + +// ErrLockOutOfDate identifies a project declaration that is not pinned. +var ErrLockOutOfDate = errors.New("stoat.lock is out of date") + +// SyncRecipes validates the current project pin and repairs its cache only +// when a lock entry is missing or no longer matches the active checkout. +// Global recipe state is not touched by an apply in a non-project directory. +func SyncRecipes() error { + err := recipes.SyncProject() + if errors.Is(err, recipes.ErrLockOutOfDate) { + return fmt.Errorf("%w; run stoat recipe lock", ErrLockOutOfDate) + } + return err +} + +// RecipeUsers returns sorted VM names that list name in their recipe set. +// It reads vm.toml only. recipes.RemoveChecked calls it while holding the +// scope lock exclusively, and List resolves every VM's manifests through +// recipes.ManifestFor, which takes that same lock on a second descriptor and +// deadlocks the process. +func RecipeUsers(name string) ([]string, error) { + vms, err := config.List() + if err != nil { + return nil, err + } + users := make([]string, 0) + for _, vm := range vms { + for _, recipe := range vm.Recipes { + if recipe == name { + users = append(users, vm.Name) + break + } + } + } + sort.Strings(users) + return users, nil +} diff --git a/internal/core/remote_recipes_test.go b/internal/core/remote_recipes_test.go new file mode 100644 index 00000000..4d694602 --- /dev/null +++ b/internal/core/remote_recipes_test.go @@ -0,0 +1,461 @@ +package core + +import ( + "errors" + "fmt" + "os" + "path/filepath" + "strings" + "syscall" + "testing" + "time" + + "github.com/novusedge/stoat/internal/config" + "github.com/novusedge/stoat/internal/recipes" + "github.com/novusedge/stoat/internal/testutil" +) + +func coreRemoteRoot(t *testing.T) string { + t.Helper() + home := root(t) + index := testutil.GitRepo(t, map[string]string{"index.toml": "schema = 1\n"}) + t.Setenv("STOAT_INDEX", index) + return home +} + +func coreRecipeRepo(t *testing.T, repoName, manifestName string) string { + t.Helper() + src := testutil.GitRepo(t, map[string]string{ + "recipe.toml": fmt.Sprintf("schema = 3\nname = %q\nscript = \"install.sh\"\n", manifestName), + "install.sh": "#!/bin/sh\necho v1\n", + }) + dst := filepath.Join(filepath.Dir(src), repoName+".git") + if err := os.Rename(src, dst); err != nil { + t.Fatal(err) + } + return dst +} + +func TestSyncRecipesReportsAStaleProjectLock(t *testing.T) { + coreRemoteRoot(t) + project := t.TempDir() + t.Chdir(project) + if err := os.WriteFile(filepath.Join(project, "stoat.toml"), []byte("[recipes]\ndemo = \"main\"\n"), 0o644); err != nil { + t.Fatal(err) + } + + err := SyncRecipes() + if !errors.Is(err, ErrLockOutOfDate) || !strings.Contains(err.Error(), "run stoat recipe lock") { + t.Fatalf("SyncRecipes() = %v, want ErrLockOutOfDate with repair command", err) + } +} + +func TestRecipeUsersReturnsSortedVMNames(t *testing.T) { + coreRemoteRoot(t) + t.Chdir(t.TempDir()) + for _, v := range []*config.VM{ + {Name: "zeta", Mode: "live", RAM: 1024, CPUs: 1, SSHPort: 2201, Recipes: []string{"demo"}}, + {Name: "alpha", Mode: "live", RAM: 1024, CPUs: 1, SSHPort: 2202, Recipes: []string{"demo"}}, + {Name: "other", Mode: "live", RAM: 1024, CPUs: 1, SSHPort: 2203, Recipes: []string{"other"}}, + } { + if err := v.Save(); err != nil { + t.Fatal(err) + } + } + + users, err := RecipeUsers("demo") + if err != nil { + t.Fatal(err) + } + if got, want := strings.Join(users, ","), "alpha,zeta"; got != want { + t.Fatalf("RecipeUsers(demo) = %q, want %q", got, want) + } +} + +func TestRemoveCheckedWithRecipeUsersDoesNotDeadlock(t *testing.T) { + coreRemoteRoot(t) + project := t.TempDir() + t.Chdir(project) + src := coreRecipeRepo(t, "demo", "demo") + if err := os.WriteFile(filepath.Join(project, "stoat.toml"), []byte("[recipes]\n"), 0o644); err != nil { + t.Fatal(err) + } + scope, err := recipes.ScopeFor(false) + if err != nil { + t.Fatal(err) + } + if _, err := recipes.Add(scope, src, false); err != nil { + t.Fatal(err) + } + // A VM that lists any recipe makes core.List resolve manifests, which + // takes the scope lock RemoveChecked already holds. + v := &config.VM{Name: "other", Mode: "live", OS: "alpine", RAM: 1024, CPUs: 1, SSHPort: 2200, Recipes: []string{"xfce"}} + if err := v.Save(); err != nil { + t.Fatal(err) + } + + done := make(chan error, 1) + go func() { + done <- recipes.RemoveChecked(scope, "demo", func() ([]string, error) { return RecipeUsers("demo") }) + }() + select { + case err := <-done: + if err != nil { + t.Fatalf("RemoveChecked() = %v", err) + } + case <-time.After(10 * time.Second): + t.Fatal("RemoveChecked() blocked on its own scope lock") + } +} + +func TestPlanApplyAndNeedsProvisionUseAFreshProjectCacheWithoutSync(t *testing.T) { + home := coreRemoteRoot(t) + project := t.TempDir() + t.Chdir(project) + src := coreRecipeRepo(t, "demo", "demo") + commit := currentRecipeHead(t, src) + if err := os.WriteFile(filepath.Join(project, "stoat.toml"), []byte(fmt.Sprintf("[recipes]\ndemo = { source = %q, ref = \"main\" }\n", src)), 0o644); err != nil { + t.Fatal(err) + } + scope, err := recipes.ScopeFor(false) + if err != nil { + t.Fatal(err) + } + clone := testutil.GitClone(t, src) + if err := os.MkdirAll(scope.CachePath, 0o755); err != nil { + t.Fatal(err) + } + if err := os.Rename(clone, filepath.Join(scope.CachePath, "demo")); err != nil { + t.Fatal(err) + } + if err := recipes.SaveLock(scope.LockPath, recipes.Lock{Recipes: map[string]recipes.LockEntry{ + "demo": {Source: src, Ref: "main", Commit: commit}, + }}); err != nil { + t.Fatal(err) + } + if err := os.RemoveAll(src); err != nil { + t.Fatal(err) + } + v := &config.VM{Name: "work", Mode: "live", OS: "alpine", RAM: 1024, CPUs: 1, SSHPort: 2200, Recipes: []string{"demo"}} + if err := v.Save(); err != nil { + t.Fatal(err) + } + + plan, err := PlanApply("work", ApplyOpts{}) + if err != nil { + t.Fatalf("PlanApply() on a current cache = %v; a fresh cache must not fetch", err) + } + if len(plan) != 1 || plan[0].Name != "demo" { + t.Fatalf("PlanApply() = %+v, want demo", plan) + } + need, err := NeedsProvision(v) + if err != nil { + t.Fatalf("NeedsProvision() on a current cache = %v", err) + } + if !need { + t.Fatal("NeedsProvision() = false, want the never-applied recipe to remain work") + } + if _, err := os.Stat(filepath.Join(home, "stoat.lock")); !os.IsNotExist(err) { + t.Fatalf("fresh project planning touched global lock: stat = %v", err) + } +} + +func TestPlanApplyLazilySyncsMissingOrMismatchedProjectCache(t *testing.T) { + for _, wantMismatch := range []bool{false, true} { + name := "missing cache" + if wantMismatch { + name = "mismatched cache" + } + t.Run(name, func(t *testing.T) { + coreRemoteRoot(t) + project := t.TempDir() + t.Chdir(project) + src := coreRecipeRepo(t, "demo", "demo") + oldClone := "" + if wantMismatch { + oldClone = testutil.GitClone(t, src) + } + newCommit := testutil.GitCommit(t, src, map[string]string{"install.sh": "#!/bin/sh\necho v2\n"}, "") + if err := os.WriteFile(filepath.Join(project, "stoat.toml"), []byte(fmt.Sprintf("[recipes]\ndemo = { source = %q, ref = \"main\" }\n", src)), 0o644); err != nil { + t.Fatal(err) + } + scope, err := recipes.ScopeFor(false) + if err != nil { + t.Fatal(err) + } + if wantMismatch { + if err := os.MkdirAll(scope.CachePath, 0o755); err != nil { + t.Fatal(err) + } + if err := os.Rename(oldClone, filepath.Join(scope.CachePath, "demo")); err != nil { + t.Fatal(err) + } + } + if err := recipes.SaveLock(scope.LockPath, recipes.Lock{Recipes: map[string]recipes.LockEntry{ + "demo": {Source: src, Ref: "main", Commit: newCommit}, + }}); err != nil { + t.Fatal(err) + } + v := &config.VM{Name: "work", Mode: "live", OS: "alpine", RAM: 1024, CPUs: 1, SSHPort: 2200, Recipes: []string{"demo"}} + if err := v.Save(); err != nil { + t.Fatal(err) + } + + plan, err := PlanApply("work", ApplyOpts{}) + if err != nil { + t.Fatalf("PlanApply() = %v, want lazy sync to repair %s", err, name) + } + if len(plan) != 1 || plan[0].Name != "demo" { + t.Fatalf("PlanApply() = %+v, want demo", plan) + } + if got := currentRecipeHead(t, filepath.Join(scope.CachePath, "demo")); got != newCommit { + t.Fatalf("cache HEAD = %s, want lock commit %s", got, newCommit) + } + }) + } +} + +func TestPlanApplyRefusesDirtyOrBrokenProjectCaches(t *testing.T) { + for _, brokenGit := range []bool{false, true} { + name := "dirty cache" + if brokenGit { + name = "broken git cache" + } + t.Run(name, func(t *testing.T) { + home := coreRemoteRoot(t) + project := t.TempDir() + t.Chdir(project) + src := coreRecipeRepo(t, "demo", "demo") + commit := currentRecipeHead(t, src) + if err := os.WriteFile(filepath.Join(project, "stoat.toml"), []byte(fmt.Sprintf("[recipes]\ndemo = { source = %q, ref = \"main\" }\n", src)), 0o644); err != nil { + t.Fatal(err) + } + scope, err := recipes.ScopeFor(false) + if err != nil { + t.Fatal(err) + } + clone := testutil.GitClone(t, src) + if err := os.MkdirAll(scope.CachePath, 0o755); err != nil { + t.Fatal(err) + } + cache := filepath.Join(scope.CachePath, "demo") + if err := os.Rename(clone, cache); err != nil { + t.Fatal(err) + } + if err := recipes.SaveLock(scope.LockPath, recipes.Lock{Recipes: map[string]recipes.LockEntry{ + "demo": {Source: src, Ref: "main", Commit: commit}, + }}); err != nil { + t.Fatal(err) + } + if brokenGit { + if err := os.RemoveAll(filepath.Join(cache, ".git")); err != nil { + t.Fatal(err) + } + } else if err := os.WriteFile(filepath.Join(cache, "install.sh"), []byte("edited\n"), 0o644); err != nil { + t.Fatal(err) + } + cacheBefore, err := os.ReadFile(filepath.Join(cache, "install.sh")) + if err != nil { + t.Fatal(err) + } + lockBefore, err := os.ReadFile(scope.LockPath) + if err != nil { + t.Fatal(err) + } + v := &config.VM{Name: "work", Mode: "live", OS: "alpine", RAM: 1024, CPUs: 1, SSHPort: 2200, Recipes: []string{"demo"}} + if err := v.Save(); err != nil { + t.Fatal(err) + } + _, err = PlanApply("work", ApplyOpts{}) + if err == nil { + t.Fatal("PlanApply() = nil, want dirty/probe error") + } + if !brokenGit && !errors.Is(err, recipes.ErrDirty) { + t.Fatalf("PlanApply() = %v, want recipes.ErrDirty", err) + } + if brokenGit && strings.Contains(err.Error(), "local changes") { + t.Fatalf("broken-git PlanApply was converted to ErrDirty: %v", err) + } + if brokenGit && !strings.Contains(strings.ToLower(err.Error()), "not a git repository") { + t.Fatalf("PlanApply() = %v, want the original git probe context", err) + } + if _, statErr := os.Stat(filepath.Join(home, "stoat.lock")); !os.IsNotExist(statErr) { + t.Fatalf("PlanApply touched global lock: %v", statErr) + } + cacheAfter, _ := os.ReadFile(filepath.Join(cache, "install.sh")) + lockAfter, _ := os.ReadFile(scope.LockPath) + if string(cacheAfter) != string(cacheBefore) || string(lockAfter) != string(lockBefore) { + t.Fatal("PlanApply dirty/probe failure changed active cache or lock") + } + }) + } +} + +func TestProjectReadersRemoveHiddenUnlockedCacheEntries(t *testing.T) { + for _, reader := range []string{"SyncRecipes", "PlanApply"} { + t.Run(reader, func(t *testing.T) { + coreRemoteRoot(t) + project := t.TempDir() + t.Chdir(project) + src := coreRecipeRepo(t, "demo", "demo") + commit := currentRecipeHead(t, src) + if err := os.WriteFile(filepath.Join(project, "stoat.toml"), []byte(fmt.Sprintf( + "[recipes]\ndemo = { source = %q, ref = \"main\" }\n", src)), 0o644); err != nil { + t.Fatal(err) + } + scope, err := recipes.ScopeFor(false) + if err != nil { + t.Fatal(err) + } + clone := testutil.GitClone(t, src) + if err := os.MkdirAll(scope.CachePath, 0o755); err != nil { + t.Fatal(err) + } + if err := os.Rename(clone, filepath.Join(scope.CachePath, "demo")); err != nil { + t.Fatal(err) + } + if err := recipes.SaveLock(scope.LockPath, recipes.Lock{Recipes: map[string]recipes.LockEntry{ + "demo": {Source: src, Ref: "main", Commit: commit}, + }}); err != nil { + t.Fatal(err) + } + writeCoreRemoteFile(t, filepath.Join(scope.CachePath, ".stray", "recipe.toml"), "name = \".stray\"\nscript = \"install.sh\"\n") + writeCoreRemoteFile(t, filepath.Join(scope.CachePath, ".stray", "install.sh"), "#!/bin/sh\n") + v := &config.VM{Name: "work", Mode: "live", OS: "alpine", RAM: 1024, CPUs: 1, SSHPort: 2200, Recipes: []string{"demo"}} + if err := v.Save(); err != nil { + t.Fatal(err) + } + + if reader == "SyncRecipes" { + err = SyncRecipes() + } else { + _, err = PlanApply("work", ApplyOpts{}) + } + if err != nil { + t.Fatalf("%s() = %v, want one successful repair", reader, err) + } + if _, err := os.Stat(filepath.Join(scope.CachePath, ".stray")); !os.IsNotExist(err) { + t.Fatalf("hidden cache stray stat = %v, want not exist after %s", err, reader) + } + if _, err := os.Stat(filepath.Join(scope.CachePath, "demo", "recipe.toml")); err != nil { + t.Fatalf("current cache disappeared after %s: %v", reader, err) + } + }) + } +} + +func TestPlanApplyBlocksAcrossAnIntermediateProjectRemoval(t *testing.T) { + coreRemoteRoot(t) + project := t.TempDir() + t.Chdir(project) + src := coreRecipeRepo(t, "demo", "demo") + if err := os.WriteFile(filepath.Join(project, "stoat.toml"), []byte("[recipes]\n"), 0o644); err != nil { + t.Fatal(err) + } + scope, err := recipes.ScopeFor(false) + if err != nil { + t.Fatal(err) + } + if _, err := recipes.Add(scope, src, false); err != nil { + t.Fatal(err) + } + v := &config.VM{Name: "work", Mode: "live", OS: "alpine", RAM: 1024, CPUs: 1, SSHPort: 2200, Recipes: []string{"demo"}} + if err := v.Save(); err != nil { + t.Fatal(err) + } + + coordPath := filepath.Join(filepath.Dir(scope.CachePath), "recipe.lock") + coord, err := os.OpenFile(coordPath, os.O_CREATE|os.O_RDWR, 0o644) + if err != nil { + t.Fatal(err) + } + defer func() { _ = coord.Close() }() + if err := syscall.Flock(int(coord.Fd()), syscall.LOCK_EX); err != nil { + t.Fatal(err) + } + defer func() { _ = syscall.Flock(int(coord.Fd()), syscall.LOCK_UN) }() + lockBackup := filepath.Join(t.TempDir(), "old.lock") + if err := os.Rename(scope.LockPath, lockBackup); err != nil { + t.Fatal(err) + } + + type planResult struct { + plan []ApplyPlan + err error + } + done := make(chan planResult, 1) + go func() { + plan, planErr := PlanApply("work", ApplyOpts{}) + done <- planResult{plan: plan, err: planErr} + }() + select { + case got := <-done: + t.Fatalf("PlanApply returned while project removal lock was held: plan=%+v err=%v", got.plan, got.err) + case <-time.After(500 * time.Millisecond): + } + + if err := scope.RemoveDecl("demo"); err != nil { + t.Fatal(err) + } + if err := os.RemoveAll(filepath.Join(scope.CachePath, "demo")); err != nil { + t.Fatal(err) + } + if err := os.Remove(lockBackup); err != nil { + t.Fatal(err) + } + if err := syscall.Flock(int(coord.Fd()), syscall.LOCK_UN); err != nil { + t.Fatal(err) + } + var got planResult + select { + case got = <-done: + case <-time.After(5 * time.Second): + t.Fatal("PlanApply did not finish after the project removal lock was released") + } + if got.err == nil || errors.Is(got.err, ErrLockOutOfDate) || !errors.Is(got.err, ErrRecipeNotApplicable) { + t.Fatalf("PlanApply after complete removal = plan=%+v err=%v, want the final missing-recipe result", got.plan, got.err) + } +} + +func writeCoreRemoteFile(t *testing.T, path, body string) { + t.Helper() + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(path, []byte(body), 0o644); err != nil { + t.Fatal(err) + } +} + +func TestSyncRecipesOutsideAProjectDoesNotTouchGlobalCache(t *testing.T) { + home := coreRemoteRoot(t) + t.Chdir(t.TempDir()) + if err := SyncRecipes(); err != nil { + t.Fatalf("SyncRecipes() outside project = %v, want nil", err) + } + if _, err := os.Stat(filepath.Join(home, "stoat.lock")); !os.IsNotExist(err) { + t.Fatalf("global lock stat = %v, want no project gate write", err) + } + if _, err := os.Stat(filepath.Join(home, "recipes")); !os.IsNotExist(err) { + t.Fatalf("global cache stat = %v, want no project gate write", err) + } +} + +func currentRecipeHead(t *testing.T, dir string) string { + t.Helper() + work := testutil.GitClone(t, dir) + b, err := os.ReadFile(filepath.Join(work, ".git", "HEAD")) + if err != nil { + t.Fatal(err) + } + head := strings.TrimSpace(string(b)) + if !strings.HasPrefix(head, "ref: ") { + return head + } + ref := strings.TrimPrefix(head, "ref: ") + b, err = os.ReadFile(filepath.Join(work, ".git", filepath.FromSlash(ref))) + if err != nil { + t.Fatal(err) + } + return strings.TrimSpace(string(b)) +} diff --git a/internal/gitx/gitx.go b/internal/gitx/gitx.go new file mode 100644 index 00000000..fe078758 --- /dev/null +++ b/internal/gitx/gitx.go @@ -0,0 +1,115 @@ +// Package gitx runs git as a subprocess. Every operation has one fixed argv, +// so the command a user sees in an error is the command stoat ran. +package gitx + +import ( + "errors" + "fmt" + "os/exec" + "regexp" + "strings" +) + +var ( + ErrNoGit = errors.New("git is not installed") + ErrNoRef = errors.New("no such tag or branch") +) + +// commitRef matches a ref that names a commit rather than a tag or branch: a +// full sha, or an abbreviation git can still resolve unambiguously (7 hex +// chars is git's own minimum). --branch rejects anything but a real ref, so a +// commit has to go through CloneFull and a checkout instead. +var commitRef = regexp.MustCompile(`^[0-9a-fA-F]{7,40}$`) + +// Available reports whether git is on PATH. +func Available() bool { + _, err := exec.LookPath("git") + return err == nil +} + +// Clone makes a shallow clone of url at ref into dst. An empty ref takes the +// remote's default branch. +func Clone(url, ref, dst string) error { + if commitRef.MatchString(ref) { + if err := CloneFull(url, dst); err != nil { + return err + } + return Checkout(dst, ref) + } + args := []string{"clone", "--quiet", "--depth", "1"} + if ref != "" { + args = append(args, "--branch", ref) + } + args = append(args, "--", url, dst) + out, err := run("", args...) + if err != nil && ref != "" && strings.Contains(strings.ToLower(out), "not found in upstream origin") { + return fmt.Errorf("%w %q", ErrNoRef, ref) + } + return err +} + +// CloneFull clones every commit of url into dst and checks out nothing. Sync +// uses it because a shallow fetch of an arbitrary commit needs a server +// setting that a plain git host does not have. +func CloneFull(url, dst string) error { + _, err := run("", "clone", "--quiet", "--no-checkout", "--", url, dst) + return err +} + +// Checkout moves dir's work tree to rev and detaches HEAD. +func Checkout(dir, rev string) error { + out, err := run(dir, "checkout", "--quiet", "--detach", rev) + if err != nil && (strings.Contains(strings.ToLower(out), "did not match any file") || strings.Contains(strings.ToLower(out), "unable to read tree")) { + return fmt.Errorf("%w %q", ErrNoRef, rev) + } + return err +} + +// Fetch downloads ref from dir's origin and leaves it as FETCH_HEAD. +func Fetch(dir, ref string) error { + args := []string{"fetch", "--quiet", "--depth", "1", "origin"} + if ref != "" { + args = append(args, ref) + } + out, err := run(dir, args...) + if err != nil && ref != "" && strings.Contains(strings.ToLower(out), "couldn't find remote ref") { + return fmt.Errorf("%w %q", ErrNoRef, ref) + } + return err +} + +// RevParse returns the full sha rev resolves to in dir. +func RevParse(dir, rev string) (string, error) { + out, err := run(dir, "rev-parse", rev) + if err != nil { + return "", err + } + return strings.TrimSpace(out), nil +} + +// Dirty reports whether dir's work tree has uncommitted changes, including +// untracked files. +func Dirty(dir string) (bool, error) { + out, err := run(dir, "status", "--porcelain") + if err != nil { + return false, err + } + return strings.TrimSpace(out) != "", nil +} + +// run executes git in dir and returns its combined output. The output is +// returned on failure too, because the caller matches git's own message to +// tell a missing ref from a transport failure. +func run(dir string, args ...string) (string, error) { + if !Available() { + return "", ErrNoGit + } + cmd := exec.Command("git", args...) + cmd.Dir = dir + cmd.Env = append(cmd.Environ(), "GIT_TERMINAL_PROMPT=0", "LC_ALL=C") + out, err := cmd.CombinedOutput() + if err != nil { + return string(out), fmt.Errorf("git %s: %w: %s", strings.Join(args, " "), err, strings.TrimSpace(string(out))) + } + return string(out), nil +} diff --git a/internal/gitx/gitx_test.go b/internal/gitx/gitx_test.go new file mode 100644 index 00000000..5ce0ef14 --- /dev/null +++ b/internal/gitx/gitx_test.go @@ -0,0 +1,81 @@ +package gitx_test + +import ( + "errors" + "os" + "path/filepath" + "testing" + + "github.com/novusedge/stoat/internal/gitx" + "github.com/novusedge/stoat/internal/testutil" +) + +func TestCloneAtTagAndRevParse(t *testing.T) { + bare := testutil.GitRepo(t, map[string]string{"recipe.toml": "name = \"demo\"\n"}) + want := testutil.GitCommit(t, bare, map[string]string{"install.sh": "echo hi\n"}, "v1.2") + + dst := filepath.Join(t.TempDir(), "demo") + if err := gitx.Clone(bare, "v1.2", dst); err != nil { + t.Fatal(err) + } + got, err := gitx.RevParse(dst, "HEAD") + if err != nil { + t.Fatal(err) + } + if got != want { + t.Errorf("HEAD = %q, want %q", got, want) + } + if _, err := os.Stat(filepath.Join(dst, "install.sh")); err != nil { + t.Error(err) + } +} + +func TestCloneAtAbbreviatedCommit(t *testing.T) { + bare := testutil.GitRepo(t, map[string]string{"recipe.toml": "name = \"demo\"\n"}) + want := testutil.GitCommit(t, bare, map[string]string{"install.sh": "echo hi\n"}, "v1.2") + + dst := filepath.Join(t.TempDir(), "demo") + if err := gitx.Clone(bare, want[:7], dst); err != nil { + t.Fatal(err) + } + got, err := gitx.RevParse(dst, "HEAD") + if err != nil { + t.Fatal(err) + } + if got != want { + t.Errorf("HEAD = %q, want %q", got, want) + } +} + +func TestCloneUnknownRef(t *testing.T) { + bare := testutil.GitRepo(t, map[string]string{"recipe.toml": "name = \"demo\"\n"}) + err := gitx.Clone(bare, "v9", filepath.Join(t.TempDir(), "demo")) + if !errors.Is(err, gitx.ErrNoRef) { + t.Fatalf("err = %v, want ErrNoRef", err) + } +} + +func TestFetchUnknownRef(t *testing.T) { + bare := testutil.GitRepo(t, map[string]string{"recipe.toml": "name = \"demo\"\n"}) + dst := testutil.GitClone(t, bare) + err := gitx.Fetch(dst, "v9") + if !errors.Is(err, gitx.ErrNoRef) { + t.Fatalf("err = %v, want ErrNoRef", err) + } +} + +func TestDirtyReportsAnEditedWorkTree(t *testing.T) { + bare := testutil.GitRepo(t, map[string]string{"recipe.toml": "name = \"demo\"\n"}) + dst := testutil.GitClone(t, bare) + dirty, err := gitx.Dirty(dst) + if err != nil || dirty { + t.Fatalf("Dirty = %v, %v, want false, nil", dirty, err) + } + if err := os.WriteFile(filepath.Join(dst, "recipe.toml"), []byte("edited\n"), 0o644); err != nil { + t.Fatal(err) + } + dirty, err = gitx.Dirty(dst) + if err != nil || !dirty { + t.Fatalf("Dirty = %v, %v, want true, nil", dirty, err) + } +} diff --git a/internal/hostcheck/checks.go b/internal/hostcheck/checks.go index 1b4ac0e8..8c7d6a05 100644 --- a/internal/hostcheck/checks.go +++ b/internal/hostcheck/checks.go @@ -15,19 +15,23 @@ type Check struct { OK bool Detail string // "/usr/bin", "not found", "permission denied" Fix []string // shell commands, already distro-resolved; empty when OK + // Optional marks a binary some commands need; a missing one is not a broken host. + Optional bool } // binChecks are the executables stoat shells out to. qemu-system-x86_64 and // qemu-img come from one package on Arch and two on Debian, which is exactly // why Pkg names them per distro rather than per binary. var binChecks = []struct { - name string - pkg Pkg + name string + pkg Pkg + optional bool }{ - {"qemu-system-x86_64", Pkg{Arch: "qemu-full", Debian: "qemu-system-x86", Fedora: "qemu-kvm"}}, - {"qemu-img", Pkg{Arch: "qemu-full", Debian: "qemu-utils", Fedora: "qemu-img"}}, - {"ssh", Pkg{Arch: "openssh", Debian: "openssh-client", Fedora: "openssh-clients"}}, - {"xorriso", Pkg{Arch: "libisoburn", Debian: "xorriso", Fedora: "xorriso"}}, + {"qemu-system-x86_64", Pkg{Arch: "qemu-full", Debian: "qemu-system-x86", Fedora: "qemu-kvm"}, false}, + {"qemu-img", Pkg{Arch: "qemu-full", Debian: "qemu-utils", Fedora: "qemu-img"}, false}, + {"ssh", Pkg{Arch: "openssh", Debian: "openssh-client", Fedora: "openssh-clients"}, false}, + {"xorriso", Pkg{Arch: "libisoburn", Debian: "xorriso", Fedora: "xorriso"}, false}, + {"git", Pkg{Arch: "git", Debian: "git", Fedora: "git"}, true}, } // RunChecks probes every host requirement, in the order they are displayed. @@ -37,7 +41,9 @@ var binChecks = []struct { func RunChecks(d Distro) []Check { checks := make([]Check, 0, len(binChecks)+1) for _, b := range binChecks { - checks = append(checks, lookPathCheck(b.name, d.InstallCmd(b.pkg))) + c := lookPathCheck(b.name, d.InstallCmd(b.pkg)) + c.Optional = b.optional + checks = append(checks, c) } return append(checks, KVMCheck()) } @@ -54,7 +60,7 @@ func lookPathCheck(name string, fix []string) Check { func Problems(cs []Check) []Check { var out []Check for _, c := range cs { - if !c.OK { + if !c.OK && !c.Optional { out = append(out, c) } } diff --git a/internal/hostcheck/checks_test.go b/internal/hostcheck/checks_test.go index a3527552..2e59c608 100644 --- a/internal/hostcheck/checks_test.go +++ b/internal/hostcheck/checks_test.go @@ -31,7 +31,7 @@ func TestRunChecksAllMissing(t *testing.T) { for _, c := range checks { names = append(names, c.Name) } - want := []string{"qemu-system-x86_64", "qemu-img", "ssh", "xorriso", "/dev/kvm"} + want := []string{"qemu-system-x86_64", "qemu-img", "ssh", "xorriso", "git", "/dev/kvm"} if len(names) != len(want) { t.Fatalf("got %d checks %v, want %d %v", len(names), names, len(want), want) } @@ -41,7 +41,7 @@ func TestRunChecksAllMissing(t *testing.T) { } } - for _, c := range checks[:4] { + for _, c := range checks[:5] { if c.OK { t.Errorf("%s: OK with an empty PATH", c.Name) } @@ -54,6 +54,12 @@ func TestRunChecksAllMissing(t *testing.T) { if !strings.HasPrefix(c.Fix[0], "sudo pacman") { t.Errorf("%s: Fix = %v, want an arch command", c.Name, c.Fix) } + if c.Name == "git" && !c.Optional { + t.Errorf("%s: Optional = false, want true", c.Name) + } + if c.Name != "git" && c.Optional { + t.Errorf("%s: Optional = true, want false", c.Name) + } } } @@ -113,6 +119,27 @@ func TestRunChecksUnknownDistroHasNoCommand(t *testing.T) { } } +func TestRunChecksReportsGitAsOptionalWithDistroFix(t *testing.T) { + t.Setenv("PATH", t.TempDir()) + checks := RunChecks(DistroArch) + for _, c := range checks { + if c.Name != "git" { + continue + } + if !c.Optional { + t.Fatal("git is required by recipe commands but must be optional for host readiness") + } + if c.OK { + t.Fatal("git unexpectedly found with an empty PATH") + } + if got := strings.Join(c.Fix, " "); !strings.Contains(got, "git") || !strings.Contains(got, "pacman") { + t.Fatalf("git fix = %v, want an actionable Arch install command", c.Fix) + } + return + } + t.Fatal("RunChecks omitted the optional git check") +} + func TestKVMCheckAt(t *testing.T) { dir := t.TempDir() @@ -189,3 +216,15 @@ func TestProblems(t *testing.T) { t.Errorf("Problems() = %+v, want the two failures b and d", got) } } + +func TestProblemsExcludesOptionalFailuresFromReadiness(t *testing.T) { + cs := []Check{ + {Name: "git", OK: false, Optional: true, Fix: []string{"install git"}}, + {Name: "qemu-img", OK: false, Fix: []string{"install qemu-img"}}, + {Name: "ssh", OK: true, Optional: false}, + } + got := Problems(cs) + if len(got) != 1 || got[0].Name != "qemu-img" { + t.Fatalf("Problems() = %+v, want only required qemu-img failure", got) + } +} diff --git a/internal/installer/tui.go b/internal/installer/tui.go index 1f481b61..1e9f6a52 100644 --- a/internal/installer/tui.go +++ b/internal/installer/tui.go @@ -371,11 +371,31 @@ func (m Model) checkTable() string { StyleFunc(func(_, _ int) lipgloss.Style { return cellStyle }) for _, c := range m.checks { - t.Row(" "+status(c), c.Name, c.Detail) + t.Row(" "+status(c), checkLabel(c), c.Detail) } return t.Render() } +// repairProblems is the display subset of failed checks. hostcheck.Problems +// intentionally excludes optional failures for readiness aggregation, while +// the installer must still tell the user how to repair every missing tool. +func repairProblems(cs []Check) []Check { + var out []Check + for _, c := range cs { + if !c.OK { + out = append(out, c) + } + } + return out +} + +func checkLabel(c Check) string { + if c.Optional { + return c.Name + " (optional)" + } + return c.Name +} + func (m Model) ruleWidth() int { w := m.width - 4 if w > 60 { @@ -473,11 +493,11 @@ func (m Model) done() string { } } - if problems := Problems(m.checks); len(problems) > 0 { + if problems := repairProblems(m.checks); len(problems) > 0 { lines = append(lines, "", "before your first VM:") seen := map[string]bool{} for _, c := range problems { - lines = append(lines, "", " "+warnStyle.Render(c.Name)+": "+c.Detail) + lines = append(lines, "", " "+warnStyle.Render(checkLabel(c))+": "+c.Detail) for _, f := range c.Fix { // Fixes are deduplicated here, not in Check. Two checks // (qemu-img, qemu-system-x86_64) can share one package. The diff --git a/internal/installer/tui_test.go b/internal/installer/tui_test.go index 90ec7469..80f4753e 100644 --- a/internal/installer/tui_test.go +++ b/internal/installer/tui_test.go @@ -265,6 +265,7 @@ func TestDoneListsEveryProblemWithItsFix(t *testing.T) { m.binPath = "/home/x/.local/bin/stoat" m.checks = []Check{ {Name: "qemu-img", OK: true, Detail: "/usr/bin"}, + {Name: "git", Optional: true, Detail: "not found", Fix: []string{"sudo pacman -S --needed git"}}, {Name: "xorriso", OK: false, Detail: "not found", Fix: []string{"sudo pacman -S --needed libisoburn"}}, {Name: "/dev/kvm", OK: false, Detail: "permission denied", Fix: []string{`sudo usermod -aG kvm "$USER"`}}, } @@ -273,6 +274,8 @@ func TestDoneListsEveryProblemWithItsFix(t *testing.T) { for _, want := range []string{ "v0.3.1", "/home/x/.local/bin/stoat", + "git", + "sudo pacman -S --needed git", "xorriso", "sudo pacman -S --needed libisoburn", "/dev/kvm", @@ -292,6 +295,9 @@ func TestDoneListsEveryProblemWithItsFix(t *testing.T) { if strings.Contains(advice, "qemu-img") { t.Errorf("the advice section lists a passing check:\n%s", advice) } + if !strings.Contains(advice, "git") || !strings.Contains(advice, "sudo pacman -S --needed git") { + t.Errorf("the advice section omits optional Git repair guidance:\n%s", advice) + } } // qemu-img and qemu-system-x86_64 share the qemu-full package on Arch, so two diff --git a/internal/recipes/index.go b/internal/recipes/index.go new file mode 100644 index 00000000..09e2014e --- /dev/null +++ b/internal/recipes/index.go @@ -0,0 +1,271 @@ +package recipes + +import ( + "fmt" + "os" + "path/filepath" + "sort" + "strings" + "syscall" + "time" + + "github.com/novusedge/stoat/internal/config" + "github.com/novusedge/stoat/internal/gitx" + "github.com/novusedge/stoat/internal/logx" + "github.com/novusedge/stoat/internal/tomlx" +) + +// DefaultIndexURL is the curated index STOAT_INDEX overrides. Git accepts a +// filesystem path as a URL, which keeps tests local. +const DefaultIndexURL = "https://github.com/novusedge/stoat-recipes" + +const indexStampName = ".fetched" +const indexSourceName = ".source" +const indexLockName = ".index.lock" +const indexSchema = 1 +const indexMaxAge = 24 * time.Hour + +type Index struct { + Schema int `toml:"schema"` + Recipes map[string]IndexEntry `toml:"recipes"` +} + +type IndexEntry struct { + Name string `toml:"-"` + Source string `toml:"source"` + Description string `toml:"description"` + OS []string `toml:"os"` +} + +// IndexURL is the index repository: STOAT_INDEX, or the curated default. +func IndexURL() string { + if u := os.Getenv("STOAT_INDEX"); u != "" { + return u + } + return DefaultIndexURL +} + +// IndexDir is the local clone of the index. +func IndexDir() string { return filepath.Join(config.Root(), "index") } + +// RefreshIndex clones the index into a staging directory, validates it, then +// swaps it into place. A failed refresh leaves the last usable clone intact. +func RefreshIndex(force bool) error { + unlock, err := lockIndex() + if err != nil { + return err + } + operationErr := refreshIndexLocked(force) + unlockErr := unlock() + if operationErr != nil { + return operationErr + } + return unlockErr +} + +func refreshIndexLocked(force bool) (err error) { + root := config.Root() + if err := os.MkdirAll(root, 0o755); err != nil { + return err + } + dir := IndexDir() + stamp := filepath.Join(dir, indexStampName) + if !force { + st, statErr := os.Stat(stamp) + if statErr == nil { + if time.Since(st.ModTime()) < indexMaxAge && indexSource(dir) == IndexURL() { + return nil + } + } else if !os.IsNotExist(statErr) { + return statErr + } + } + + stage, err := os.MkdirTemp(root, ".stoat-index-stage-*") + if err != nil { + return err + } + stageActive := true + defer func() { + if stageActive { + if removeErr := os.RemoveAll(stage); removeErr != nil && err == nil { + err = removeErr + } + } + }() + + if err := gitx.Clone(IndexURL(), "", stage); err != nil { + return err + } + if _, err := loadIndex(stage); err != nil { + return err + } + if err := os.WriteFile(filepath.Join(stage, indexStampName), nil, 0o644); err != nil { + return err + } + if err := os.WriteFile(filepath.Join(stage, indexSourceName), []byte(IndexURL()), 0o644); err != nil { + return err + } + if err := replaceIndex(stage, dir); err != nil { + return err + } + stageActive = false + return nil +} + +func indexSource(dir string) string { + b, err := os.ReadFile(filepath.Join(dir, indexSourceName)) + if err != nil { + return "" + } + return string(b) +} + +func lockIndex() (func() error, error) { + root := config.Root() + if err := os.MkdirAll(root, 0o755); err != nil { + return nil, err + } + f, err := os.OpenFile(filepath.Join(root, indexLockName), os.O_CREATE|os.O_RDWR, 0o644) + if err != nil { + return nil, err + } + if err := syscall.Flock(int(f.Fd()), syscall.LOCK_EX); err != nil { + _ = f.Close() + return nil, err + } + return func() error { + unlockErr := syscall.Flock(int(f.Fd()), syscall.LOCK_UN) + closeErr := f.Close() + if unlockErr != nil { + return unlockErr + } + return closeErr + }, nil +} + +func replaceIndex(stage, dir string) error { + parent := filepath.Dir(dir) + _, statErr := os.Stat(dir) + hasOld := statErr == nil + if statErr != nil && !os.IsNotExist(statErr) { + return statErr + } + if !hasOld { + return os.Rename(stage, dir) + } + + backup, err := os.MkdirTemp(parent, ".stoat-index-old-*") + if err != nil { + return err + } + if err := os.Remove(backup); err != nil { + return err + } + if err := os.Rename(dir, backup); err != nil { + return err + } + if err := os.Rename(stage, dir); err != nil { + if restoreErr := os.Rename(backup, dir); restoreErr != nil { + return fmt.Errorf("replace index: %w; restore old index: %v", err, restoreErr) + } + return err + } + if err := os.RemoveAll(backup); err != nil { + logx.L().Warn("published recipe index but could not remove its old cache", "dir", backup, "err", err) + } + return nil +} + +// LoadIndex reads the cloned index.toml under the index file lock. +func LoadIndex() (Index, error) { + unlock, err := lockIndex() + if err != nil { + return Index{}, err + } + idx, loadErr := loadIndex(IndexDir()) + unlockErr := unlock() + if loadErr != nil { + return Index{}, loadErr + } + return idx, unlockErr +} + +func loadIndex(dir string) (Index, error) { + var idx Index + if err := tomlx.Decode(filepath.Join(dir, "index.toml"), &idx, tomlx.Reject); err != nil { + return Index{}, err + } + if idx.Schema > indexSchema { + return Index{}, fmt.Errorf("index.toml: schema %d is newer than this stoat (%d)", idx.Schema, indexSchema) + } + if idx.Recipes == nil { + idx.Recipes = map[string]IndexEntry{} + } + for name, entry := range idx.Recipes { + if err := validateRecipeName(name); err != nil { + return Index{}, err + } + if entry.Source == "" { + return Index{}, fmt.Errorf("recipe %q: missing source", name) + } + entry.Name = name + idx.Recipes[name] = entry + } + return idx, nil +} + +// SearchIndex returns entries whose name or description contains term, sorted +// by name. An empty term returns the whole index. +func SearchIndex(term string) ([]IndexEntry, error) { + unlock, err := lockIndex() + if err != nil { + return nil, err + } + if err := refreshIndexLocked(false); err != nil { + _ = unlock() + return nil, err + } + idx, err := loadIndex(IndexDir()) + unlockErr := unlock() + if err != nil { + return nil, err + } + if unlockErr != nil { + return nil, unlockErr + } + q := strings.ToLower(term) + var out []IndexEntry + for _, entry := range idx.Recipes { + if strings.Contains(strings.ToLower(entry.Name), q) || strings.Contains(strings.ToLower(entry.Description), q) { + out = append(out, entry) + } + } + sort.Slice(out, func(i, j int) bool { return out[i].Name < out[j].Name }) + return out, nil +} + +// IndexLookup refreshes the index and returns one entry by name. +func IndexLookup(name string) (IndexEntry, bool, error) { + if err := validateRecipeName(name); err != nil { + return IndexEntry{}, false, err + } + unlock, err := lockIndex() + if err != nil { + return IndexEntry{}, false, err + } + if err := refreshIndexLocked(false); err != nil { + _ = unlock() + return IndexEntry{}, false, err + } + idx, err := loadIndex(IndexDir()) + unlockErr := unlock() + if err != nil { + return IndexEntry{}, false, err + } + if unlockErr != nil { + return IndexEntry{}, false, unlockErr + } + entry, ok := idx.Recipes[name] + return entry, ok, nil +} diff --git a/internal/recipes/index_test.go b/internal/recipes/index_test.go new file mode 100644 index 00000000..7e1bb8ab --- /dev/null +++ b/internal/recipes/index_test.go @@ -0,0 +1,259 @@ +package recipes + +import ( + "os" + "path/filepath" + "strings" + "testing" + "time" + + "github.com/novusedge/stoat/internal/config" + "github.com/novusedge/stoat/internal/testutil" +) + +const sampleIndex = `schema = 1 + +[recipes.tailscale] +source = "https://example.invalid/x/stoat-tailscale" +description = "join a tailnet on boot" +os = ["alpine", "debian"] + +[recipes.xfce] +source = "https://example.invalid/x/stoat-xfce" +description = "a desktop over vnc" +os = ["debian"] +` + +func indexRoot(t *testing.T) { + t.Helper() + t.Setenv("STOAT_HOME", t.TempDir()) + t.Chdir(t.TempDir()) + t.Setenv("STOAT_INDEX", testutil.GitRepo(t, map[string]string{"index.toml": sampleIndex})) +} + +func TestRefreshIndexClonesAndLoads(t *testing.T) { + indexRoot(t) + if err := RefreshIndex(true); err != nil { + t.Fatal(err) + } + idx, err := LoadIndex() + if err != nil { + t.Fatal(err) + } + if idx.Recipes["tailscale"].Source == "" { + t.Fatalf("index = %+v", idx) + } +} + +func TestRefreshIndexIsSkippedWhenFresh(t *testing.T) { + indexRoot(t) + if err := RefreshIndex(true); err != nil { + t.Fatal(err) + } + stamp := filepath.Join(IndexDir(), indexStampName) + before, err := os.Stat(stamp) + if err != nil { + t.Fatal(err) + } + if err := RefreshIndex(false); err != nil { + t.Fatal(err) + } + after, err := os.Stat(stamp) + if err != nil { + t.Fatal(err) + } + if !after.ModTime().Equal(before.ModTime()) { + t.Error("a fresh index was fetched again") + } +} + +func TestRefreshIndexUsesChangedSourceWhenCacheIsFresh(t *testing.T) { + home := t.TempDir() + t.Setenv("STOAT_HOME", home) + t.Chdir(t.TempDir()) + sourceA := testutil.GitRepo(t, map[string]string{"index.toml": sampleIndex}) + sourceB := testutil.GitRepo(t, map[string]string{"index.toml": `schema = 1 + +[recipes.other] +source = "https://example.invalid/x/stoat-other" +description = "only from source B" +os = ["alpine"] +`}) + t.Setenv("STOAT_INDEX", sourceA) + if err := RefreshIndex(true); err != nil { + t.Fatal(err) + } + t.Setenv("STOAT_INDEX", sourceB) + got, err := SearchIndex("") + if err != nil { + t.Fatal(err) + } + if len(got) != 1 || got[0].Name != "other" { + t.Fatalf("fresh cache after source change = %+v, want source B", got) + } +} + +func TestRefreshIndexFetchesAStaleIndex(t *testing.T) { + indexRoot(t) + if err := RefreshIndex(true); err != nil { + t.Fatal(err) + } + stamp := filepath.Join(IndexDir(), indexStampName) + old := time.Now().Add(-25 * time.Hour) + if err := os.Chtimes(stamp, old, old); err != nil { + t.Fatal(err) + } + indexSource := os.Getenv("STOAT_INDEX") + testutil.GitCommit(t, indexSource, map[string]string{ + "index.toml": strings.Replace(sampleIndex, "join a tailnet on boot", "updated description", 1), + }, "") + if err := RefreshIndex(false); err != nil { + t.Fatal(err) + } + idx, err := LoadIndex() + if err != nil { + t.Fatal(err) + } + if got := idx.Recipes["tailscale"].Description; got != "updated description" { + t.Fatalf("stale index description = %q, want updated description", got) + } +} + +func TestRefreshIndexFailureKeepsUsableCache(t *testing.T) { + indexRoot(t) + if err := RefreshIndex(true); err != nil { + t.Fatal(err) + } + t.Setenv("STOAT_INDEX", filepath.Join(t.TempDir(), "missing-index.git")) + if err := RefreshIndex(true); err == nil { + t.Fatal("RefreshIndex unexpectedly succeeded for a missing local source") + } + idx, err := LoadIndex() + if err != nil { + t.Fatal(err) + } + if idx.Recipes["tailscale"].Source == "" { + t.Fatalf("cached index was lost after refresh failure: %+v", idx) + } +} + +func TestRefreshIndexSucceedsWhenOldCacheCleanupFails(t *testing.T) { + if os.Geteuid() == 0 { + t.Skip("permission-based cleanup failure is unavailable to root") + } + home := t.TempDir() + t.Setenv("STOAT_HOME", home) + t.Chdir(t.TempDir()) + t.Cleanup(func() { + if err := restoreIndexTreePermissions(home); err != nil { + t.Error(err) + } + }) + sourceA := testutil.GitRepo(t, map[string]string{"index.toml": sampleIndex}) + sourceB := testutil.GitRepo(t, map[string]string{"index.toml": `schema = 1 + +[recipes.other] +source = "https://example.invalid/x/stoat-other" +description = "only from source B" +os = ["alpine"] +`}) + t.Setenv("STOAT_INDEX", sourceA) + if err := RefreshIndex(true); err != nil { + t.Fatal(err) + } + managedVM := "name = \"cache-workspace\"\nmode = \"live\"\n" + writeFile(t, filepath.Join(IndexDir(), "vm.toml"), managedVM) + private := filepath.Join(IndexDir(), "old-cache", "private") + if err := os.MkdirAll(private, 0o755); err != nil { + t.Fatal(err) + } + writeFile(t, filepath.Join(private, "keep.me"), "caller-owned\n") + if err := os.Chmod(private, 0o500); err != nil { + t.Fatal(err) + } + if err := os.Chmod(IndexDir(), 0o500); err != nil { + t.Fatal(err) + } + + t.Setenv("STOAT_INDEX", sourceB) + if err := RefreshIndex(true); err != nil { + t.Fatalf("published refresh was reported as failed: %v", err) + } + idx, err := LoadIndex() + if err != nil { + t.Fatal(err) + } + if len(idx.Recipes) != 1 || idx.Recipes["other"].Description != "only from source B" { + t.Fatalf("published index = %+v, want source B", idx) + } + foundManagedVM := false + if err := filepath.Walk(home, func(path string, info os.FileInfo, err error) error { + if err != nil { + return err + } + if !info.IsDir() && filepath.Base(path) == "vm.toml" { + data, err := os.ReadFile(path) + if err != nil { + return err + } + if string(data) == managedVM { + foundManagedVM = true + } + } + return nil + }); err != nil { + t.Fatal(err) + } + if !foundManagedVM { + t.Fatal("old index vm.toml was removed before cleanup completed") + } + listed, err := config.List() + if err != nil { + t.Fatal(err) + } + if len(listed) != 0 { + t.Fatalf("internal index cleanup directories appeared in VM list: %+v", listed) + } + broken, err := config.ListBroken() + if err != nil { + t.Fatal(err) + } + if len(broken) != 0 { + t.Fatalf("internal index cleanup directories appeared as broken VMs: %+v", broken) + } +} + +func TestSearchIndexMatchesNameAndDescription(t *testing.T) { + indexRoot(t) + got, err := SearchIndex("tailnet") + if err != nil { + t.Fatal(err) + } + if len(got) != 1 || got[0].Name != "tailscale" { + t.Fatalf("results = %+v", got) + } +} + +func TestIndexLookupUnknownName(t *testing.T) { + indexRoot(t) + _, ok, err := IndexLookup("tailscal") + if err != nil { + t.Fatal(err) + } + if ok { + t.Error("tailscal resolved") + } +} + +func restoreIndexTreePermissions(root string) error { + return filepath.Walk(root, func(path string, info os.FileInfo, err error) error { + if err != nil { + return err + } + mode := os.FileMode(0o600) + if info.IsDir() { + mode = 0o700 + } + return os.Chmod(path, mode) + }) +} diff --git a/internal/recipes/lock.go b/internal/recipes/lock.go new file mode 100644 index 00000000..27df24e0 --- /dev/null +++ b/internal/recipes/lock.go @@ -0,0 +1,153 @@ +package recipes + +import ( + "encoding/hex" + "fmt" + "os" + "path/filepath" + + "github.com/novusedge/stoat/internal/tomlx" +) + +// LockSchema is the lock format this stoat writes and the highest it reads. +const LockSchema = 1 + +// lockHeader sits above the encoded lock. tomlx.Encode writes no comments, so +// SaveLock prepends this itself. +const lockHeader = "# stoat.lock: written by stoat; do not edit\n" + +// Lock pins every remote recipe in one scope to a commit. +type Lock struct { + Schema int `toml:"schema"` + Recipes map[string]LockEntry `toml:"recipes"` +} + +// LockEntry is one recipe's pin. Ref is the tag, branch, or commit the user +// asked for. Commit is the full sha that ref resolved to when the lock was +// written. +type LockEntry struct { + // Name is the resolved recipe name returned by add/update. It is not + // persisted in stoat.lock because the map key already carries it. + Name string `toml:"-"` + Source string `toml:"source"` + Ref string `toml:"ref"` + Commit string `toml:"commit"` + Added string `toml:"added"` +} + +// LoadLock reads path. A missing file is an empty lock, not an error: the +// first recipe add in a scope creates it. +func LoadLock(path string) (Lock, error) { + l := Lock{Schema: LockSchema, Recipes: map[string]LockEntry{}} + if _, err := os.Stat(path); err != nil { + if os.IsNotExist(err) { + return l, nil + } + return Lock{}, err + } + if err := tomlx.Decode(path, &l, tomlx.Reject); err != nil { + return Lock{}, err + } + if l.Schema > LockSchema { + return Lock{}, fmt.Errorf("%s: schema %d is newer than this stoat (%d)", filepath.Base(path), l.Schema, LockSchema) + } + if l.Recipes == nil { + l.Recipes = map[string]LockEntry{} + } + for name := range l.Recipes { + if err := validateRecipeName(name); err != nil { + return Lock{}, err + } + } + return l, nil +} + +func validateRemoteLock(l Lock) error { + for name, entry := range l.Recipes { + if err := validateRecipeName(name); err != nil { + return err + } + if entry.Source == "" { + return fmt.Errorf("recipe %q: missing source", name) + } + if len(entry.Commit) != 40 || !isHexCommit(entry.Commit) { + return fmt.Errorf("recipe %q: commit must be a full 40-character hexadecimal commit", name) + } + } + return nil +} + +func isHexCommit(commit string) bool { + if len(commit) != 40 { + return false + } + _, err := hex.DecodeString(commit) + return err == nil +} + +// SaveLock writes l with the do-not-edit header above it. The completed file +// is renamed into place only after encoding succeeds. +func SaveLock(path string, l Lock) (err error) { + l.Schema = LockSchema + if l.Recipes == nil { + l.Recipes = map[string]LockEntry{} + } + dir := filepath.Dir(path) + if err := os.MkdirAll(dir, 0o755); err != nil { + return err + } + mode, err := existingFileMode(path) + if err != nil { + return err + } + stageDir, err := os.MkdirTemp(dir, ".stoat-lock-*") + if err != nil { + return err + } + tmpPath := filepath.Join(stageDir, "lock") + tmp, err := os.OpenFile(tmpPath, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0o644) + if err != nil { + _ = os.RemoveAll(stageDir) + return err + } + if err := tmp.Close(); err != nil { + _ = os.RemoveAll(stageDir) + return err + } + defer func() { + if removeErr := os.RemoveAll(stageDir); removeErr != nil && err == nil { + err = removeErr + } + }() + + if err := tomlx.Encode(tmpPath, l); err != nil { + return err + } + body, err := os.ReadFile(tmpPath) + if err != nil { + return err + } + if err := os.WriteFile(tmpPath, append([]byte(lockHeader), body...), 0o644); err != nil { + return err + } + if mode != 0 { + if err := os.Chmod(tmpPath, mode); err != nil { + return err + } + } + if err := os.Rename(tmpPath, path); err != nil { + return err + } + return nil +} + +func existingFileMode(path string) (os.FileMode, error) { + info, err := os.Stat(path) + if err == nil { + return info.Mode().Perm(), nil + } + if os.IsNotExist(err) { + return 0, nil + } + return 0, err +} diff --git a/internal/recipes/lock_test.go b/internal/recipes/lock_test.go new file mode 100644 index 00000000..e1bd71e0 --- /dev/null +++ b/internal/recipes/lock_test.go @@ -0,0 +1,121 @@ +package recipes + +import ( + "os" + "path/filepath" + "strings" + "testing" +) + +func TestSaveLockRoundTrips(t *testing.T) { + path := filepath.Join(t.TempDir(), "stoat.lock") + want := Lock{Schema: LockSchema, Recipes: map[string]LockEntry{ + "tailscale": { + Source: "https://github.com/x/stoat-tailscale", + Ref: "v1.2", + Commit: "9f3c1e2a7b0000000000000000000000000000ab", + Added: "2026-09-04T10:00:00Z", + }, + }} + if err := SaveLock(path, want); err != nil { + t.Fatal(err) + } + got, err := LoadLock(path) + if err != nil { + t.Fatal(err) + } + if got.Schema != want.Schema { + t.Errorf("schema = %d, want %d", got.Schema, want.Schema) + } + if got.Recipes["tailscale"] != want.Recipes["tailscale"] { + t.Errorf("entry = %+v, want %+v", got.Recipes["tailscale"], want.Recipes["tailscale"]) + } + b := readFile(t, path) + if !strings.HasPrefix(b, "# stoat.lock: written by stoat; do not edit\n") { + t.Errorf("lock has no header comment:\n%s", b) + } + info, err := os.Stat(path) + if err != nil { + t.Fatal(err) + } + if got := info.Mode().Perm(); got != 0o644 { + t.Errorf("new lock mode = %o, want 644", got) + } +} + +func TestLoadLockMissingFileIsEmpty(t *testing.T) { + l, err := LoadLock(filepath.Join(t.TempDir(), "stoat.lock")) + if err != nil { + t.Fatal(err) + } + if len(l.Recipes) != 0 { + t.Errorf("recipes = %v, want empty", l.Recipes) + } +} + +func TestLoadLockRejectsANewerSchema(t *testing.T) { + path := filepath.Join(t.TempDir(), "stoat.lock") + writeFile(t, path, "schema = 2\n") + _, err := LoadLock(path) + if err == nil || !strings.Contains(err.Error(), "schema 2 is newer than this stoat (1)") { + t.Fatalf("err = %v, want the newer-schema message", err) + } +} + +func TestSaveLockLeavesPreviousLockWhenReplacementFails(t *testing.T) { + if os.Geteuid() == 0 { + t.Skip("permission-based replacement failure is unavailable to root") + } + dir := t.TempDir() + path := filepath.Join(dir, "stoat.lock") + foreignPath := filepath.Join(dir, "keep.me") + foreign := []byte("caller-owned\n") + old := Lock{Schema: LockSchema, Recipes: map[string]LockEntry{ + "old": {Source: "source", Ref: "main", Commit: "old", Added: "now"}, + }} + if err := SaveLock(path, old); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(foreignPath, foreign, 0o644); err != nil { + t.Fatal(err) + } + before, err := os.ReadFile(path) + if err != nil { + t.Fatal(err) + } + if len(before) == 0 { + t.Fatal("initial lock is empty") + } + if err := os.Chmod(path, 0o444); err != nil { + t.Fatal(err) + } + if err := os.Chmod(dir, 0o500); err != nil { + t.Fatal(err) + } + t.Cleanup(func() { + if err := os.Chmod(dir, 0o700); err != nil { + t.Error(err) + } + }) + + newLock := Lock{Schema: LockSchema, Recipes: map[string]LockEntry{ + "new": {Source: "source", Ref: "main", Commit: "new", Added: "now"}, + }} + if err := SaveLock(path, newLock); err == nil { + t.Fatal("SaveLock unexpectedly replaced a lock after staging failed") + } + got, err := os.ReadFile(path) + if err != nil { + t.Fatal(err) + } + if string(got) != string(before) { + t.Errorf("previous lock bytes changed from %q to %q", before, got) + } + got, err = os.ReadFile(foreignPath) + if err != nil { + t.Fatal(err) + } + if string(got) != string(foreign) { + t.Errorf("foreign artifact changed from %q to %q", foreign, got) + } +} diff --git a/internal/recipes/manifest.go b/internal/recipes/manifest.go index 0abf4878..20274899 100644 --- a/internal/recipes/manifest.go +++ b/internal/recipes/manifest.go @@ -190,6 +190,9 @@ func ParseManifest(path string) (Manifest, error) { if m.Name == "" { return Manifest{}, fmt.Errorf("%s: missing required field %q", path, "name") } + if err := validateRecipeName(m.Name); err != nil { + return Manifest{}, fmt.Errorf("%s: %w", path, err) + } if m.Script == "" { return Manifest{}, fmt.Errorf("%s: missing required field %q", path, "script") } @@ -357,20 +360,36 @@ func validateHealth(path string, h Health) error { return nil } -// ManifestFor resolves name (an entry in the recipes root, the same -// identifier VM.Recipes/ApplyOpts.Only use) to its recipe.toml manifest -// (docs/recipe-spec-v2.md). +// ManifestFor resolves name to its recipe.toml through ResolvePath, so a +// project recipe shadows a home one of the same name. // -// ok is false with a nil error when name has no recipe.toml at all: an -// unrelated or nonexistent name that CheckRecipes/List reject elsewhere. A -// caller decides what absence means for it. A recipe.toml that exists but -// fails to parse is a real problem, and comes back as err instead. +// ok is false with a nil error when no root holds name. A recipe.toml that +// exists but fails to parse comes back as err instead. func ManifestFor(name string) (m Manifest, ok bool, err error) { - path := filepath.Join(dir(), name, "recipe.toml") - if _, statErr := os.Stat(path); statErr != nil { + locks, err := lockRecipeScopes(false) + if err != nil { + return Manifest{}, false, err + } + m, ok, readErr := manifestForLocked(name) + unlockErr := unlockRecipeScopes(locks) + if readErr != nil { + return Manifest{}, false, readErr + } + if unlockErr != nil { + return Manifest{}, false, unlockErr + } + return m, ok, nil +} + +func manifestForLocked(name string) (m Manifest, ok bool, err error) { + d, _, found, resolveErr := resolvePath(name) + if resolveErr != nil { + return Manifest{}, false, resolveErr + } + if !found { return Manifest{}, false, nil } - m, err = ParseManifest(path) + m, err = ParseManifest(filepath.Join(d, "recipe.toml")) if err != nil { return Manifest{}, false, err } @@ -402,6 +421,15 @@ func (m Manifest) ScriptContent(osName string) (string, error) { return string(b), nil } +// ScriptHash returns the hash of the script selected for osName. +func (m Manifest) ScriptHash(osName string) (string, error) { + body, err := m.ScriptContent(osName) + if err != nil { + return "", err + } + return sum([]byte(body)), nil +} + // hasCapability reports whether cap resolves against vmOS. The table comes // from the loaded guests, so a new guest file adds capabilities without a // Go edit. diff --git a/internal/recipes/manifests_test.go b/internal/recipes/manifests_test.go index ce8308f5..d27c46be 100644 --- a/internal/recipes/manifests_test.go +++ b/internal/recipes/manifests_test.go @@ -29,6 +29,7 @@ func writeV2Recipe(t *testing.T, name, description string) { func TestListManifestsFindsV2Recipes(t *testing.T) { t.Setenv("STOAT_HOME", t.TempDir()) + t.Chdir(t.TempDir()) writeV2Recipe(t, "xfce", "XFCE desktop") writeV2Recipe(t, "docker", "Docker engine") @@ -56,6 +57,7 @@ func TestListManifestsSkipsDirWithoutManifest(t *testing.T) { // scratch dir) is not a v2 recipe and must not be reported, let alone // error the whole listing. t.Setenv("STOAT_HOME", t.TempDir()) + t.Chdir(t.TempDir()) if err := os.MkdirAll(filepath.Join(dir(), "not-a-recipe"), 0o755); err != nil { t.Fatal(err) } @@ -72,6 +74,7 @@ func TestListManifestsSkipsDirWithoutManifest(t *testing.T) { func TestListManifestsSkipsInvalidManifestButKeepsOthers(t *testing.T) { t.Setenv("STOAT_HOME", t.TempDir()) + t.Chdir(t.TempDir()) writeV2Recipe(t, "xfce", "XFCE desktop") broken := filepath.Join(dir(), "broken") @@ -94,6 +97,7 @@ func TestListManifestsSkipsInvalidManifestButKeepsOthers(t *testing.T) { func TestListManifestsNoDirYet(t *testing.T) { t.Setenv("STOAT_HOME", t.TempDir()) + t.Chdir(t.TempDir()) got, err := ListManifests() if err != nil { t.Fatal(err) @@ -111,6 +115,7 @@ func TestListManifestsNoDirYet(t *testing.T) { // ("xfce/recipe.toml"), with scripts landing executable. func TestInstallCopiesBundledV2RecipeDirectories(t *testing.T) { t.Setenv("STOAT_HOME", t.TempDir()) + t.Chdir(t.TempDir()) fake := fstest.MapFS{ "xfce/recipe.toml": {Data: []byte("name = \"xfce\"\nscript = \"install.sh\"\n"), Mode: 0o644}, @@ -152,6 +157,7 @@ func TestInstallCopiesBundledV2RecipeDirectories(t *testing.T) { // already gives flat files. func TestInstallPreservesEditsInsideAV2RecipeDirectory(t *testing.T) { t.Setenv("STOAT_HOME", t.TempDir()) + t.Chdir(t.TempDir()) fake := fstest.MapFS{ "xfce/recipe.toml": {Data: []byte("name = \"xfce\"\nscript = \"install.sh\"\n"), Mode: 0o644}, diff --git a/internal/recipes/names.go b/internal/recipes/names.go new file mode 100644 index 00000000..1d6828f7 --- /dev/null +++ b/internal/recipes/names.go @@ -0,0 +1,45 @@ +package recipes + +import ( + "fmt" + "path/filepath" + "strings" + "unicode" +) + +// validateRecipeName accepts safe filesystem components while preserving the +// names used by bundled and user-authored Go recipes. +func validateRecipeName(name string) error { + if name == "" || name == "." || name == ".." { + return fmt.Errorf("invalid recipe name %q", name) + } + if filepath.Base(name) != name || strings.ContainsAny(name, `/\\`) || strings.HasPrefix(name, ".") { + return fmt.Errorf("invalid recipe name %q", name) + } + for _, r := range name { + if unicode.IsSpace(r) || r == 0 { + return fmt.Errorf("invalid recipe name %q", name) + } + } + return nil +} + +func recipeTarget(root, name string) (string, error) { + if err := validateRecipeName(name); err != nil { + return "", err + } + return containedPath(root, name) +} + +func containedPath(root, name string) (string, error) { + root, err := filepath.Abs(root) + if err != nil { + return "", err + } + target := filepath.Join(root, name) + rel, err := filepath.Rel(root, target) + if err != nil || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) { + return "", fmt.Errorf("recipe %q escapes %s", name, root) + } + return target, nil +} diff --git a/internal/recipes/recipes.go b/internal/recipes/recipes.go index 4a7a2e93..2ecf0121 100644 --- a/internal/recipes/recipes.go +++ b/internal/recipes/recipes.go @@ -298,39 +298,69 @@ func List(osName, _ string) ([]string, error) { return out, nil } -// ListManifests scans dir() for v2 recipes: subdirectories holding a -// recipe.toml (docs/recipe-spec-v2.md). Unlike List, it does not filter by -// OS or backend. A caller that needs that filters against the parsed -// Manifest's OS/Requires fields (see MatchesVM). +// ListManifests scans every root for v2 recipes: subdirectories holding a +// recipe.toml (docs/recipe-spec-v2.md). Unlike List, it does not filter by OS +// or backend. A caller that needs that filters against the parsed Manifest's +// OS/Requires fields (see MatchesVM). // // A subdirectory with no recipe.toml, a stray directory or leftover .bak // territory, is silently skipped. A directory that is a recipe but fails to -// parse is also skipped, but logged: one typo'd manifest should not take -// every other recipe down with it. +// parse is claimed by its root and logged, so a lower-priority recipe cannot +// replace it. func ListManifests() ([]Manifest, error) { - entries, err := os.ReadDir(dir()) + locks, err := lockRecipeScopes(false) if err != nil { - if os.IsNotExist(err) { - return nil, nil - } return nil, err } + manifests, readErr := listManifestsLocked() + unlockErr := unlockRecipeScopes(locks) + if readErr != nil { + return nil, readErr + } + return manifests, unlockErr +} +func listManifestsLocked() ([]Manifest, error) { + seen := map[string]bool{} var out []Manifest - for _, e := range entries { - if !e.IsDir() { - continue - } - path := filepath.Join(dir(), e.Name(), "recipe.toml") - if _, err := os.Stat(path); err != nil { - continue // not a v2 recipe directory - } - m, err := ParseManifest(path) + roots, err := Roots() + if err != nil { + return nil, err + } + for _, root := range roots { + entries, err := os.ReadDir(root.Path) if err != nil { - logx.L().Warn("skipping recipe with an invalid manifest", "dir", e.Name(), "err", err) - continue + if os.IsNotExist(err) { + continue + } + return nil, err + } + for _, e := range entries { + if !e.IsDir() || seen[e.Name()] { + continue + } + owned, err := owns(root, e.Name()) + if err != nil { + return nil, err + } + if !owned { + continue + } + path := filepath.Join(root.Path, e.Name(), "recipe.toml") + if _, err := os.Stat(path); err != nil { + if os.IsNotExist(err) { + continue + } + return nil, err + } + seen[e.Name()] = true + m, err := ParseManifest(path) + if err != nil { + logx.L().Warn("skipping recipe with an invalid manifest", "dir", e.Name(), "err", err) + continue + } + out = append(out, m) } - out = append(out, m) } sort.Slice(out, func(i, j int) bool { return out[i].Name < out[j].Name }) return out, nil diff --git a/internal/recipes/remote.go b/internal/recipes/remote.go new file mode 100644 index 00000000..4a5bb073 --- /dev/null +++ b/internal/recipes/remote.go @@ -0,0 +1,1025 @@ +package recipes + +import ( + "errors" + "fmt" + "io" + "net/url" + "os" + "path/filepath" + "sort" + "strings" + "time" + + "github.com/novusedge/stoat/internal/gitx" + "github.com/novusedge/stoat/internal/tomlx" +) + +// ErrDirty identifies a cache checkout with uncommitted changes. +var ErrDirty = errors.New("local changes") + +// ErrLockOutOfDate identifies a project declaration that is not pinned. +var ErrLockOutOfDate = errors.New("stoat.lock is out of date") + +// RemoveInUse reports a recipe that became referenced after confirmation. +type RemoveInUse struct { + Users []string +} + +func (e *RemoveInUse) Error() string { + return fmt.Sprintf("recipe is used by %s", strings.Join(e.Users, ", ")) +} + +// ParseRef splits an index name or repository URL from its optional ref. +// Scp-style URLs retain the username because their first at-sign is followed +// by a host and colon, not by a ref. +func ParseRef(in string) (source, gitRef string, isURL bool) { + source = in + if i := strings.LastIndexByte(in, '@'); i > 0 && i+1 < len(in) { + candidateSource, candidateRef := in[:i], in[i+1:] + sourceLike := strings.Contains(candidateSource, "://") || strings.Contains(candidateSource, ":") || strings.Contains(candidateSource, "/") || + strings.HasPrefix(candidateSource, "/") || strings.HasPrefix(candidateSource, ".") + indexLike := !strings.ContainsAny(candidateSource, "/:") && !strings.HasPrefix(candidateSource, ".") + authorityAt := false + if scheme := strings.Index(in, "://"); scheme >= 0 { + authorityStart := scheme + len("://") + authorityEnd := strings.IndexByte(in[authorityStart:], '/') + if authorityEnd < 0 { + authorityEnd = len(in) - authorityStart + } + authorityAt = i < authorityStart+authorityEnd + } + // An scp-style source has the form user@host:path. Its colon is + // part of the source unless the source itself already contains path + // syntax before a second at-sign carrying the requested ref. + if !authorityAt && (sourceLike || (indexLike && !strings.Contains(candidateRef, ":"))) { + source, gitRef = candidateSource, candidateRef + } + } + isURL = strings.Contains(source, "://") || strings.Contains(source, ":") || + strings.Contains(source, "/") || strings.HasPrefix(source, ".") || strings.HasSuffix(source, ".git") + return source, gitRef, isURL +} + +func nameFromURL(source string) string { + name := strings.TrimSuffix(strings.TrimSuffix(source, "/"), ".git") + if i := strings.LastIndexAny(name, "/:"); i >= 0 { + name = name[i+1:] + } + return strings.TrimPrefix(name, "stoat-") +} + +func resolveSource(in string) (name, source, gitRef string, err error) { + source, gitRef, isURL := ParseRef(in) + if isURL { + name := nameFromURL(source) + if err := validateRecipeName(name); err != nil { + return "", "", "", err + } + return name, source, gitRef, nil + } + entry, ok, err := IndexLookup(source) + if err != nil { + return "", "", "", err + } + if !ok { + return "", "", "", fmt.Errorf("no recipe %q in the index; run stoat recipe search %s", source, source) + } + return entry.Name, entry.Source, gitRef, nil +} + +// Preview clones a source into a temporary directory and parses its manifest. +// The returned directory remains available to the caller until it removes it. +func Preview(source, gitRef string) (Manifest, string, error) { + tmp, err := os.MkdirTemp("", "stoat-preview-") + if err != nil { + return Manifest{}, "", err + } + dst := filepath.Join(tmp, "recipe") + if err := gitx.Clone(source, gitRef, dst); err != nil { + if removeErr := os.RemoveAll(tmp); removeErr != nil { + return Manifest{}, "", fmt.Errorf("%w; remove preview: %v", refError(source, gitRef, err), removeErr) + } + return Manifest{}, "", refError(source, gitRef, err) + } + m, err := ParseManifest(filepath.Join(dst, "recipe.toml")) + if err != nil { + if os.IsNotExist(err) || strings.Contains(err.Error(), "no such file or directory") { + err = fmt.Errorf("%s: no recipe.toml at the repository root", source) + } + if removeErr := os.RemoveAll(tmp); removeErr != nil { + return Manifest{}, "", fmt.Errorf("%w; remove preview: %v", err, removeErr) + } + return Manifest{}, "", err + } + if err := validateRecipeName(m.Name); err != nil { + if removeErr := os.RemoveAll(tmp); removeErr != nil { + return Manifest{}, "", fmt.Errorf("%w; remove preview: %v", err, removeErr) + } + return Manifest{}, "", err + } + return m, tmp, nil +} + +func refError(source, gitRef string, err error) error { + if errors.Is(err, gitx.ErrNoRef) { + kind := "tag or branch" + if isHexCommit(gitRef) { + kind = "tag, branch, or commit" + } + return fmt.Errorf("%s: no %s %q", refLabel(source), kind, gitRef) + } + return err +} + +func refLabel(source string) string { + if strings.Contains(source, "://") { + if parsed, err := url.Parse(source); err == nil && parsed.Path != "" { + return strings.TrimSuffix(strings.TrimPrefix(strings.Trim(parsed.Path, "/"), "/"), ".git") + } + } + if colon := strings.IndexByte(source, ':'); colon >= 0 && !strings.Contains(source[:colon], "/") { + source = source[colon+1:] + } + return strings.TrimSuffix(strings.Trim(source, "/"), ".git") +} + +// Add stages a validated checkout and all related files before replacing the +// active cache, lock, declaration, and gitignore entries. +func Add(s Scope, in string, force bool) (LockEntry, error) { + name, source, gitRef, err := resolveSource(in) + if err != nil { + return LockEntry{}, err + } + unlock, err := lockScope(s) + if err != nil { + return LockEntry{}, err + } + defer func() { _ = unlock() }() + if !force { + if err := CheckCollision(name, s.Name); err != nil { + return LockEntry{}, err + } + } + + if err := os.MkdirAll(filepath.Dir(s.CachePath), 0o755); err != nil { + return LockEntry{}, err + } + stageRoot, err := os.MkdirTemp(filepath.Dir(s.CachePath), ".stoat-recipe-add-*") + if err != nil { + return LockEntry{}, err + } + defer func() { _ = os.RemoveAll(stageRoot) }() + stageCache, err := recipeTarget(stageRoot, name) + if err != nil { + return LockEntry{}, err + } + if err := gitx.Clone(source, gitRef, stageCache); err != nil { + return LockEntry{}, refError(source, gitRef, err) + } + if err := ValidateTree(stageCache, name); err != nil { + return LockEntry{}, err + } + commit, err := gitx.RevParse(stageCache, "HEAD") + if err != nil { + return LockEntry{}, err + } + entry := LockEntry{Name: name, Source: source, Ref: gitRef, Commit: commit, Added: time.Now().UTC().Format(time.RFC3339)} + + lock, err := s.Lock() + if err != nil { + return LockEntry{}, err + } + if lock.Recipes == nil { + lock.Recipes = map[string]LockEntry{} + } + persistEntry := entry + persistEntry.Name = "" + lock.Recipes[name] = persistEntry + artifacts, cleanup, err := prepareAddArtifacts(s, name, lock, in, source, gitRef, stageCache, stageRoot) + if err != nil { + return LockEntry{}, err + } + defer func() { _ = cleanup() }() + if err := publishArtifacts(artifacts); err != nil { + return LockEntry{}, err + } + return entry, nil +} + +type artifact struct { + target string + stage string + isDir bool + remove bool +} + +type publishedArtifact struct { + artifact + backup string + oldExists bool + targetMoved bool + published bool +} + +// These defaults are the standard filesystem operations. Tests can replace +// them briefly to induce a deterministic publication or backup-cleanup fault +// through Add/Remove and then restore the defaults with t.Cleanup; production +// behavior remains the direct os.Rename/os.RemoveAll path. +var transactionRename = os.Rename +var transactionRemoveBackup = func(target, backup string) error { + _ = target + return os.RemoveAll(backup) +} + +func prepareAddArtifacts(s Scope, name string, lock Lock, input, source, gitRef, stageCache, stageRoot string) ([]artifact, func() error, error) { + var artifacts []artifact + lockStageDir, err := os.MkdirTemp(filepath.Dir(s.LockPath), ".stoat-lock-stage-*") + if err != nil { + return nil, func() error { return nil }, err + } + cleanup := func() error { + return os.RemoveAll(lockStageDir) + } + lockStage := filepath.Join(lockStageDir, "stoat.lock") + if err := SaveLock(lockStage, lock); err != nil { + _ = cleanup() + return nil, func() error { return nil }, err + } + if mode, modeErr := existingFileMode(s.LockPath); modeErr != nil { + _ = cleanup() + return nil, func() error { return nil }, modeErr + } else if mode != 0 { + if err := os.Chmod(lockStage, mode); err != nil { + _ = cleanup() + return nil, func() error { return nil }, err + } + } + artifacts = append(artifacts, artifact{target: s.LockPath, stage: lockStage}) + + if s.Name != "project" { + target, err := recipeTarget(s.CachePath, name) + if err != nil { + _ = cleanup() + return nil, func() error { return nil }, err + } + return append(artifacts, artifact{target: target, stage: stageCache, isDir: true}), cleanup, nil + } + decls, err := s.Decls() + if err != nil { + _ = cleanup() + return nil, func() error { return nil }, err + } + if _, _, isURL := ParseRef(input); isURL { + decls[name] = Decl{Source: source, Ref: gitRef} + } else { + decls[name] = Decl{Ref: gitRef} + } + var project map[string]any + if err := tomlx.Decode(s.ConfigPath, &project, tomlx.Warn(io.Discard)); err != nil { + _ = cleanup() + return nil, func() error { return nil }, err + } + raw := make(map[string]any, len(decls)) + for n, d := range decls { + if d.Source == "" { + raw[n] = d.Ref + } else { + raw[n] = map[string]any{"source": d.Source, "ref": d.Ref} + } + } + project["recipes"] = raw + projectStageDir, err := os.MkdirTemp(filepath.Dir(s.ConfigPath), ".stoat-project-stage-*") + if err != nil { + _ = cleanup() + return nil, func() error { return nil }, err + } + oldMode, err := existingFileMode(s.ConfigPath) + if err != nil { + _ = os.RemoveAll(projectStageDir) + _ = cleanup() + return nil, func() error { return nil }, err + } + projectStage := filepath.Join(projectStageDir, "stoat.toml") + if err := tomlx.Encode(projectStage, project); err != nil { + _ = os.RemoveAll(projectStageDir) + _ = cleanup() + return nil, func() error { return nil }, err + } + if oldMode != 0 { + if err := os.Chmod(projectStage, oldMode); err != nil { + _ = os.RemoveAll(projectStageDir) + _ = cleanup() + return nil, func() error { return nil }, err + } + } + cleanup = joinCleanup(cleanup, func() error { return os.RemoveAll(projectStageDir) }) + artifacts = append(artifacts, artifact{target: s.ConfigPath, stage: projectStage}) + + ignore, err := prepareIgnoreArtifact(s.Dir, stageRoot) + if err != nil { + _ = cleanup() + return nil, func() error { return nil }, err + } + if ignore != nil { + artifacts = append(artifacts, *ignore) + } + target, err := recipeTarget(s.CachePath, name) + if err != nil { + _ = cleanup() + return nil, func() error { return nil }, err + } + return append(artifacts, artifact{target: target, stage: stageCache, isDir: true}), cleanup, nil +} + +func joinCleanup(first, second func() error) func() error { + return func() error { + err1 := first() + err2 := second() + if err1 != nil && err2 != nil { + return fmt.Errorf("%v; %v", err1, err2) + } + if err1 != nil { + return err1 + } + return err2 + } +} + +func prepareIgnoreArtifact(dir, stageRoot string) (*artifact, error) { + gitPath := filepath.Join(dir, ".git") + if _, err := os.Stat(gitPath); err != nil { + if os.IsNotExist(err) { + return nil, nil + } + return nil, err + } + path := filepath.Join(dir, ".gitignore") + body, err := os.ReadFile(path) + if err != nil && !os.IsNotExist(err) { + return nil, err + } + for _, line := range strings.Split(string(body), "\n") { + if strings.TrimSpace(line) == ".stoat/" { + return nil, nil + } + } + if len(body) > 0 && !strings.HasSuffix(string(body), "\n") { + body = append(body, '\n') + } + body = append(body, ".stoat/\n"...) + stage := filepath.Join(stageRoot, ".gitignore") + mode := os.FileMode(0o644) + if info, statErr := os.Stat(path); statErr == nil { + mode = info.Mode().Perm() + } else if !os.IsNotExist(statErr) { + return nil, statErr + } + if err := os.WriteFile(stage, body, mode); err != nil { + return nil, err + } + if err := os.Chmod(stage, mode); err != nil { + return nil, err + } + return &artifact{target: path, stage: stage}, nil +} + +func publishArtifacts(artifacts []artifact) error { + published := make([]publishedArtifact, 0, len(artifacts)) + for _, a := range artifacts { + if err := os.MkdirAll(filepath.Dir(a.target), 0o755); err != nil { + return rollbackPublished(published, err) + } + p := publishedArtifact{artifact: a} + if _, err := os.Lstat(a.target); err == nil { + backup, backupErr := makeBackupPath(filepath.Dir(a.target)) + if backupErr != nil { + return rollbackPublished(published, backupErr) + } + if err := transactionRename(a.target, backup); err != nil { + _ = os.Remove(backup) + return rollbackPublished(published, err) + } + p.backup, p.oldExists = backup, true + } else if !os.IsNotExist(err) { + return rollbackPublished(published, err) + } + p.targetMoved = p.oldExists + published = append(published, p) + if a.remove { + p.published = true + published[len(published)-1] = p + continue + } + if err := transactionRename(a.stage, a.target); err != nil { + return rollbackPublished(published, err) + } + p.published = true + published[len(published)-1] = p + } + for _, p := range published { + if p.oldExists { + if err := transactionRemoveBackup(p.target, p.backup); err != nil { + return fmt.Errorf("published recipe changes; old artifact backup %s remains: %w", p.backup, err) + } + } + } + return nil +} + +func rollbackPublished(published []publishedArtifact, cause error) error { + var rollbackErr error + for i := len(published) - 1; i >= 0; i-- { + p := published[i] + if !p.targetMoved && !p.published { + continue + } + if p.published { + if err := os.RemoveAll(p.target); err != nil && rollbackErr == nil { + rollbackErr = err + continue + } + } + if p.oldExists { + if err := transactionRename(p.backup, p.target); err != nil && rollbackErr == nil { + rollbackErr = err + } + } + } + if rollbackErr != nil { + return fmt.Errorf("%w; rollback failed: %v", cause, rollbackErr) + } + return cause +} + +func makeBackupPath(parent string) (string, error) { + dir, err := os.MkdirTemp(parent, ".stoat-recipe-backup-*") + if err != nil { + return "", err + } + if err := os.Remove(dir); err != nil { + return "", err + } + return dir, nil +} + +func lockScope(s Scope) (func() error, error) { + return lockScopeMode(s, true) +} + +// LockAll resolves every project declaration to a fresh commit and persists +// the result under the scope coordination lock. It does not touch the cache. +func LockAll(s Scope) (Lock, error) { + // Resolve index names before entering the scope lock. The index lock and a + // scope lock have no shared acquisition order, so they must never nest. + decls, err := s.Decls() + if err != nil { + return Lock{}, err + } + indexEntries := make(map[string]IndexEntry) + if s.Name == "global" { + old, loadErr := s.Lock() + if loadErr != nil { + return Lock{}, loadErr + } + decls = make(map[string]Decl, len(old.Recipes)) + for name, entry := range old.Recipes { + decls[name] = Decl{Source: entry.Source, Ref: entry.Ref} + } + } + for name, decl := range decls { + if err := validateRecipeName(name); err != nil { + return Lock{}, err + } + if decl.Source == "" { + entry, ok, lookupErr := IndexLookup(name) + if lookupErr != nil { + return Lock{}, lookupErr + } + if !ok { + return Lock{}, fmt.Errorf("no recipe %q in the index; run stoat recipe search %s", name, name) + } + indexEntries[name] = entry + } + } + unlock, err := lockScope(s) + if err != nil { + return Lock{}, err + } + defer func() { _ = unlock() }() + old, err := s.Lock() + if err != nil { + return Lock{}, err + } + decls, err = s.Decls() + if err != nil { + return Lock{}, err + } + if s.Name == "global" { + decls = make(map[string]Decl, len(old.Recipes)) + for name, entry := range old.Recipes { + decls[name] = Decl{Source: entry.Source, Ref: entry.Ref} + } + } + names := make([]string, 0, len(decls)) + for name := range decls { + names = append(names, name) + } + sort.Strings(names) + next := Lock{Schema: LockSchema, Recipes: make(map[string]LockEntry, len(names))} + for _, name := range names { + decl := decls[name] + source := decl.Source + if source == "" { + entry, ok := indexEntries[name] + if !ok { + return Lock{}, fmt.Errorf("no recipe %q in the index; run stoat recipe search %s", name, name) + } + source = entry.Source + } + commit, resolveErr := resolveCommit(source, decl.Ref) + if resolveErr != nil { + return Lock{}, refError(source, decl.Ref, resolveErr) + } + added := time.Now().UTC().Format(time.RFC3339) + if previous, ok := old.Recipes[name]; ok && previous.Added != "" { + added = previous.Added + } + next.Recipes[name] = LockEntry{Source: source, Ref: decl.Ref, Commit: commit, Added: added} + } + if err := s.Save(next); err != nil { + return Lock{}, err + } + return s.Lock() +} + +func resolveCommit(source, gitRef string) (string, error) { + tmp, err := os.MkdirTemp("", "stoat-lock-") + if err != nil { + return "", err + } + defer func() { _ = os.RemoveAll(tmp) }() + dst := filepath.Join(tmp, "recipe") + if err := gitx.Clone(source, gitRef, dst); err != nil { + return "", refError(source, gitRef, err) + } + return gitx.RevParse(dst, "HEAD") +} + +// Sync stages every missing or mismatched checkout, validates them, then +// publishes the complete cache transaction. Project caches remove stray +// entries; the global cache leaves non-remote recipes untouched. +func Sync(s Scope) error { + unlock, err := lockScope(s) + if err != nil { + return err + } + defer func() { _ = unlock() }() + return syncLocked(s) +} + +func syncLocked(s Scope) error { + lock, err := s.Lock() + if err != nil { + return err + } + if err := validateRemoteLock(lock); err != nil { + return err + } + if err := os.MkdirAll(filepath.Dir(s.CachePath), 0o755); err != nil { + return err + } + stageRoot, err := os.MkdirTemp(filepath.Dir(s.CachePath), ".stoat-recipe-sync-*") + if err != nil { + return err + } + defer func() { _ = os.RemoveAll(stageRoot) }() + + names := make([]string, 0, len(lock.Recipes)) + for name := range lock.Recipes { + names = append(names, name) + } + sort.Strings(names) + artifacts := make([]artifact, 0, len(names)) + for _, name := range names { + entry := lock.Recipes[name] + dst, targetErr := recipeTarget(s.CachePath, name) + if targetErr != nil { + return targetErr + } + matches, matchErr := cacheMatches(dst, name, entry) + if matchErr != nil { + return matchErr + } + if matches { + continue + } + stage, targetErr := recipeTarget(stageRoot, name) + if targetErr != nil { + return targetErr + } + if err := gitx.CloneFull(entry.Source, stage); err != nil { + return err + } + if err := gitx.Checkout(stage, entry.Commit); err != nil { + return err + } + if err := ValidateTree(stage, name); err != nil { + return err + } + artifacts = append(artifacts, artifact{target: dst, stage: stage, isDir: true}) + } + if s.Name == "project" { + entries, readErr := os.ReadDir(s.CachePath) + if readErr != nil && !os.IsNotExist(readErr) { + return readErr + } + for _, entry := range entries { + if !entry.IsDir() { + continue + } + if _, ok := lock.Recipes[entry.Name()]; !ok { + target, targetErr := containedPath(s.CachePath, entry.Name()) + if targetErr != nil { + return targetErr + } + artifacts = append(artifacts, artifact{target: target, remove: true, isDir: true}) + } + } + } + return publishArtifacts(artifacts) +} + +func cacheMatches(path, name string, entry LockEntry) (bool, error) { + if _, err := os.Lstat(path); err != nil { + if os.IsNotExist(err) { + return false, nil + } + return false, err + } + dirty, err := gitx.Dirty(path) + if err != nil { + return false, err + } + if dirty { + return false, fmt.Errorf("%s: %w; copy it to a local recipe first", name, ErrDirty) + } + have, err := gitx.RevParse(path, "HEAD") + if err != nil { + return false, err + } + if have != entry.Commit { + return false, nil + } + if err := ValidateTree(path, name); err != nil { + return false, err + } + return true, nil +} + +// StaleLock reports the first project declaration that is absent or differs +// from its lock pin. Global scope has no declaration and is never stale here. +func StaleLock(s Scope) (string, bool, error) { + unlock, err := lockScopeMode(s, false) + if err != nil { + return "", false, err + } + name, stale, readErr := staleLockLocked(s) + unlockErr := unlock() + if readErr != nil { + return "", false, readErr + } + return name, stale, unlockErr +} + +func staleLockLocked(s Scope) (string, bool, error) { + decls, err := s.Decls() + if err != nil { + return "", false, err + } + lock, err := s.Lock() + if err != nil { + return "", false, err + } + names := make([]string, 0, len(decls)) + for name := range decls { + names = append(names, name) + } + sort.Strings(names) + for _, name := range names { + decl := decls[name] + entry, ok := lock.Recipes[name] + if !ok || entry.Ref != decl.Ref || (decl.Source != "" && entry.Source != decl.Source) { + return name, true, nil + } + } + return "", false, nil +} + +func cacheCurrentScope(scope Scope, lock Lock) (bool, error) { + entries, err := os.ReadDir(scope.CachePath) + if err != nil && !os.IsNotExist(err) { + return false, err + } + if os.IsNotExist(err) { + return len(lock.Recipes) == 0, nil + } + for name, entry := range lock.Recipes { + path, targetErr := recipeTarget(scope.CachePath, name) + if targetErr != nil { + return false, targetErr + } + if _, statErr := os.Lstat(path); statErr != nil { + if os.IsNotExist(statErr) { + return false, nil + } + return false, statErr + } + dirty, dirtyErr := gitx.Dirty(path) + if dirtyErr != nil { + return false, dirtyErr + } + if dirty { + return false, fmt.Errorf("%s: %w; copy it to a local recipe first", name, ErrDirty) + } + have, revErr := gitx.RevParse(path, "HEAD") + if revErr != nil { + return false, revErr + } + if have != entry.Commit { + return false, nil + } + if validateErr := ValidateTree(path, name); validateErr != nil { + return false, validateErr + } + } + for _, entry := range entries { + if !entry.IsDir() { + continue + } + if _, ok := lock.Recipes[entry.Name()]; !ok { + return false, nil + } + } + return true, nil +} + +func repairProjectLocked(scope Scope) error { + if scope.Name != "project" { + return nil + } + if name, stale, err := staleLockLocked(scope); err != nil { + return err + } else if stale { + return fmt.Errorf("%w: %s; run stoat recipe lock", ErrLockOutOfDate, name) + } + lock, err := scope.Lock() + if err != nil { + return err + } + if err := validateRemoteLock(lock); err != nil { + return err + } + fresh, err := cacheCurrentScope(scope, lock) + if err != nil { + return err + } + if fresh { + return nil + } + return syncLocked(scope) +} + +// SyncProject performs the project read-may-repair transaction under one +// exclusive scope lock. Global state is not touched outside project scope. +func SyncProject() error { + scope, err := ScopeFor(false) + if err != nil { + return err + } + if scope.Name != "project" { + return nil + } + unlock, err := lockScope(scope) + if err != nil { + return err + } + repairErr := repairProjectLocked(scope) + unlockErr := unlock() + if repairErr != nil { + return repairErr + } + return unlockErr +} + +// Update stages every requested ref, validates every resulting tree, and +// publishes the cache and lock together. A dirty or unreadable checkout is +// never replaced implicitly. +func Update(s Scope, names []string) ([]LockEntry, error) { + unlock, err := lockScope(s) + if err != nil { + return nil, err + } + defer func() { _ = unlock() }() + lock, err := s.Lock() + if err != nil { + return nil, err + } + if err := validateRemoteLock(lock); err != nil { + return nil, err + } + if len(names) == 0 { + for name := range lock.Recipes { + names = append(names, name) + } + sort.Strings(names) + } + stageRoot, err := os.MkdirTemp(filepath.Dir(s.CachePath), ".stoat-recipe-update-*") + if err != nil { + return nil, err + } + defer func() { _ = os.RemoveAll(stageRoot) }() + artifacts := make([]artifact, 0, len(names)+1) + result := make([]LockEntry, 0, len(names)) + for _, name := range names { + if err := validateRecipeName(name); err != nil { + return nil, err + } + entry, ok := lock.Recipes[name] + if !ok { + return nil, fmt.Errorf("%s is not a remote recipe in this scope", name) + } + dir, targetErr := recipeTarget(s.CachePath, name) + if targetErr != nil { + return nil, targetErr + } + if _, statErr := os.Lstat(dir); statErr != nil { + if !os.IsNotExist(statErr) { + return nil, statErr + } + } else { + dirty, dirtyErr := gitx.Dirty(dir) + if dirtyErr != nil { + return nil, dirtyErr + } + if dirty { + return nil, fmt.Errorf("%s: %w; copy it to a local recipe first", name, ErrDirty) + } + } + stage, targetErr := recipeTarget(stageRoot, name) + if targetErr != nil { + return nil, targetErr + } + if err := gitx.Clone(entry.Source, entry.Ref, stage); err != nil { + return nil, refError(entry.Source, entry.Ref, err) + } + if err := ValidateTree(stage, name); err != nil { + return nil, err + } + commit, err := gitx.RevParse(stage, "HEAD") + if err != nil { + return nil, err + } + entry.Name = name + entry.Commit = commit + persistEntry := entry + persistEntry.Name = "" + lock.Recipes[name] = persistEntry + result = append(result, entry) + artifacts = append(artifacts, artifact{target: dir, stage: stage, isDir: true}) + } + lockArtifact, lockCleanup, err := prepareLockArtifact(s, lock, stageRoot) + if err != nil { + return nil, err + } + defer func() { _ = lockCleanup() }() + artifacts = append(artifacts, lockArtifact) + if err := publishArtifacts(artifacts); err != nil { + return nil, err + } + return result, nil +} + +// Remove stages a lock, declaration, and cache removal before publishing any +// of them. A malformed declaration or persistence failure leaves all three +// active artifacts in place. +func Remove(s Scope, name string) error { + return removeChecked(s, name, nil) +} + +// RemoveChecked revalidates a caller's mutable precondition while holding the +// scope lock, after any confirmation prompt has already returned. +func RemoveChecked(s Scope, name string, users func() ([]string, error)) error { + return removeChecked(s, name, users) +} + +func removeChecked(s Scope, name string, users func() ([]string, error)) error { + if err := validateRecipeName(name); err != nil { + return err + } + unlock, err := lockScope(s) + if err != nil { + return err + } + defer func() { _ = unlock() }() + lock, err := s.Lock() + if err != nil { + return err + } + if err := validateRemoteLock(lock); err != nil { + return err + } + if _, ok := lock.Recipes[name]; !ok { + return fmt.Errorf("%s is not a remote recipe in this scope", name) + } + if users != nil { + current, usersErr := users() + if usersErr != nil { + return usersErr + } + if len(current) > 0 { + return &RemoveInUse{Users: current} + } + } + delete(lock.Recipes, name) + stageRoot, err := os.MkdirTemp(filepath.Dir(s.CachePath), ".stoat-recipe-remove-*") + if err != nil { + return err + } + defer func() { _ = os.RemoveAll(stageRoot) }() + lockArtifact, lockCleanup, err := prepareLockArtifact(s, lock, stageRoot) + if err != nil { + return err + } + defer func() { _ = lockCleanup() }() + target, err := recipeTarget(s.CachePath, name) + if err != nil { + return err + } + artifacts := []artifact{lockArtifact, {target: target, remove: true, isDir: true}} + if s.Name == "project" { + projectArtifact, projectCleanup, err := prepareProjectWithout(s, name, stageRoot) + if err != nil { + return err + } + defer func() { _ = projectCleanup() }() + artifacts = append(artifacts, projectArtifact) + } + return publishArtifacts(artifacts) +} + +func prepareLockArtifact(s Scope, lock Lock, stageRoot string) (artifact, func() error, error) { + stageDir, err := os.MkdirTemp(filepath.Dir(s.LockPath), ".stoat-lock-stage-*") + if err != nil { + return artifact{}, func() error { return nil }, err + } + cleanup := func() error { return os.RemoveAll(stageDir) } + stage := filepath.Join(stageDir, "stoat.lock") + if err := SaveLock(stage, lock); err != nil { + _ = cleanup() + return artifact{}, func() error { return nil }, err + } + if mode, err := existingFileMode(s.LockPath); err != nil { + _ = cleanup() + return artifact{}, func() error { return nil }, err + } else if mode != 0 { + if err := os.Chmod(stage, mode); err != nil { + _ = cleanup() + return artifact{}, func() error { return nil }, err + } + } + return artifact{target: s.LockPath, stage: stage}, cleanup, nil +} + +func prepareProjectWithout(s Scope, name, stageRoot string) (artifact, func() error, error) { + var project map[string]any + if err := tomlx.Decode(s.ConfigPath, &project, tomlx.Warn(io.Discard)); err != nil { + return artifact{}, func() error { return nil }, err + } + raw, ok := project["recipes"].(map[string]any) + if !ok { + return artifact{}, func() error { return nil }, fmt.Errorf("%s: recipes must be a table", ProjectFile) + } + delete(raw, name) + project["recipes"] = raw + stageDir, err := os.MkdirTemp(filepath.Dir(s.ConfigPath), ".stoat-project-stage-*") + if err != nil { + return artifact{}, func() error { return nil }, err + } + cleanup := func() error { return os.RemoveAll(stageDir) } + stage := filepath.Join(stageDir, "stoat.toml") + if err := tomlx.Encode(stage, project); err != nil { + _ = cleanup() + return artifact{}, func() error { return nil }, err + } + if mode, err := existingFileMode(s.ConfigPath); err != nil { + _ = cleanup() + return artifact{}, func() error { return nil }, err + } else if mode != 0 { + if err := os.Chmod(stage, mode); err != nil { + _ = cleanup() + return artifact{}, func() error { return nil }, err + } + } + return artifact{target: s.ConfigPath, stage: stage}, cleanup, nil +} diff --git a/internal/recipes/remote_boundary_final_test.go b/internal/recipes/remote_boundary_final_test.go new file mode 100644 index 00000000..8dc62358 --- /dev/null +++ b/internal/recipes/remote_boundary_final_test.go @@ -0,0 +1,229 @@ +package recipes + +import ( + "fmt" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/novusedge/stoat/internal/testutil" +) + +func unsafeExternalRecipeNames() []string { + return []string{"", ".", "..", ".hidden", "recipe/part", "../escape", "recipe name", "recipe\\part"} +} + +func writeRawLock(t *testing.T, path, name, source, ref, commit string) { + t.Helper() + writeFile(t, path, fmt.Sprintf( + "schema = 1\n\n[recipes.%q]\nsource = %q\nref = %q\ncommit = %q\nadded = \"2026-09-05T00:00:00Z\"\n", + name, source, ref, commit, + )) +} + +func assertOutsideRecipeSentinels(t *testing.T, before map[string]string) { + t.Helper() + for path, want := range before { + if got := readFile(t, path); got != want { + t.Fatalf("outside-cache sentinel %s changed from %q to %q", path, want, got) + } + } +} + +func assertRejectedBeforeGit(t *testing.T, err error, source string) { + t.Helper() + lower := strings.ToLower(err.Error()) + if strings.Contains(lower, "git clone") || (source != "" && strings.Contains(err.Error(), source)) { + t.Fatalf("rejection happened after a Git attempt: %v", err) + } +} + +func TestRefreshIndexRejectsMalformedDecodedNamesBeforePublication(t *testing.T) { + for _, name := range unsafeExternalRecipeNames() { + t.Run(fmt.Sprintf("name-%q", name), func(t *testing.T) { + home := t.TempDir() + t.Setenv("STOAT_HOME", home) + t.Chdir(t.TempDir()) + valid := testutil.GitRepo(t, map[string]string{ + "index.toml": "schema = 1\n\n[recipes.good]\nsource = \"local\"\ndescription = \"good\"\nos = [\"alpine\"]\n", + }) + t.Setenv("STOAT_INDEX", valid) + if err := RefreshIndex(true); err != nil { + t.Fatal(err) + } + sentinel := filepath.Join(home, "escape", "keep.txt") + writeFile(t, sentinel, "caller-owned\n") + malformed := testutil.GitRepo(t, map[string]string{ + "index.toml": fmt.Sprintf( + "schema = 1\n\n[recipes.%q]\nsource = \"local\"\ndescription = \"bad\"\nos = [\"alpine\"]\n", + name, + ), + }) + t.Setenv("STOAT_INDEX", malformed) + if err := RefreshIndex(true); err == nil { + t.Fatalf("RefreshIndex() accepted malformed index name %q", name) + } + idx, err := LoadIndex() + if err != nil { + t.Fatal(err) + } + if _, ok := idx.Recipes["good"]; !ok { + t.Fatalf("failed refresh replaced the active index: %+v", idx.Recipes) + } + assertOutsideRecipeSentinels(t, map[string]string{sentinel: "caller-owned\n"}) + }) + } +} + +func TestIndexLookupRejectsMalformedCallerNamesBeforeFetching(t *testing.T) { + home := t.TempDir() + t.Setenv("STOAT_HOME", home) + t.Chdir(t.TempDir()) + missingIndex := filepath.Join(t.TempDir(), "not-a-repository") + t.Setenv("STOAT_INDEX", missingIndex) + for _, name := range unsafeExternalRecipeNames() { + t.Run(fmt.Sprintf("name-%q", name), func(t *testing.T) { + _, _, err := IndexLookup(name) + if err == nil { + t.Fatalf("IndexLookup(%q) accepted malformed caller name", name) + } + if strings.Contains(err.Error(), "git clone") || strings.Contains(err.Error(), missingIndex) { + t.Fatalf("IndexLookup(%q) fetched before rejecting the caller name: %v", name, err) + } + if _, statErr := os.Stat(missingIndex); !os.IsNotExist(statErr) { + t.Fatalf("IndexLookup(%q) created or changed the missing source: %v", name, statErr) + } + }) + } +} + +func TestRefreshIndexRejectsAnIndexEntryWithoutARequiredSource(t *testing.T) { + home := t.TempDir() + t.Setenv("STOAT_HOME", home) + t.Chdir(t.TempDir()) + good := testutil.GitRepo(t, map[string]string{ + "index.toml": "schema = 1\n\n[recipes.good]\nsource = \"local\"\ndescription = \"good\"\nos = [\"alpine\"]\n", + }) + t.Setenv("STOAT_INDEX", good) + if err := RefreshIndex(true); err != nil { + t.Fatal(err) + } + bad := testutil.GitRepo(t, map[string]string{ + "index.toml": "schema = 1\n\n[recipes.bad]\ndescription = \"missing source\"\nos = [\"alpine\"]\n", + }) + t.Setenv("STOAT_INDEX", bad) + if err := RefreshIndex(true); err == nil { + t.Fatal("RefreshIndex() accepted an index entry without source") + } + idx, err := LoadIndex() + if err != nil { + t.Fatal(err) + } + if _, ok := idx.Recipes["good"]; !ok { + t.Fatalf("missing-source refresh replaced the active index: %+v", idx.Recipes) + } +} + +func TestRemoteOperationsRejectMalformedExternalNamesBeforePublication(t *testing.T) { + operations := []struct { + name string + call func(Scope, string) error + }{ + {name: "LockAll", call: func(s Scope, _ string) error { _, err := LockAll(s); return err }}, + {name: "Sync", call: func(s Scope, _ string) error { return Sync(s) }}, + {name: "Update", call: func(s Scope, _ string) error { _, err := Update(s, nil); return err }}, + {name: "Remove", call: func(s Scope, name string) error { return Remove(s, name) }}, + } + for _, operation := range operations { + for _, name := range unsafeExternalRecipeNames() { + t.Run(operation.name+"/name-"+fmt.Sprintf("%q", name), func(t *testing.T) { + home := remoteRoot(t) + src := filepath.Join(home, "missing-source.git") + commit := strings.Repeat("a", 40) + s, err := ScopeFor(true) + if err != nil { + t.Fatal(err) + } + outside := filepath.Join(home, "escape", "keep.txt") + writeFile(t, outside, "caller-owned\n") + writeRawLock(t, s.LockPath, name, src, "main", commit) + lockBefore := readFile(t, s.LockPath) + err = operation.call(s, name) + if err == nil { + t.Fatalf("%s accepted malformed external name %q", operation.name, name) + } + assertRejectedBeforeGit(t, err, src) + if got := readFile(t, s.LockPath); got != lockBefore { + t.Fatalf("%s changed the active lock after rejecting %q", operation.name, name) + } + assertOutsideRecipeSentinels(t, map[string]string{outside: "caller-owned\n"}) + }) + } + } +} + +func TestRemoteOperationsRejectMissingSourceAndMalformedCommitPins(t *testing.T) { + for _, tc := range []struct { + name string + commit string + }{ + {name: "missing source", commit: strings.Repeat("a", 40)}, + {name: "short commit", commit: strings.Repeat("b", 39)}, + } { + for _, operation := range []struct { + name string + call func(Scope) error + }{ + {name: "Sync", call: Sync}, + {name: "Update", call: func(s Scope) error { _, err := Update(s, nil); return err }}, + {name: "Remove", call: func(s Scope) error { return Remove(s, "demo") }}, + } { + t.Run(operation.name+"/"+tc.name, func(t *testing.T) { + home := remoteRoot(t) + source := "" + if tc.name == "short commit" { + source = namedRecipeRepo(t, "demo", "demo") + } + s, err := ScopeFor(true) + if err != nil { + t.Fatal(err) + } + writeFile(t, filepath.Join(home, "escape", "keep.txt"), "caller-owned\n") + writeRawLock(t, s.LockPath, "demo", source, "main", tc.commit) + lockBefore := readFile(t, s.LockPath) + if err := operation.call(s); err == nil { + t.Fatalf("%s accepted %s", operation.name, tc.name) + } else { + assertRejectedBeforeGit(t, err, source) + } + if got := readFile(t, s.LockPath); got != lockBefore { + t.Fatalf("%s changed the active lock after rejecting %s", operation.name, tc.name) + } + assertOutsideRecipeSentinels(t, map[string]string{ + filepath.Join(home, "escape", "keep.txt"): "caller-owned\n", + }) + }) + } + } +} + +func TestGoRemoteRecipeNamesRemainCompatibleWithExistingComponents(t *testing.T) { + for _, name := range []string{"Foo", "a_b"} { + t.Run(name, func(t *testing.T) { + remoteRoot(t) + src := namedRecipeRepo(t, name, name) + s, err := ScopeFor(true) + if err != nil { + t.Fatal(err) + } + if _, err := Add(s, src, false); err != nil { + t.Fatalf("Add(%q) = %v, want existing Go-safe recipe names to remain valid", name, err) + } + path, scope, ok, err := ResolvePath(name) + if err != nil || !ok || scope != "global" || filepath.Base(path) != name { + t.Fatalf("ResolvePath(%q) = %q, %q, %v, %v", name, path, scope, ok, err) + } + }) + } +} diff --git a/internal/recipes/remote_test.go b/internal/recipes/remote_test.go new file mode 100644 index 00000000..f7f5ff24 --- /dev/null +++ b/internal/recipes/remote_test.go @@ -0,0 +1,1227 @@ +package recipes + +import ( + "errors" + "fmt" + "os" + "path/filepath" + "reflect" + "strings" + "testing" + + "github.com/novusedge/stoat/internal/testutil" +) + +func remoteRoot(t *testing.T) string { + t.Helper() + home := t.TempDir() + t.Setenv("STOAT_HOME", home) + t.Chdir(t.TempDir()) + // Every remote-recipe test binds the index to a local bare repository. + index := testutil.GitRepo(t, map[string]string{"index.toml": "schema = 1\n"}) + t.Setenv("STOAT_INDEX", index) + return home +} + +func namedRecipeRepo(t *testing.T, repoName, manifestName string) string { + t.Helper() + src := testutil.GitRepo(t, map[string]string{ + "recipe.toml": fmt.Sprintf("schema = 3\nname = %q\nscript = \"install.sh\"\n", manifestName), + "install.sh": "#!/bin/sh\nset -e\necho v1\n", + }) + dst := filepath.Join(filepath.Dir(src), repoName+".git") + if err := os.Rename(src, dst); err != nil { + t.Fatal(err) + } + return dst +} + +func localIndex(t *testing.T, entries string) string { + t.Helper() + index := testutil.GitRepo(t, map[string]string{"index.toml": "schema = 1\n\n" + entries}) + t.Setenv("STOAT_INDEX", index) + return index +} + +func rewriteGitURLToBareRepo(t *testing.T, prefix, repo string) { + t.Helper() + emptyGlobal := filepath.Join(t.TempDir(), "empty.gitconfig") + if err := os.WriteFile(emptyGlobal, nil, 0o600); err != nil { + t.Fatal(err) + } + base := "file://" + filepath.ToSlash(filepath.Dir(repo)) + "/" + t.Setenv("GIT_CONFIG_NOSYSTEM", "1") + t.Setenv("GIT_CONFIG_GLOBAL", emptyGlobal) + t.Setenv("GIT_CONFIG_COUNT", "1") + t.Setenv("GIT_CONFIG_KEY_0", "url."+base+".insteadOf") + t.Setenv("GIT_CONFIG_VALUE_0", prefix) +} + +func TestParseRefSplitsIndexNamesURLsAndSCPLikeSources(t *testing.T) { + tests := []struct { + input, source, ref string + url bool + }{ + {input: "tailscale", source: "tailscale"}, + {input: "tailscale@v1.2", source: "tailscale", ref: "v1.2"}, + {input: "tailscale@feature/x", source: "tailscale", ref: "feature/x"}, + {input: "https://example.test/stoat-demo@v1.2", source: "https://example.test/stoat-demo", ref: "v1.2", url: true}, + {input: "https://example.test/x/stoat-demo@feature/x", source: "https://example.test/x/stoat-demo", ref: "feature/x", url: true}, + {input: "ssh://git@example.test/x/stoat-demo.git", source: "ssh://git@example.test/x/stoat-demo.git", url: true}, + {input: "ssh://git@example.test/x/stoat-demo.git@feature/topic", source: "ssh://git@example.test/x/stoat-demo.git", ref: "feature/topic", url: true}, + {input: "https://user@example.test/x/stoat-demo.git", source: "https://user@example.test/x/stoat-demo.git", url: true}, + {input: "https://user@example.test/x/stoat-demo.git@feature/topic", source: "https://user@example.test/x/stoat-demo.git", ref: "feature/topic", url: true}, + {input: "git@example.test:x/stoat-demo.git", source: "git@example.test:x/stoat-demo.git", url: true}, + {input: "git@example.test:x/stoat-demo.git@main", source: "git@example.test:x/stoat-demo.git", ref: "main", url: true}, + {input: "git@example.test:x/stoat-demo.git@feature/x", source: "git@example.test:x/stoat-demo.git", ref: "feature/x", url: true}, + {input: "./x/stoat-demo.git@feature/x", source: "./x/stoat-demo.git", ref: "feature/x", url: true}, + } + for _, tc := range tests { + source, ref, isURL := ParseRef(tc.input) + if source != tc.source || ref != tc.ref || isURL != tc.url { + t.Errorf("ParseRef(%q) = %q, %q, %v; want %q, %q, %v", tc.input, source, ref, isURL, tc.source, tc.ref, tc.url) + } + } +} + +func TestAddFromURLReturnsNameAndPinsTheRequestedTag(t *testing.T) { + home := remoteRoot(t) + src := namedRecipeRepo(t, "demo", "demo") + commit := testutil.GitCommit(t, src, map[string]string{ + "recipe.toml": "schema = 3\nname = \"demo\"\nscript = \"install.sh\"\n\n[params.channel]\ntype = \"enum\"\nvalues = [\"stable\", \"test\"]\ndefault = \"stable\"\n", + "install.sh": "#!/bin/sh\nset -e\necho v2\n", + }, "v1.2") + preview, previewDir, err := Preview(src, "v1.2") + if previewDir != "" { + defer func() { + if err := os.RemoveAll(previewDir); err != nil { + t.Errorf("remove Preview temp dir: %v", err) + } + }() + } + if err != nil { + t.Fatalf("Preview() = %v", err) + } + if preview.Name != "demo" || preview.Params["channel"].Default != "stable" { + t.Fatalf("Preview() = %+v, want schema-3 channel params", preview) + } + + s, err := ScopeFor(true) + if err != nil { + t.Fatal(err) + } + entry, err := Add(s, src+"@v1.2", false) + if err != nil { + t.Fatal(err) + } + if entry.Name != "demo" || entry.Source != src || entry.Ref != "v1.2" || entry.Commit != commit { + t.Fatalf("entry = %+v, want name/source/ref/commit for demo", entry) + } + lock, err := LoadLock(filepath.Join(home, "stoat.lock")) + if err != nil { + t.Fatal(err) + } + if got := lock.Recipes["demo"].Commit; got != commit { + t.Errorf("lock commit = %q, want %q", got, commit) + } + if strings.Contains(readFile(t, filepath.Join(home, "stoat.lock")), "name =") { + t.Fatal("caller-only LockEntry.Name was serialized into stoat.lock") + } + path, scope, ok, err := ResolvePath("demo") + if err != nil || !ok || scope != "global" || filepath.Base(path) != "demo" { + t.Fatalf("ResolvePath(demo) = %q, %q, %v, %v", path, scope, ok, err) + } +} + +func TestAddByIndexNameUsesTheLocalIndexAndDefaultBranch(t *testing.T) { + remoteRoot(t) + src := namedRecipeRepo(t, "demo", "demo") + localIndex(t, fmt.Sprintf("[recipes.demo]\nsource = %q\ndescription = \"demo\"\nos = [\"alpine\"]\n", src)) + + s, err := ScopeFor(true) + if err != nil { + t.Fatal(err) + } + entry, err := Add(s, "demo", false) + if err != nil { + t.Fatal(err) + } + if entry.Name != "demo" || entry.Source != src || entry.Ref != "" || len(entry.Commit) != 40 { + t.Fatalf("entry = %+v, want index source and default branch pin", entry) + } +} + +func TestAddReportsUnknownNamesAndMissingRefs(t *testing.T) { + tests := []struct { + name string + inputKind string + wantSource string + wantRef string + }{ + {name: "unknown index name", inputKind: "unknown"}, + {name: "missing ref over HTTPS", inputKind: "https-missing-ref", wantSource: "https://github.com/x/stoat-tailscale", wantRef: "does-not-exist"}, + {name: "missing ref over scp", inputKind: "scp-missing-ref", wantSource: "git@github.com:x/stoat-tailscale", wantRef: "does-not-exist"}, + {name: "transport failure", inputKind: "transport", wantSource: "missing.git", wantRef: "main"}, + } + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + remoteRoot(t) + input := "tailscal" + switch tc.inputKind { + case "https-missing-ref", "scp-missing-ref": + src := namedRecipeRepo(t, "stoat-tailscale", "stoat-tailscale") + repo := filepath.Join(filepath.Dir(src), "stoat-tailscale") + if err := os.Rename(src, repo); err != nil { + t.Fatal(err) + } + prefix := "https://github.com/x/" + if tc.inputKind == "scp-missing-ref" { + prefix = "git@github.com:x/" + } + rewriteGitURLToBareRepo(t, prefix, repo) + input = tc.wantSource + "@" + tc.wantRef + case "transport": + input = tc.wantSource + "@" + tc.wantRef + } + s, err := ScopeFor(true) + if err != nil { + t.Fatal(err) + } + _, err = Add(s, input, false) + if err == nil { + t.Fatalf("Add(%q) = nil, want an error", input) + } + switch tc.inputKind { + case "unknown": + want := `no recipe "tailscal" in the index; run stoat recipe search tailscal` + if err.Error() != want { + t.Fatalf("Add(%q) = %q, want %q", input, err, want) + } + case "https-missing-ref", "scp-missing-ref": + want := `x/stoat-tailscale: no tag or branch "does-not-exist"` + if err.Error() != want { + t.Fatalf("Add(%q) = %q, want exact missing-ref error %q", input, err, want) + } + case "transport": + for _, want := range []string{"git clone", tc.wantSource, tc.wantRef} { + if !strings.Contains(err.Error(), want) { + t.Fatalf("Add(%q) = %q, want transport context %q", input, err, want) + } + } + if strings.Contains(err.Error(), "no tag or branch") { + t.Fatalf("transport failure was misclassified as a missing ref: %v", err) + } + } + }) + } +} + +func TestAddValidationFailurePreservesProjectState(t *testing.T) { + remoteRoot(t) + project := t.TempDir() + t.Chdir(project) + writeFile(t, filepath.Join(project, "stoat.toml"), "[vm]\nname = \"kept\"\n\n[recipes]\n") + if err := os.Chmod(filepath.Join(project, "stoat.toml"), 0o600); err != nil { + t.Fatal(err) + } + good := namedRecipeRepo(t, "demo", "demo") + s, err := ScopeFor(false) + if err != nil { + t.Fatal(err) + } + if _, err := Add(s, good, false); err != nil { + t.Fatal(err) + } + + lockBefore, err := os.ReadFile(s.LockPath) + if err != nil { + t.Fatal(err) + } + declBefore, err := os.ReadFile(s.ConfigPath) + if err != nil { + t.Fatal(err) + } + cacheScript := filepath.Join(s.CachePath, "demo", "install.sh") + cacheBefore, err := os.ReadFile(cacheScript) + if err != nil { + t.Fatal(err) + } + configInfo, err := os.Stat(s.ConfigPath) + if err != nil { + t.Fatal(err) + } + + // The source basename still resolves to demo, but its manifest deliberately + // names another recipe so validation fails after staging. + bad := namedRecipeRepo(t, "demo", "other") + if _, err := Add(s, bad, true); err == nil || !strings.Contains(err.Error(), `named "other"`) { + t.Fatalf("Add(invalid replacement) = %v, want a name mismatch", err) + } + lockAfter, _ := os.ReadFile(s.LockPath) + declAfter, _ := os.ReadFile(s.ConfigPath) + cacheAfter, _ := os.ReadFile(cacheScript) + afterInfo, _ := os.Stat(s.ConfigPath) + if !reflect.DeepEqual(lockAfter, lockBefore) || !reflect.DeepEqual(declAfter, declBefore) || !reflect.DeepEqual(cacheAfter, cacheBefore) { + t.Fatal("validation failure changed the active lock, declaration, or cache") + } + if afterInfo.Mode().Perm() != configInfo.Mode().Perm() { + t.Fatalf("stoat.toml mode = %o, want %o", afterInfo.Mode().Perm(), configInfo.Mode().Perm()) + } + if !strings.Contains(string(declAfter), `name = "kept"`) { + t.Fatal("unrelated project value disappeared") + } + if _, err := os.Stat(filepath.Join(s.CachePath, "demo", "recipe.toml")); err != nil { + t.Fatalf("old cache disappeared: %v", err) + } + + // A malformed alternate declaration target must reject Add while preparing + // declaration state, before any active artifact is published. + brokenConfig := filepath.Join(project, "broken.toml") + writeFile(t, brokenConfig, "[recipes\n") + persistenceScope := s + persistenceScope.ConfigPath = brokenConfig + if _, err := Add(persistenceScope, good, true); err == nil { + t.Fatal("Add() with an invalid declaration target = nil, want persistence failure") + } + lockAfterPersistence, _ := os.ReadFile(s.LockPath) + declAfterPersistence, _ := os.ReadFile(s.ConfigPath) + cacheAfterPersistence, _ := os.ReadFile(cacheScript) + if !reflect.DeepEqual(lockAfterPersistence, lockBefore) || !reflect.DeepEqual(declAfterPersistence, declBefore) || !reflect.DeepEqual(cacheAfterPersistence, cacheBefore) { + t.Fatal("persistence failure changed the active lock, declaration, or cache") + } +} + +func TestProjectAddPreservesGitignoreAndLeavesNoRootCoordinationFile(t *testing.T) { + remoteRoot(t) + project := t.TempDir() + t.Chdir(project) + writeFile(t, filepath.Join(project, "stoat.toml"), "[vm]\nname = \"keep\"\n\n[recipes]\n") + if err := os.Mkdir(filepath.Join(project, ".git"), 0o755); err != nil { + t.Fatal(err) + } + gitignore := filepath.Join(project, ".gitignore") + if err := os.WriteFile(gitignore, []byte("keep/\n"), 0o660); err != nil { + t.Fatal(err) + } + if err := os.Chmod(gitignore, 0o660); err != nil { + t.Fatal(err) + } + src := namedRecipeRepo(t, "demo", "demo") + s, err := ScopeFor(false) + if err != nil { + t.Fatal(err) + } + if _, err := Add(s, src, false); err != nil { + t.Fatal(err) + } + info, err := os.Stat(gitignore) + if err != nil { + t.Fatal(err) + } + if info.Mode().Perm() != 0o660 { + t.Fatalf(".gitignore mode = %o, want 660", info.Mode().Perm()) + } + body := readFile(t, gitignore) + if body != "keep/\n.stoat/\n" || strings.Count(body, ".stoat/") != 1 { + t.Fatalf(".gitignore = %q, want preserved content and one .stoat/ line", body) + } + if _, err := os.Stat(filepath.Join(project, ".stoat-recipe.lock")); !os.IsNotExist(err) { + t.Fatalf("root coordination artifact stat = %v, want absent", err) + } +} + +type transactionSnapshot struct { + lock, project, gitignore, cache []byte + lockMode, projectMode, ignoreMode, cacheMode os.FileMode +} + +func setupTransactionalAdd(t *testing.T) (Scope, string, transactionSnapshot) { + t.Helper() + remoteRoot(t) + project := t.TempDir() + t.Chdir(project) + projectPath := filepath.Join(project, "stoat.toml") + writeFile(t, projectPath, "[vm]\nname = \"keep\"\n\n[recipes]\n") + if err := os.Chmod(projectPath, 0o600); err != nil { + t.Fatal(err) + } + if err := os.Mkdir(filepath.Join(project, ".git"), 0o755); err != nil { + t.Fatal(err) + } + ignorePath := filepath.Join(project, ".gitignore") + if err := os.WriteFile(ignorePath, []byte("keep/\n"), 0o660); err != nil { + t.Fatal(err) + } + if err := os.Chmod(ignorePath, 0o660); err != nil { + t.Fatal(err) + } + src := namedRecipeRepo(t, "demo", "demo") + scope, err := ScopeFor(false) + if err != nil { + t.Fatal(err) + } + if _, err := Add(scope, src, false); err != nil { + t.Fatal(err) + } + testutil.GitCommit(t, src, map[string]string{"install.sh": "#!/bin/sh\necho v2\n"}, "") + cachePath := filepath.Join(scope.CachePath, "demo", "install.sh") + snapshot := transactionSnapshot{ + lock: readBytes(t, scope.LockPath), + project: readBytes(t, scope.ConfigPath), + gitignore: readBytes(t, ignorePath), + cache: readBytes(t, cachePath), + } + snapshot.lockMode = fileMode(t, scope.LockPath) + snapshot.projectMode = fileMode(t, scope.ConfigPath) + snapshot.ignoreMode = fileMode(t, ignorePath) + snapshot.cacheMode = fileMode(t, cachePath) + return scope, src, snapshot +} + +func readBytes(t *testing.T, path string) []byte { + t.Helper() + b, err := os.ReadFile(path) + if err != nil { + t.Fatal(err) + } + return b +} + +func fileMode(t *testing.T, path string) os.FileMode { + t.Helper() + info, err := os.Stat(path) + if err != nil { + t.Fatal(err) + } + return info.Mode().Perm() +} + +func assertTransactionSnapshot(t *testing.T, s Scope, before transactionSnapshot) { + t.Helper() + if got := readBytes(t, s.LockPath); !reflect.DeepEqual(got, before.lock) { + t.Fatal("lock bytes changed during failed transaction") + } + if got := readBytes(t, s.ConfigPath); !reflect.DeepEqual(got, before.project) { + t.Fatal("project declaration changed during failed transaction") + } + if got := readBytes(t, filepath.Join(s.Dir, ".gitignore")); !reflect.DeepEqual(got, before.gitignore) { + t.Fatal(".gitignore changed during failed transaction") + } + if got := readBytes(t, filepath.Join(s.CachePath, "demo", "install.sh")); !reflect.DeepEqual(got, before.cache) { + t.Fatal("active cache changed during failed transaction") + } + if got := fileMode(t, s.LockPath); got != before.lockMode { + t.Fatalf("lock mode = %o, want %o", got, before.lockMode) + } + if got := fileMode(t, s.ConfigPath); got != before.projectMode { + t.Fatalf("project mode = %o, want %o", got, before.projectMode) + } + if got := fileMode(t, filepath.Join(s.Dir, ".gitignore")); got != before.ignoreMode { + t.Fatalf(".gitignore mode = %o, want %o", got, before.ignoreMode) + } + if got := fileMode(t, filepath.Join(s.CachePath, "demo", "install.sh")); got != before.cacheMode { + t.Fatalf("cache script mode = %o, want %o", got, before.cacheMode) + } +} + +func hasRecoverableBackup(t *testing.T, dir string) bool { + t.Helper() + entries, err := os.ReadDir(dir) + if err != nil { + t.Fatal(err) + } + for _, entry := range entries { + if strings.HasPrefix(entry.Name(), ".stoat-recipe-backup-") { + return true + } + } + return false +} + +func TestAddPostPublicationFailuresRestoreOrReportCommittedState(t *testing.T) { + t.Run("rollback restores every artifact", func(t *testing.T) { + s, src, before := setupTransactionalAdd(t) + cacheTarget := filepath.Join(s.CachePath, "demo") + injected := errors.New("injected cache publication failure") + transactionRename = func(old, new string) error { + if new == cacheTarget { + body, readErr := os.ReadFile(filepath.Join(old, "install.sh")) + if readErr == nil && strings.Contains(string(body), "v2") { + return injected + } + } + return os.Rename(old, new) + } + t.Cleanup(func() { transactionRename = os.Rename }) + if _, err := Add(s, src, true); err == nil || !errors.Is(err, injected) { + t.Fatalf("Add() = %v, want injected publication failure", err) + } + assertTransactionSnapshot(t, s, before) + }) + + t.Run("rollback failure reports both causes and keeps backup", func(t *testing.T) { + s, src, before := setupTransactionalAdd(t) + cacheTarget := filepath.Join(s.CachePath, "demo") + publicationErr := errors.New("injected cache publication failure") + rollbackErr := errors.New("injected cache restore failure") + transactionRename = func(old, new string) error { + if new == cacheTarget { + if strings.HasPrefix(filepath.Base(old), ".stoat-recipe-backup-") { + return rollbackErr + } + return publicationErr + } + return os.Rename(old, new) + } + t.Cleanup(func() { transactionRename = os.Rename }) + err := error(nil) + _, err = Add(s, src, true) + if err == nil || !errors.Is(err, publicationErr) || !strings.Contains(err.Error(), rollbackErr.Error()) || !strings.Contains(err.Error(), "rollback failed") { + t.Fatalf("Add() = %v, want publication and rollback causes", err) + } + if _, statErr := os.Stat(cacheTarget); !os.IsNotExist(statErr) { + t.Fatalf("cache target stat = %v, want absent after failed restore", statErr) + } + if !hasRecoverableBackup(t, filepath.Dir(cacheTarget)) { + t.Fatal("rollback failure did not preserve a recoverable cache backup") + } + if !reflect.DeepEqual(readBytes(t, s.LockPath), before.lock) || !reflect.DeepEqual(readBytes(t, s.ConfigPath), before.project) { + t.Fatal("rollback failure changed an earlier published lock or declaration") + } + }) + + t.Run("cleanup failure reports committed state and keeps backup", func(t *testing.T) { + s, src, before := setupTransactionalAdd(t) + injected := errors.New("injected backup cleanup failure") + transactionRemoveBackup = func(target, backup string) error { + if target == s.LockPath { + return injected + } + return os.RemoveAll(backup) + } + t.Cleanup(func() { + transactionRemoveBackup = func(target, backup string) error { _ = target; return os.RemoveAll(backup) } + }) + err := error(nil) + _, err = Add(s, src, true) + if err == nil || !strings.Contains(err.Error(), "published recipe changes") { + t.Fatalf("Add() = %v, want committed-state cleanup report", err) + } + if string(readBytes(t, filepath.Join(s.CachePath, "demo", "install.sh"))) == string(before.cache) { + t.Fatal("cleanup failure did not leave the new committed cache") + } + if !hasRecoverableBackup(t, filepath.Dir(s.LockPath)) { + t.Fatal("cleanup failure did not leave a recoverable lock backup") + } + }) +} + +func TestRemovePostPublicationFailureRestoresEveryArtifact(t *testing.T) { + s, _, before := setupTransactionalAdd(t) + projectTarget := s.ConfigPath + injected := errors.New("injected project publication failure") + transactionRename = func(old, new string) error { + if new == projectTarget { + body, readErr := os.ReadFile(old) + if readErr == nil && !strings.Contains(string(body), "demo") { + return injected + } + } + return os.Rename(old, new) + } + t.Cleanup(func() { transactionRename = os.Rename }) + if err := Remove(s, "demo"); err == nil || !errors.Is(err, injected) { + t.Fatalf("Remove() = %v, want injected publication failure", err) + } + assertTransactionSnapshot(t, s, before) +} + +func TestLockAllUsesDeclarationsAndDropsRemovedEntries(t *testing.T) { + remoteRoot(t) + project := t.TempDir() + t.Chdir(project) + src := namedRecipeRepo(t, "demo", "demo") + writeFile(t, filepath.Join(project, "stoat.toml"), fmt.Sprintf("[recipes]\ndemo = { source = %q, ref = \"main\" }\n", src)) + s, err := ScopeFor(false) + if err != nil { + t.Fatal(err) + } + if err := SaveLock(s.LockPath, Lock{Recipes: map[string]LockEntry{ + "gone": {Source: "unused", Ref: "main", Commit: strings.Repeat("b", 40)}, + }}); err != nil { + t.Fatal(err) + } + + got, err := LockAll(s) + if err != nil { + t.Fatal(err) + } + entry, ok := got.Recipes["demo"] + if !ok || len(entry.Commit) != 40 || entry.Source != src || entry.Ref != "main" { + t.Fatalf("LockAll() = %+v, want demo declaration pinned", got.Recipes) + } + if _, ok := got.Recipes["gone"]; ok { + t.Fatalf("LockAll() retained removed declaration: %+v", got.Recipes) + } + persisted, err := s.Lock() + if err != nil { + t.Fatal(err) + } + if !reflect.DeepEqual(persisted, got) { + t.Fatalf("persisted LockAll result = %+v, want returned %+v", persisted, got) + } + if strings.Contains(readFile(t, s.LockPath), "name =") { + t.Fatal("LockAll persisted a caller-only name field") + } + cacheSentinel := filepath.Join(s.CachePath, "unrelated", "keep.txt") + writeFile(t, cacheSentinel, "keep\n") + beforeCache := readFile(t, cacheSentinel) + if _, err := LockAll(s); err != nil { + t.Fatal(err) + } + if afterCache := readFile(t, cacheSentinel); afterCache != beforeCache { + t.Fatalf("LockAll mutated cache sentinel from %q to %q", beforeCache, afterCache) + } +} + +func TestSyncBuildsTheProjectCacheAndRemovesProjectStrays(t *testing.T) { + remoteRoot(t) + project := t.TempDir() + t.Chdir(project) + src := namedRecipeRepo(t, "demo", "demo") + writeFile(t, filepath.Join(project, "stoat.toml"), fmt.Sprintf("[recipes]\ndemo = { source = %q, ref = \"main\" }\n", src)) + s, err := ScopeFor(false) + if err != nil { + t.Fatal(err) + } + entry := LockEntry{Source: src, Ref: "main", Commit: currentHead(t, src)} + if err := SaveLock(s.LockPath, Lock{Recipes: map[string]LockEntry{"demo": entry}}); err != nil { + t.Fatal(err) + } + writeFile(t, filepath.Join(s.CachePath, "stray", "recipe.toml"), "name = \"stray\"\nscript = \"install.sh\"\n") + writeFile(t, filepath.Join(s.CachePath, "stray", "install.sh"), "#!/bin/sh\n") + writeFile(t, filepath.Join(s.CachePath, ".stray", "recipe.toml"), "name = \".stray\"\nscript = \"install.sh\"\n") + writeFile(t, filepath.Join(s.CachePath, ".stray", "install.sh"), "#!/bin/sh\n") + + if err := Sync(s); err != nil { + t.Fatal(err) + } + if _, err := os.Stat(filepath.Join(s.CachePath, "demo", "recipe.toml")); err != nil { + t.Fatalf("demo cache missing after Sync: %v", err) + } + if _, err := os.Stat(filepath.Join(s.CachePath, "stray")); !os.IsNotExist(err) { + t.Fatalf("project stray stat = %v, want not exist", err) + } + if _, err := os.Stat(filepath.Join(s.CachePath, ".stray")); !os.IsNotExist(err) { + t.Fatalf("hidden project stray stat = %v, want not exist", err) + } +} + +func TestSyncFailureLeavesThePreviouslyActiveCache(t *testing.T) { + remoteRoot(t) + project := t.TempDir() + t.Chdir(project) + src := namedRecipeRepo(t, "demo", "demo") + writeFile(t, filepath.Join(project, "stoat.toml"), fmt.Sprintf("[recipes]\ndemo = { source = %q, ref = \"main\" }\n", src)) + s, err := ScopeFor(false) + if err != nil { + t.Fatal(err) + } + oldCommit := currentHead(t, src) + clone := testutil.GitClone(t, src) + cache := filepath.Join(s.CachePath, "demo") + if err := os.MkdirAll(filepath.Dir(cache), 0o755); err != nil { + t.Fatal(err) + } + if err := os.Rename(clone, cache); err != nil { + t.Fatal(err) + } + if err := SaveLock(s.LockPath, Lock{Recipes: map[string]LockEntry{"demo": { + Source: src, Ref: "main", Commit: oldCommit, + }}}); err != nil { + t.Fatal(err) + } + newCommit := testutil.GitCommit(t, src, map[string]string{ + "recipe.toml": "schema = 3\nname = \"other\"\nscript = \"install.sh\"\n", + "install.sh": "#!/bin/sh\necho broken\n", + }, "") + if err := SaveLock(s.LockPath, Lock{Recipes: map[string]LockEntry{"demo": { + Source: src, Ref: "main", Commit: newCommit, + }}}); err != nil { + t.Fatal(err) + } + + err = Sync(s) + if err == nil || !strings.Contains(err.Error(), `named "other"`) { + t.Fatalf("Sync() = %v, want validation failure", err) + } + if got := currentHead(t, cache); got != oldCommit { + t.Fatalf("cache HEAD = %s, want old commit %s after failed Sync", got, oldCommit) + } + if got := readFile(t, filepath.Join(cache, "recipe.toml")); !strings.Contains(got, `name = "demo"`) { + t.Fatalf("active cache manifest changed after failed Sync: %s", got) + } +} + +func TestSyncRefusesDirtyOrBrokenLockOwnedCaches(t *testing.T) { + for _, brokenGit := range []bool{false, true} { + name := "dirty cache" + if brokenGit { + name = "broken git cache" + } + t.Run(name, func(t *testing.T) { + remoteRoot(t) + src := namedRecipeRepo(t, "demo", "demo") + s, err := ScopeFor(true) + if err != nil { + t.Fatal(err) + } + if _, err := Add(s, src, false); err != nil { + t.Fatal(err) + } + cache := filepath.Join(s.CachePath, "demo") + if brokenGit { + if err := os.RemoveAll(filepath.Join(cache, ".git")); err != nil { + t.Fatal(err) + } + } else if err := os.WriteFile(filepath.Join(cache, "install.sh"), []byte("edited\n"), 0o644); err != nil { + t.Fatal(err) + } + lockBefore, err := os.ReadFile(s.LockPath) + if err != nil { + t.Fatal(err) + } + cacheBefore, err := os.ReadFile(filepath.Join(cache, "install.sh")) + if err != nil { + t.Fatal(err) + } + err = Sync(s) + if err == nil { + t.Fatal("Sync() = nil, want a dirty/probe error") + } + if !brokenGit && !errors.Is(err, ErrDirty) { + t.Fatalf("Sync() = %v, want ErrDirty", err) + } + if brokenGit && strings.Contains(err.Error(), "local changes") { + t.Fatalf("broken-git probe was converted to ErrDirty: %v", err) + } + if brokenGit && !strings.Contains(strings.ToLower(err.Error()), "not a git repository") { + t.Fatalf("Sync() = %v, want the original git probe context", err) + } + lockAfter, _ := os.ReadFile(s.LockPath) + cacheAfter, _ := os.ReadFile(filepath.Join(cache, "install.sh")) + if !reflect.DeepEqual(lockAfter, lockBefore) || !reflect.DeepEqual(cacheAfter, cacheBefore) { + t.Fatal("dirty/probe failure changed the active lock or cache") + } + }) + } +} + +func TestStaleLockComparesRefAndExplicitSource(t *testing.T) { + cases := []struct { + name string + lockSource string + lockRef string + wantStale bool + }{ + {name: "missing entry", wantStale: true}, + {name: "changed ref", lockRef: "old", wantStale: true}, + {name: "changed source", lockSource: "/different/source", lockRef: "main", wantStale: true}, + {name: "matching pin", lockSource: "/source", lockRef: "main", wantStale: false}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + remoteRoot(t) + project := t.TempDir() + t.Chdir(project) + writeFile(t, filepath.Join(project, "stoat.toml"), "[recipes]\ndemo = { source = \"/source\", ref = \"main\" }\n") + s, err := ScopeFor(false) + if err != nil { + t.Fatal(err) + } + entries := map[string]LockEntry{} + if tc.name != "missing entry" { + entries["demo"] = LockEntry{Source: tc.lockSource, Ref: tc.lockRef, Commit: strings.Repeat("c", 40)} + } + if err := SaveLock(s.LockPath, Lock{Recipes: entries}); err != nil { + t.Fatal(err) + } + name, stale, err := StaleLock(s) + if err != nil { + t.Fatal(err) + } + if stale != tc.wantStale || (stale && name != "demo") { + t.Fatalf("StaleLock() = %q, %v; want demo, %v", name, stale, tc.wantStale) + } + }) + } +} + +func TestGlobalSyncLeavesAHandMadeGitRecipeAlone(t *testing.T) { + home := remoteRoot(t) + src := namedRecipeRepo(t, "remote", "remote") + s, err := ScopeFor(true) + if err != nil { + t.Fatal(err) + } + if err := SaveLock(s.LockPath, Lock{Recipes: map[string]LockEntry{"remote": { + Source: src, Ref: "main", Commit: currentHead(t, src), + }}}); err != nil { + t.Fatal(err) + } + local := filepath.Join(home, "recipes", "handmade") + writeFile(t, filepath.Join(local, "recipe.toml"), "schema = 3\nname = \"handmade\"\nscript = \"install.sh\"\n") + writeFile(t, filepath.Join(local, "install.sh"), "#!/bin/sh\n") + if err := os.MkdirAll(filepath.Join(local, ".git"), 0o755); err != nil { + t.Fatal(err) + } + + if err := Sync(s); err != nil { + t.Fatal(err) + } + if _, err := os.Stat(filepath.Join(s.CachePath, "remote", "recipe.toml")); err != nil { + t.Fatalf("remote cache missing after global Sync: %v", err) + } + if _, err := os.Stat(filepath.Join(local, "recipe.toml")); err != nil { + t.Fatalf("hand-made local recipe was removed: %v", err) + } +} + +func currentHead(t *testing.T, dir string) string { + t.Helper() + work := testutil.GitClone(t, dir) + // GitCommit's helper already establishes a work tree, but this read-only + // fixture avoids adding a production git API solely for tests. + b, err := os.ReadFile(filepath.Join(work, ".git", "HEAD")) + if err != nil { + t.Fatal(err) + } + ref := strings.TrimSpace(string(b)) + if !strings.HasPrefix(ref, "ref: ") { + return ref + } + refPath := strings.TrimPrefix(ref, "ref: ") + b, err = os.ReadFile(filepath.Join(work, ".git", filepath.FromSlash(refPath))) + if err != nil { + t.Fatal(err) + } + return strings.TrimSpace(string(b)) +} + +func TestUpdateMovesTheCommitAndReturnsItsName(t *testing.T) { + remoteRoot(t) + src := namedRecipeRepo(t, "demo", "demo") + s, err := ScopeFor(true) + if err != nil { + t.Fatal(err) + } + before, err := Add(s, src, false) + if err != nil { + t.Fatal(err) + } + newCommit := testutil.GitCommit(t, src, map[string]string{ + "install.sh": "#!/bin/sh\nset -e\necho newer\n", + }, "") + + got, err := Update(s, nil) + if err != nil { + t.Fatal(err) + } + if len(got) != 1 || got[0].Name != "demo" || got[0].Commit != newCommit || got[0].Commit == before.Commit { + t.Fatalf("Update() = %+v, before = %+v, want the new named pin", got, before) + } + lock, err := s.Lock() + if err != nil { + t.Fatal(err) + } + if lock.Recipes["demo"].Commit != newCommit { + t.Fatalf("lock commit = %q, want %q", lock.Recipes["demo"].Commit, newCommit) + } + if strings.Contains(readFile(t, s.LockPath), "name =") { + t.Fatal("caller-only LockEntry.Name was serialized into stoat.lock after Update") + } +} + +func TestAddAcceptsAFullCommitRefAndChecksOutThePinnedCommit(t *testing.T) { + remoteRoot(t) + src := namedRecipeRepo(t, "demo", "demo") + commit := testutil.GitCommit(t, src, map[string]string{ + "recipe.toml": "schema = 3\nname = \"demo\"\ndescription = \"commit ref\"\nscript = \"install.sh\"\n", + "install.sh": "#!/bin/sh\necho commit\n", + }, "") + s, err := ScopeFor(true) + if err != nil { + t.Fatal(err) + } + + entry, err := Add(s, src+"@"+commit, false) + if err != nil { + t.Fatalf("Add() with full commit ref = %v", err) + } + if entry.Name != "demo" || entry.Ref != commit || entry.Commit != commit { + t.Fatalf("Add() entry = %+v, want the requested full commit pinned", entry) + } + lock, err := s.Lock() + if err != nil { + t.Fatal(err) + } + if got := lock.Recipes["demo"]; got.Ref != commit || got.Commit != commit { + t.Fatalf("persisted commit-ref entry = %+v, want ref and commit %s", got, commit) + } + if got := currentHead(t, filepath.Join(s.CachePath, "demo")); got != commit { + t.Fatalf("cache HEAD = %s, want %s", got, commit) + } +} + +func TestLockAllAcceptsAFullCommitRefAndPersistsThePin(t *testing.T) { + remoteRoot(t) + project := t.TempDir() + t.Chdir(project) + src := namedRecipeRepo(t, "demo", "demo") + commit := testutil.GitCommit(t, src, map[string]string{ + "install.sh": "#!/bin/sh\necho commit\n", + }, "") + writeFile(t, filepath.Join(project, "stoat.toml"), fmt.Sprintf( + "[recipes]\ndemo = { source = %q, ref = %q }\n", src, commit)) + s, err := ScopeFor(false) + if err != nil { + t.Fatal(err) + } + + got, err := LockAll(s) + if err != nil { + t.Fatalf("LockAll() with full commit ref = %v", err) + } + entry, ok := got.Recipes["demo"] + if !ok || entry.Ref != commit || entry.Commit != commit { + t.Fatalf("LockAll() = %+v, want demo ref and commit %s", got.Recipes, commit) + } + persisted, err := s.Lock() + if err != nil { + t.Fatal(err) + } + if persisted.Recipes["demo"] != entry { + t.Fatalf("persisted LockAll entry = %+v, want %+v", persisted.Recipes["demo"], entry) + } +} + +func TestUpdateAcceptsAFullCommitRefAndChecksOutTheNewPin(t *testing.T) { + remoteRoot(t) + src := namedRecipeRepo(t, "demo", "demo") + s, err := ScopeFor(true) + if err != nil { + t.Fatal(err) + } + if _, err := Add(s, src, false); err != nil { + t.Fatal(err) + } + commit := testutil.GitCommit(t, src, map[string]string{ + "install.sh": "#!/bin/sh\necho commit\n", + }, "") + lock, err := s.Lock() + if err != nil { + t.Fatal(err) + } + entry := lock.Recipes["demo"] + entry.Ref = commit + lock.Recipes["demo"] = entry + if err := s.Save(lock); err != nil { + t.Fatal(err) + } + + got, err := Update(s, []string{"demo"}) + if err != nil { + t.Fatalf("Update() with full commit ref = %v", err) + } + if len(got) != 1 || got[0].Name != "demo" || got[0].Ref != commit || got[0].Commit != commit { + t.Fatalf("Update() = %+v, want demo ref and commit %s", got, commit) + } + if got := currentHead(t, filepath.Join(s.CachePath, "demo")); got != commit { + t.Fatalf("cache HEAD = %s, want %s", got, commit) + } + persisted, err := s.Lock() + if err != nil { + t.Fatal(err) + } + if persisted.Recipes["demo"].Ref != commit || persisted.Recipes["demo"].Commit != commit { + t.Fatalf("persisted Update entry = %+v, want ref and commit %s", persisted.Recipes["demo"], commit) + } +} + +func TestAddMissingFullCommitRefKeepsSourceAndCommitContext(t *testing.T) { + remoteRoot(t) + src := namedRecipeRepo(t, "demo", "demo") + missing := strings.Repeat("f", 40) + s, err := ScopeFor(true) + if err != nil { + t.Fatal(err) + } + + _, err = Add(s, src+"@"+missing, false) + if err == nil { + t.Fatal("Add() with a missing full commit ref = nil, want an error") + } + assertMissingCommitContext(t, err, src, missing) +} + +func TestLockAllMissingFullCommitRefKeepsSourceAndCommitContext(t *testing.T) { + remoteRoot(t) + project := t.TempDir() + t.Chdir(project) + src := namedRecipeRepo(t, "demo", "demo") + missing := strings.Repeat("e", 40) + writeFile(t, filepath.Join(project, "stoat.toml"), fmt.Sprintf( + "[recipes]\ndemo = { source = %q, ref = %q }\n", src, missing)) + s, err := ScopeFor(false) + if err != nil { + t.Fatal(err) + } + + _, err = LockAll(s) + if err == nil { + t.Fatal("LockAll() with a missing full commit ref = nil, want an error") + } + assertMissingCommitContext(t, err, src, missing) +} + +func TestUpdateMissingFullCommitRefKeepsSourceAndCommitContext(t *testing.T) { + remoteRoot(t) + src := namedRecipeRepo(t, "demo", "demo") + missing := strings.Repeat("d", 40) + s, err := ScopeFor(true) + if err != nil { + t.Fatal(err) + } + if _, err := Add(s, src, false); err != nil { + t.Fatal(err) + } + lock, err := s.Lock() + if err != nil { + t.Fatal(err) + } + entry := lock.Recipes["demo"] + entry.Ref = missing + lock.Recipes["demo"] = entry + if err := s.Save(lock); err != nil { + t.Fatal(err) + } + + _, err = Update(s, []string{"demo"}) + if err == nil { + t.Fatal("Update() with a missing full commit ref = nil, want an error") + } + assertMissingCommitContext(t, err, src, missing) +} + +func assertMissingCommitContext(t *testing.T, err error, source, commit string) { + t.Helper() + label := strings.TrimSuffix(source, ".git") + labelWithoutLeadingSlash := strings.TrimPrefix(label, string(filepath.Separator)) + if !strings.Contains(err.Error(), commit) || (!strings.Contains(err.Error(), label) && !strings.Contains(err.Error(), labelWithoutLeadingSlash)) { + t.Fatalf("missing commit error = %q, want source/ref context %q and %q", err, label, commit) + } +} + +func TestUpdateRebuildsAConfirmedMissingCache(t *testing.T) { + remoteRoot(t) + src := namedRecipeRepo(t, "demo", "demo") + s, err := ScopeFor(true) + if err != nil { + t.Fatal(err) + } + if _, err := Add(s, src, false); err != nil { + t.Fatal(err) + } + if err := os.RemoveAll(filepath.Join(s.CachePath, "demo")); err != nil { + t.Fatal(err) + } + newCommit := testutil.GitCommit(t, src, map[string]string{ + "install.sh": "#!/bin/sh\necho rebuilt\n", + }, "") + + got, err := Update(s, nil) + if err != nil { + t.Fatal(err) + } + if len(got) != 1 || got[0].Name != "demo" || got[0].Commit != newCommit { + t.Fatalf("Update() = %+v, want the rebuilt named pin", got) + } + lock, err := s.Lock() + if err != nil { + t.Fatal(err) + } + if lock.Recipes["demo"].Commit != newCommit { + t.Fatalf("persisted commit = %q, want %q", lock.Recipes["demo"].Commit, newCommit) + } + if _, err := os.Stat(filepath.Join(s.CachePath, "demo", "recipe.toml")); err != nil { + t.Fatalf("missing cache was not rebuilt: %v", err) + } + if strings.Contains(readFile(t, s.LockPath), "name =") { + t.Fatal("caller-only LockEntry.Name was serialized after missing-cache Update") + } +} + +func TestUpdateRefusesADirtyTreeWithErrDirty(t *testing.T) { + remoteRoot(t) + src := namedRecipeRepo(t, "demo", "demo") + s, err := ScopeFor(true) + if err != nil { + t.Fatal(err) + } + if _, err := Add(s, src, false); err != nil { + t.Fatal(err) + } + writeFile(t, filepath.Join(s.CachePath, "demo", "install.sh"), "edited\n") + + _, err = Update(s, nil) + if !errors.Is(err, ErrDirty) || !strings.Contains(err.Error(), "demo: local changes; copy it to a local recipe first") { + t.Fatalf("Update() = %v, want ErrDirty with the copy-to-local message", err) + } + + t.Run("dirty probe error is propagated without overwrite", func(t *testing.T) { + remoteRoot(t) + src := namedRecipeRepo(t, "demo", "demo") + s, err := ScopeFor(true) + if err != nil { + t.Fatal(err) + } + if _, err := Add(s, src, false); err != nil { + t.Fatal(err) + } + lockBefore, err := os.ReadFile(s.LockPath) + if err != nil { + t.Fatal(err) + } + cacheScript := filepath.Join(s.CachePath, "demo", "install.sh") + cacheBefore, err := os.ReadFile(cacheScript) + if err != nil { + t.Fatal(err) + } + if err := os.RemoveAll(filepath.Join(s.CachePath, "demo", ".git")); err != nil { + t.Fatal(err) + } + _, err = Update(s, nil) + if err == nil || errors.Is(err, ErrDirty) || !strings.Contains(strings.ToLower(err.Error()), "not a git repository") { + t.Fatalf("Update() after Dirty probe failure = %v, want the git probe error", err) + } + lockAfter, _ := os.ReadFile(s.LockPath) + cacheAfter, _ := os.ReadFile(cacheScript) + if !reflect.DeepEqual(lockAfter, lockBefore) || !reflect.DeepEqual(cacheAfter, cacheBefore) { + t.Fatal("Dirty probe failure overwrote the active lock or cache") + } + }) +} + +func TestUpdateIsAllOrNothingAcrossMultipleRecipes(t *testing.T) { + remoteRoot(t) + srcA := namedRecipeRepo(t, "a", "a") + srcB := namedRecipeRepo(t, "b", "b") + s, err := ScopeFor(true) + if err != nil { + t.Fatal(err) + } + if _, err := Add(s, srcA, false); err != nil { + t.Fatal(err) + } + if _, err := Add(s, srcB, false); err != nil { + t.Fatal(err) + } + oldA := currentHead(t, filepath.Join(s.CachePath, "a")) + oldB := currentHead(t, filepath.Join(s.CachePath, "b")) + lockBefore, err := os.ReadFile(s.LockPath) + if err != nil { + t.Fatal(err) + } + newA := testutil.GitCommit(t, srcA, map[string]string{"install.sh": "#!/bin/sh\necho newer-a\n"}, "") + _ = newA + testutil.GitCommit(t, srcB, map[string]string{ + "recipe.toml": "schema = 3\nname = \"not-b\"\nscript = \"install.sh\"\n", + "install.sh": "#!/bin/sh\necho invalid-b\n", + }, "") + + _, err = Update(s, []string{"a", "b"}) + if err == nil || !strings.Contains(err.Error(), `named "not-b"`) { + t.Fatalf("Update(a,b) = %v, want the second validation failure", err) + } + if got := currentHead(t, filepath.Join(s.CachePath, "a")); got != oldA { + t.Fatalf("recipe a moved to %s after all-or-nothing failure; want %s", got, oldA) + } + if got := currentHead(t, filepath.Join(s.CachePath, "b")); got != oldB { + t.Fatalf("recipe b moved to %s after all-or-nothing failure; want %s", got, oldB) + } + lockAfter, _ := os.ReadFile(s.LockPath) + if !reflect.DeepEqual(lockAfter, lockBefore) { + t.Fatal("multi-update failure changed the active lock") + } +} + +func TestRemoveDropsProjectDeclarationLockAndCache(t *testing.T) { + remoteRoot(t) + project := t.TempDir() + t.Chdir(project) + writeFile(t, filepath.Join(project, "stoat.toml"), "[recipes]\n") + src := namedRecipeRepo(t, "demo", "demo") + s, err := ScopeFor(false) + if err != nil { + t.Fatal(err) + } + if _, err := Add(s, src, false); err != nil { + t.Fatal(err) + } + if err := Remove(s, "demo"); err != nil { + t.Fatal(err) + } + lock, err := s.Lock() + if err != nil { + t.Fatal(err) + } + if _, ok := lock.Recipes["demo"]; ok { + t.Fatal("lock still pins demo after Remove") + } + decls, err := s.Decls() + if err != nil { + t.Fatal(err) + } + if _, ok := decls["demo"]; ok { + t.Fatal("project declaration still contains demo after Remove") + } + if _, err := os.Stat(filepath.Join(s.CachePath, "demo")); !os.IsNotExist(err) { + t.Fatalf("cache stat = %v, want not exist", err) + } +} + +func TestRemoveFailureRestoresTheExistingLockAndCache(t *testing.T) { + remoteRoot(t) + project := t.TempDir() + t.Chdir(project) + writeFile(t, filepath.Join(project, "stoat.toml"), "[recipes]\n") + src := namedRecipeRepo(t, "demo", "demo") + s, err := ScopeFor(false) + if err != nil { + t.Fatal(err) + } + if _, err := Add(s, src, false); err != nil { + t.Fatal(err) + } + lockBefore, err := os.ReadFile(s.LockPath) + if err != nil { + t.Fatal(err) + } + cache := filepath.Join(s.CachePath, "demo") + if _, err := os.Stat(cache); err != nil { + t.Fatal(err) + } + // A malformed declaration must reject Remove before publication. The active + // lock and cache must remain in place. + writeFile(t, s.ConfigPath, "[recipes\n") + if err := Remove(s, "demo"); err == nil { + t.Fatal("Remove() = nil, want declaration validation failure") + } + lockAfter, _ := os.ReadFile(s.LockPath) + if !reflect.DeepEqual(lockAfter, lockBefore) { + t.Fatal("Remove failure changed the active lock") + } + if _, err := os.Stat(cache); err != nil { + t.Fatalf("Remove failure deleted active cache: %v", err) + } +} diff --git a/internal/recipes/roots.go b/internal/recipes/roots.go new file mode 100644 index 00000000..f3c88629 --- /dev/null +++ b/internal/recipes/roots.go @@ -0,0 +1,139 @@ +package recipes + +import ( + "os" + "path/filepath" + + "github.com/novusedge/stoat/internal/config" +) + +// Root is one directory searched for recipes, with the scope a name found +// there reports. +type Root struct { + Path string + Scope string // "project", "global", "local" or "bundled" +} + +// Roots lists every recipe directory in shadow order: the project cache first, +// then the home directory three times, once per label it can carry. +// +// The global cache and the bundled set share ~/.stoat/recipes, so the label +// comes from the bookkeeping beside them rather than from the path: the home +// lock names the remote recipes and .manifest names stoat's own copies. +// +// ScopeFor fails when the working directory cannot be read (os.Getwd) or +// stat'd for stoat.toml. Surfacing that error keeps a broken cwd from +// silently downgrading a project recipe to the global one. +func Roots() ([]Root, error) { + var roots []Root + s, err := ScopeFor(false) + if err != nil { + return nil, err + } + if s.Name == "project" { + roots = append(roots, Root{Path: s.CachePath, Scope: "project"}) + } + home := dir() + return append(roots, + Root{Path: home, Scope: "global"}, + Root{Path: home, Scope: "local"}, + Root{Path: home, Scope: "bundled"}, + ), nil +} + +// ResolvePath finds name's recipe directory and the scope it belongs to. The +// first root that both holds the directory and owns the name wins. +func ResolvePath(name string) (path, scope string, ok bool, err error) { + locks, err := lockRecipeScopes(false) + if err != nil { + return "", "", false, err + } + path, scope, ok, readErr := resolvePath(name) + unlockErr := unlockRecipeScopes(locks) + if readErr != nil { + return "", "", false, readErr + } + if unlockErr != nil { + return "", "", false, unlockErr + } + return path, scope, ok, nil +} + +func resolvePath(name string) (path, scope string, ok bool, err error) { + if err := validateRecipeName(name); err != nil { + return "", "", false, err + } + roots, err := Roots() + if err != nil { + return "", "", false, err + } + for _, root := range roots { + d, targetErr := recipeTarget(root.Path, name) + if targetErr != nil { + return "", "", false, targetErr + } + if _, statErr := os.Stat(filepath.Join(d, "recipe.toml")); statErr != nil { + if os.IsNotExist(statErr) { + continue + } + return "", "", false, statErr + } + owned, ownsErr := owns(root, name) + if ownsErr != nil { + return "", "", false, ownsErr + } + if owned { + return d, root.Scope, true, nil + } + } + return "", "", false, nil +} + +// ScopeOf returns name's scope, or "" when no root holds it. +func ScopeOf(name string) (string, error) { + locks, err := lockRecipeScopes(false) + if err != nil { + return "", err + } + _, scope, _, readErr := resolvePath(name) + unlockErr := unlockRecipeScopes(locks) + if readErr != nil { + return "", readErr + } + return scope, unlockErr +} + +// owns reports whether a name found under root carries root's label. +func owns(root Root, name string) (bool, error) { + switch root.Scope { + case "project": + return true, nil + case "global": + return homeLockHas(name) + case "bundled": + return bundledHas(name), nil + default: // local + remote, err := homeLockHas(name) + if err != nil { + return false, err + } + return !remote && !bundledHas(name), nil + } +} + +// homeLockHas reports whether the home lock pins name. +func homeLockHas(name string) (bool, error) { + lock, err := LoadLock(filepath.Join(config.Root(), "stoat.lock")) + if err != nil { + return false, err + } + _, ok := lock.Recipes[name] + return ok, nil +} + +// bundledHas reports whether stoat wrote name from the embedded set. The +// manifest is keyed by the path relative to the recipes root. +func bundledHas(name string) bool { + _, ok := readManifest()[name+"/recipe.toml"] + return ok +} diff --git a/internal/recipes/roots_test.go b/internal/recipes/roots_test.go new file mode 100644 index 00000000..a7cd9767 --- /dev/null +++ b/internal/recipes/roots_test.go @@ -0,0 +1,147 @@ +package recipes + +import ( + "os" + "path/filepath" + "strings" + "testing" +) + +// seedRecipe writes a minimal recipe directory under root. +func seedRecipe(t *testing.T, root, name string) { + t.Helper() + writeFile(t, filepath.Join(root, name, "recipe.toml"), + "name = \""+name+"\"\nscript = \"install.sh\"\n") + writeFile(t, filepath.Join(root, name, "install.sh"), "set -e\n") +} + +func TestRootsOrderAndShadowing(t *testing.T) { + home := t.TempDir() + t.Setenv("STOAT_HOME", home) + wd := t.TempDir() + writeFile(t, filepath.Join(wd, "stoat.toml"), "[recipes]\n") + t.Chdir(wd) + + seedRecipe(t, filepath.Join(home, "recipes"), "shared") + seedRecipe(t, filepath.Join(wd, ".stoat", "recipes"), "shared") + writeFile(t, filepath.Join(wd, ".stoat", "recipes", "shared", "recipe.toml"), + "name = \"shared\"\n") + writeFile(t, filepath.Join(wd, "stoat.lock"), "schema = 1\n[recipes.shared]\nsource = \"s\"\nref = \"main\"\ncommit = \"abc\"\nadded = \"now\"\n") + + path, scope, ok, err := ResolvePath("shared") + if err != nil { + t.Fatal(err) + } + if !ok { + t.Fatal("shared did not resolve") + } + if scope != "project" { + t.Errorf("scope = %q, want %q", scope, "project") + } + if path != filepath.Join(wd, ".stoat", "recipes", "shared") { + t.Errorf("path = %q, want the project cache", path) + } +} + +func TestScopeOfLabelsHomeEntries(t *testing.T) { + home := t.TempDir() + t.Setenv("STOAT_HOME", home) + t.Chdir(t.TempDir()) + + root := filepath.Join(home, "recipes") + seedRecipe(t, root, "mine") + seedRecipe(t, root, "docker") + seedRecipe(t, root, "tailscale") + writeFile(t, filepath.Join(root, ManifestName), "0 docker/recipe.toml\n") + writeFile(t, filepath.Join(home, "stoat.lock"), + "schema = 1\n[recipes.tailscale]\nsource = \"s\"\nref = \"v1\"\ncommit = \"abc\"\nadded = \"now\"\n") + + for name, want := range map[string]string{"mine": "local", "docker": "bundled", "tailscale": "global"} { + got, err := ScopeOf(name) + if err != nil { + t.Fatalf("ScopeOf(%q): %v", name, err) + } + if got != want { + t.Errorf("ScopeOf(%q) = %q, want %q", name, got, want) + } + } +} + +func TestGlobalLockErrorsSurfaceAtResolutionBoundaries(t *testing.T) { + for name, lock := range map[string]string{ + "newer": "schema = 2\n", + "corrupt": "schema = [\n", + } { + t.Run(name, func(t *testing.T) { + home := t.TempDir() + t.Setenv("STOAT_HOME", home) + t.Chdir(t.TempDir()) + seedRecipe(t, filepath.Join(home, "recipes"), "tailscale") + writeFile(t, filepath.Join(home, "stoat.lock"), lock) + + _, _, ok, err := ResolvePath("tailscale") + if err == nil { + t.Fatalf("ResolvePath error = nil, ok = %v", ok) + } + if name == "newer" && !strings.Contains(err.Error(), "schema 2 is newer than this stoat (1)") { + t.Fatalf("ResolvePath error = %v", err) + } + + scope, err := ScopeOf("tailscale") + if err == nil { + t.Fatalf("ScopeOf error = nil, scope = %q", scope) + } + if name == "newer" && !strings.Contains(err.Error(), "schema 2 is newer than this stoat (1)") { + t.Fatalf("ScopeOf error = %v", err) + } + }) + } +} + +// TestResolvePathFailsRatherThanDowngradeOnAnUnreadableCwd checks that a +// deleted working directory surfaces the Getwd error instead of silently +// falling back to the global recipe root. +func TestResolvePathFailsRatherThanDowngradeOnAnUnreadableCwd(t *testing.T) { + home := t.TempDir() + t.Setenv("STOAT_HOME", home) + seedRecipe(t, filepath.Join(home, "recipes"), "tailscale") + writeFile(t, filepath.Join(home, "stoat.lock"), + "schema = 1\n[recipes.tailscale]\nsource = \"s\"\nref = \"v1\"\ncommit = \"abc\"\nadded = \"now\"\n") + + gone := t.TempDir() + wd := filepath.Join(gone, "wd") + if err := os.Mkdir(wd, 0o755); err != nil { + t.Fatal(err) + } + t.Chdir(wd) + if err := os.RemoveAll(gone); err != nil { + t.Fatal(err) + } + + _, scope, ok, err := ResolvePath("tailscale") + if err == nil { + t.Fatalf("ResolvePath error = nil, scope = %q, ok = %v, want the deleted-cwd error", scope, ok) + } +} + +func TestListManifestsDoesNotFallThroughAnInvalidProjectShadow(t *testing.T) { + home := t.TempDir() + t.Setenv("STOAT_HOME", home) + wd := t.TempDir() + writeFile(t, filepath.Join(wd, "stoat.toml"), "[recipes]\n") + t.Chdir(wd) + + seedRecipe(t, filepath.Join(home, "recipes"), "shared") + seedRecipe(t, filepath.Join(home, "recipes"), "onlyhome") + seedRecipe(t, filepath.Join(wd, ".stoat", "recipes"), "shared") + writeFile(t, filepath.Join(wd, ".stoat", "recipes", "shared", "recipe.toml"), + "name = \"shared\"\n") + + ms, err := ListManifests() + if err != nil { + t.Fatal(err) + } + if len(ms) != 1 || ms[0].Name != "onlyhome" { + t.Fatalf("manifests = %d, want only the unshadowed home recipe: %+v", len(ms), ms) + } +} diff --git a/internal/recipes/scaffold.go b/internal/recipes/scaffold.go index 3e9535c5..be93a378 100644 --- a/internal/recipes/scaffold.go +++ b/internal/recipes/scaffold.go @@ -116,18 +116,3 @@ func New(name, osName, _ string) (string, error) { return recipeDir, nil } - -// Installed lists every recipe in the data root, whether bundled or written -// by the user. Unlike List it does not filter by os: this is "what is on -// disk", for a human looking for something to edit. -func Installed() ([]string, error) { - manifests, err := ListManifests() - if err != nil { - return nil, err - } - out := make([]string, len(manifests)) - for i, m := range manifests { - out[i] = m.Name - } - return out, nil -} diff --git a/internal/recipes/scaffold_test.go b/internal/recipes/scaffold_test.go index c8e989d0..bdf0c011 100644 --- a/internal/recipes/scaffold_test.go +++ b/internal/recipes/scaffold_test.go @@ -111,26 +111,6 @@ func TestNewRejectsBadNames(t *testing.T) { } } -func TestInstalledListsRecipes(t *testing.T) { - t.Setenv("STOAT_HOME", t.TempDir()) - if names, err := Installed(); err != nil || len(names) != 0 { - t.Errorf("Installed() on a fresh root = %v, %v; want empty", names, err) - } - if err := Install(); err != nil { - t.Fatal(err) - } - names, err := Installed() - if err != nil { - t.Fatal(err) - } - // Should list recipe names, not filenames - for _, want := range []string{"xfce", "docker", "devtools", "tailscale"} { - if !contains(names, want) { - t.Errorf("Installed() missing %q: %v", want, names) - } - } -} - func contains(s []string, want string) bool { for _, v := range s { if v == want { diff --git a/internal/recipes/scope.go b/internal/recipes/scope.go new file mode 100644 index 00000000..214d320e --- /dev/null +++ b/internal/recipes/scope.go @@ -0,0 +1,255 @@ +package recipes + +import ( + "fmt" + "io" + "os" + "path/filepath" + "strings" + + "github.com/novusedge/stoat/internal/config" + "github.com/novusedge/stoat/internal/tomlx" +) + +// ProjectFile is the name that puts stoat in project scope. +const ProjectFile = "stoat.toml" + +// Scope is where one lock and its cache live. +// +// The project scope needs stoat.toml in the current directory. Stoat does not +// walk up: a recipe that appears because of a file three directories above +// the user is worse than one that does not appear at all. +type Scope struct { + Name string // "project" or "global" + Dir string // the directory holding the lock + LockPath string + CachePath string + ConfigPath string // stoat.toml at project scope, empty at global scope +} + +// Decl is one entry of stoat.toml's [recipes] table. An empty Source means the +// name resolves through the index. +type Decl struct { + Source string + Ref string +} + +// ScopeFor picks the scope a recipe command works in. global forces the home +// scope even inside a project. +func ScopeFor(global bool) (Scope, error) { + if !global { + wd, err := os.Getwd() + if err != nil { + return Scope{}, err + } + cfg := filepath.Join(wd, ProjectFile) + if _, err := os.Stat(cfg); err == nil { + return Scope{ + Name: "project", + Dir: wd, + LockPath: filepath.Join(wd, "stoat.lock"), + CachePath: filepath.Join(wd, ".stoat", "recipes"), + ConfigPath: cfg, + }, nil + } else if !os.IsNotExist(err) { + return Scope{}, err + } + } + root := config.Root() + return Scope{ + Name: "global", + Dir: root, + LockPath: filepath.Join(root, "stoat.lock"), + CachePath: filepath.Join(root, "recipes"), + }, nil +} + +// Lock reads the scope's lock. +func (s Scope) Lock() (Lock, error) { return LoadLock(s.LockPath) } + +// Save writes the scope's lock. +func (s Scope) Save(l Lock) error { return SaveLock(s.LockPath, l) } + +// Decls reads stoat.toml's [recipes] table. The global scope has no +// declaration file, so it declares nothing. +// +// A value is a bare ref string or a table with source and ref, so it decodes +// into any and converts here. The whole document is decoded to preserve +// unrelated project tables when callers edit declarations. +func (s Scope) Decls() (map[string]Decl, error) { + out := map[string]Decl{} + if s.ConfigPath == "" { + return out, nil + } + var f map[string]any + if err := tomlx.Decode(s.ConfigPath, &f, tomlx.Warn(io.Discard)); err != nil { + return nil, err + } + raw, ok := f["recipes"] + if !ok { + return out, nil + } + recipes, ok := stringMap(raw) + if !ok { + return nil, fmt.Errorf("%s: recipes must be a table", ProjectFile) + } + for name, v := range recipes { + if err := validateRecipeName(name); err != nil { + return nil, err + } + d, err := declFrom(name, v) + if err != nil { + return nil, err + } + out[name] = d + } + return out, nil +} + +func declFrom(name string, v any) (Decl, error) { + switch t := v.(type) { + case string: + return Decl{Ref: t}, nil + default: + m, ok := stringMap(t) + if !ok { + return Decl{}, fmt.Errorf("%s: recipes.%s must be a ref string or a table", ProjectFile, name) + } + d := Decl{} + for key, value := range m { + switch key { + case "source": + s, ok := value.(string) + if !ok { + return Decl{}, fmt.Errorf("%s: recipes.%s.source must be a string", ProjectFile, name) + } + d.Source = s + case "ref": + s, ok := value.(string) + if !ok { + return Decl{}, fmt.Errorf("%s: recipes.%s.ref must be a string", ProjectFile, name) + } + d.Ref = s + default: + return Decl{}, fmt.Errorf("%s: unknown key %q in recipes.%s", ProjectFile, key, name) + } + } + return d, nil + } +} + +func stringMap(value any) (map[string]any, bool) { + m, ok := value.(map[string]any) + return m, ok +} + +// SetDecl writes one entry into stoat.toml's [recipes] table. The global scope +// has no declaration file, so it does nothing. +func (s Scope) SetDecl(name string, d Decl) error { + if err := validateRecipeName(name); err != nil { + return err + } + return s.editDecls(func(m map[string]any) { + if d.Source == "" { + m[name] = d.Ref + return + } + m[name] = map[string]any{"source": d.Source, "ref": d.Ref} + }) +} + +// RemoveDecl deletes one entry from stoat.toml's [recipes] table. +func (s Scope) RemoveDecl(name string) error { + if err := validateRecipeName(name); err != nil { + return err + } + return s.editDecls(func(m map[string]any) { delete(m, name) }) +} + +func (s Scope) editDecls(edit func(map[string]any)) error { + if s.ConfigPath == "" { + return nil + } + var f map[string]any + if err := tomlx.Decode(s.ConfigPath, &f, tomlx.Warn(io.Discard)); err != nil { + return err + } + var recipes map[string]any + if raw, ok := f["recipes"]; ok { + var converted bool + recipes, converted = stringMap(raw) + if !converted { + return fmt.Errorf("%s: recipes must be a table", ProjectFile) + } + } + if recipes == nil { + recipes = map[string]any{} + } + edit(recipes) + f["recipes"] = recipes + return encodeProject(s.ConfigPath, f) +} + +func encodeProject(path string, value any) (err error) { + dir := filepath.Dir(path) + if err := os.MkdirAll(dir, 0o755); err != nil { + return err + } + mode, err := existingFileMode(path) + if err != nil { + return err + } + stageDir, err := os.MkdirTemp(dir, ".stoat-project-*") + if err != nil { + return err + } + tmpPath := filepath.Join(stageDir, "stoat.toml") + tmp, err := os.OpenFile(tmpPath, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0o644) + if err != nil { + _ = os.RemoveAll(stageDir) + return err + } + if err := tmp.Close(); err != nil { + _ = os.RemoveAll(stageDir) + return err + } + defer func() { + if removeErr := os.RemoveAll(stageDir); removeErr != nil && err == nil { + err = removeErr + } + }() + if err := tomlx.Encode(tmpPath, value); err != nil { + return err + } + if mode != 0 { + if err := os.Chmod(tmpPath, mode); err != nil { + return err + } + } + return os.Rename(tmpPath, path) +} + +// IgnoreStoatDir appends ".stoat/" to dir's .gitignore. It does nothing +// outside a git checkout, and nothing when the line is already there. +func IgnoreStoatDir(dir string) error { + if _, err := os.Stat(filepath.Join(dir, ".git")); err != nil { + if os.IsNotExist(err) { + return nil + } + return err + } + path := filepath.Join(dir, ".gitignore") + body, err := os.ReadFile(path) + if err != nil && !os.IsNotExist(err) { + return err + } + for _, line := range strings.Split(string(body), "\n") { + if strings.TrimSpace(line) == ".stoat/" { + return nil + } + } + if len(body) > 0 && !strings.HasSuffix(string(body), "\n") { + body = append(body, '\n') + } + return os.WriteFile(path, append(body, ".stoat/\n"...), 0o644) +} diff --git a/internal/recipes/scope_test.go b/internal/recipes/scope_test.go new file mode 100644 index 00000000..898c09d2 --- /dev/null +++ b/internal/recipes/scope_test.go @@ -0,0 +1,209 @@ +package recipes + +import ( + "os" + "path/filepath" + "strings" + "testing" + + "github.com/novusedge/stoat/internal/tomlx" +) + +func TestScopeForProjectWhenStoatTomlIsPresent(t *testing.T) { + wd := t.TempDir() + writeFile(t, filepath.Join(wd, "stoat.toml"), "[recipes]\n") + t.Chdir(wd) + + s, err := ScopeFor(false) + if err != nil { + t.Fatal(err) + } + if s.Name != "project" { + t.Errorf("Name = %q, want %q", s.Name, "project") + } + if s.LockPath != filepath.Join(wd, "stoat.lock") { + t.Errorf("LockPath = %q", s.LockPath) + } + if s.CachePath != filepath.Join(wd, ".stoat", "recipes") { + t.Errorf("CachePath = %q", s.CachePath) + } +} + +func TestScopeForGlobalFlagOverridesTheProject(t *testing.T) { + wd := t.TempDir() + writeFile(t, filepath.Join(wd, "stoat.toml"), "[recipes]\n") + t.Chdir(wd) + home := t.TempDir() + t.Setenv("STOAT_HOME", home) + + s, err := ScopeFor(true) + if err != nil { + t.Fatal(err) + } + if s.Name != "global" || s.LockPath != filepath.Join(home, "stoat.lock") { + t.Errorf("scope = %+v, want the home scope", s) + } + + parent := t.TempDir() + writeFile(t, filepath.Join(parent, "stoat.toml"), "[recipes]\n") + child := filepath.Join(parent, "child") + if err := os.MkdirAll(child, 0o755); err != nil { + t.Fatal(err) + } + t.Chdir(child) + s, err = ScopeFor(false) + if err != nil { + t.Fatal(err) + } + if s.Name != "global" { + t.Errorf("scope = %q, want global when only a parent has stoat.toml", s.Name) + } +} + +func TestDeclsAcceptAStringOrATable(t *testing.T) { + wd := t.TempDir() + writeFile(t, filepath.Join(wd, "stoat.toml"), `[recipes] +tailscale = "v1.2" +xfce = { source = "https://github.com/x/stoat-xfce", ref = "main" } +`) + t.Chdir(wd) + + s, err := ScopeFor(false) + if err != nil { + t.Fatal(err) + } + d, err := s.Decls() + if err != nil { + t.Fatal(err) + } + if d["tailscale"] != (Decl{Ref: "v1.2"}) { + t.Errorf("tailscale = %+v", d["tailscale"]) + } + if d["xfce"] != (Decl{Source: "https://github.com/x/stoat-xfce", Ref: "main"}) { + t.Errorf("xfce = %+v", d["xfce"]) + } +} + +func TestSetDeclAndRemoveDecl(t *testing.T) { + type projectDocument struct { + VM struct { + Name string `toml:"name"` + } `toml:"vm"` + Recipes map[string]string `toml:"recipes"` + } + + wd := t.TempDir() + writeFile(t, filepath.Join(wd, "stoat.toml"), `[vm] +name = "alpha" + +[recipes] +`) + t.Chdir(wd) + s, err := ScopeFor(false) + if err != nil { + t.Fatal(err) + } + + if err := s.SetDecl("tailscale", Decl{Ref: "v1.2"}); err != nil { + t.Fatal(err) + } + checkProjectData := func() { + t.Helper() + var project projectDocument + if err := tomlx.Decode(filepath.Join(wd, "stoat.toml"), &project, tomlx.Reject); err != nil { + t.Fatal(err) + } + if project.VM.Name != "alpha" { + t.Fatalf("unrelated project data was lost: %+v", project) + } + } + checkProjectData() + d, err := s.Decls() + if err != nil { + t.Fatal(err) + } + if d["tailscale"].Ref != "v1.2" { + t.Fatalf("decls = %+v", d) + } + if err := s.RemoveDecl("tailscale"); err != nil { + t.Fatal(err) + } + d, err = s.Decls() + if err != nil { + t.Fatal(err) + } + if _, ok := d["tailscale"]; ok { + t.Errorf("decls = %+v, want tailscale gone", d) + } + checkProjectData() +} + +func TestSetDeclAndSavePreserveExistingFileModes(t *testing.T) { + wd := t.TempDir() + configPath := filepath.Join(wd, "stoat.toml") + writeFile(t, configPath, "[recipes]\n") + if err := os.Chmod(configPath, 0o644); err != nil { + t.Fatal(err) + } + t.Chdir(wd) + s, err := ScopeFor(false) + if err != nil { + t.Fatal(err) + } + if err := s.Save(Lock{Schema: LockSchema, Recipes: map[string]LockEntry{ + "old": {Source: "source", Ref: "main", Commit: "old", Added: "now"}, + }}); err != nil { + t.Fatal(err) + } + if err := os.Chmod(s.LockPath, 0o600); err != nil { + t.Fatal(err) + } + if err := s.SetDecl("tailscale", Decl{Ref: "v1.2"}); err != nil { + t.Fatal(err) + } + if err := s.Save(Lock{Schema: LockSchema, Recipes: map[string]LockEntry{ + "new": {Source: "source", Ref: "main", Commit: "new", Added: "now"}, + }}); err != nil { + t.Fatal(err) + } + for path, want := range map[string]os.FileMode{ + configPath: 0o644, + s.LockPath: 0o600, + } { + info, err := os.Stat(path) + if err != nil { + t.Fatal(err) + } + if got := info.Mode().Perm(); got != want { + t.Errorf("%s mode = %o, want %o", path, got, want) + } + } +} + +func TestIgnoreStoatDirAppendsOnceInAGitCheckout(t *testing.T) { + wd := t.TempDir() + if err := os.MkdirAll(filepath.Join(wd, ".git"), 0o755); err != nil { + t.Fatal(err) + } + writeFile(t, filepath.Join(wd, ".gitignore"), "build/\n") + + for range 2 { + if err := IgnoreStoatDir(wd); err != nil { + t.Fatal(err) + } + } + got := readFile(t, filepath.Join(wd, ".gitignore")) + if strings.Count(got, ".stoat/") != 1 { + t.Errorf("gitignore = %q, want one .stoat/ line", got) + } +} + +func TestIgnoreStoatDirSkipsANonCheckout(t *testing.T) { + wd := t.TempDir() + if err := IgnoreStoatDir(wd); err != nil { + t.Fatal(err) + } + if _, err := os.Stat(filepath.Join(wd, ".gitignore")); !os.IsNotExist(err) { + t.Errorf("gitignore was created outside a checkout") + } +} diff --git a/internal/recipes/snapshot.go b/internal/recipes/snapshot.go new file mode 100644 index 00000000..4bb168e7 --- /dev/null +++ b/internal/recipes/snapshot.go @@ -0,0 +1,202 @@ +package recipes + +import ( + "fmt" + "os" + "path/filepath" + "sort" + "syscall" +) + +// RecipeSnapshot is a coherent view of visible manifests and their ownership +// pins. Callers receive only after all relevant scope locks are released. +type RecipeSnapshot struct { + Roots []Root + Manifests []Manifest + Scopes map[string]string + Pins map[string]LockEntry +} + +func scopeCoordinationPath(s Scope) string { + if s.Name == "project" { + return filepath.Join(filepath.Dir(s.CachePath), "recipe.lock") + } + return filepath.Join(s.Dir, "recipe.lock") +} + +func lockScopeMode(s Scope, exclusive bool) (func() error, error) { + coordinationDir := filepath.Dir(scopeCoordinationPath(s)) + if err := os.MkdirAll(coordinationDir, 0o755); err != nil { + return nil, err + } + f, err := os.OpenFile(scopeCoordinationPath(s), os.O_CREATE|os.O_RDWR, 0o644) + if err != nil { + return nil, err + } + lockType := syscall.LOCK_SH + if exclusive { + lockType = syscall.LOCK_EX + } + if err := syscall.Flock(int(f.Fd()), lockType); err != nil { + _ = f.Close() + return nil, err + } + return func() error { + unlockErr := syscall.Flock(int(f.Fd()), syscall.LOCK_UN) + closeErr := f.Close() + if unlockErr != nil { + return unlockErr + } + return closeErr + }, nil +} + +// lockRecipeScopes acquires project and global locks in one total path order. +// Callers release the returned locks in reverse order. +func lockRecipeScopes(exclusive bool) ([]func() error, error) { + global, err := ScopeFor(true) + if err != nil { + return nil, err + } + project, err := ScopeFor(false) + if err != nil { + return nil, err + } + scopes := []Scope{global} + if project.Name == "project" && scopeCoordinationPath(project) != scopeCoordinationPath(global) { + scopes = append(scopes, project) + } + sort.Slice(scopes, func(i, j int) bool { + return scopeCoordinationPath(scopes[i]) < scopeCoordinationPath(scopes[j]) + }) + locks := make([]func() error, 0, len(scopes)) + for _, scope := range scopes { + unlock, lockErr := lockScopeMode(scope, exclusive) + if lockErr != nil { + for i := len(locks) - 1; i >= 0; i-- { + _ = locks[i]() + } + return nil, lockErr + } + locks = append(locks, unlock) + } + return locks, nil +} + +func unlockRecipeScopes(locks []func() error) error { + var first error + for i := len(locks) - 1; i >= 0; i-- { + if err := locks[i](); err != nil && first == nil { + first = err + } + } + return first +} + +// ReadLock reads one scope lock under its shared coordination flock. +func ReadLock(s Scope) (Lock, error) { + unlock, err := lockScopeMode(s, false) + if err != nil { + return Lock{}, err + } + lock, readErr := s.Lock() + unlockErr := unlock() + if readErr != nil { + return Lock{}, readErr + } + return lock, unlockErr +} + +func snapshotLocked() (RecipeSnapshot, error) { + manifests, err := listManifestsLocked() + if err != nil { + return RecipeSnapshot{}, err + } + global, err := ScopeFor(true) + if err != nil { + return RecipeSnapshot{}, err + } + project, err := ScopeFor(false) + if err != nil { + return RecipeSnapshot{}, err + } + locks := map[string]Lock{} + globalLock, err := global.Lock() + if err != nil { + return RecipeSnapshot{}, err + } + locks[global.Name] = globalLock + if project.Name == "project" { + projectLock, lockErr := project.Lock() + if lockErr != nil { + return RecipeSnapshot{}, lockErr + } + locks[project.Name] = projectLock + } + roots, err := Roots() + if err != nil { + return RecipeSnapshot{}, err + } + snapshot := RecipeSnapshot{ + Roots: roots, + Manifests: manifests, + Scopes: make(map[string]string, len(manifests)), + Pins: make(map[string]LockEntry, len(manifests)), + } + for _, manifest := range manifests { + _, scope, found, resolveErr := resolvePath(manifest.Name) + if resolveErr != nil { + return RecipeSnapshot{}, resolveErr + } + if !found { + return RecipeSnapshot{}, fmt.Errorf("recipe %q disappeared while reading its snapshot", manifest.Name) + } + snapshot.Scopes[manifest.Name] = scope + if lock, ok := locks[scope]; ok { + if pin, exists := lock.Recipes[manifest.Name]; exists { + snapshot.Pins[manifest.Name] = pin + } + } + } + return snapshot, nil +} + +// ListSnapshot reads visible manifests and pins under shared scope locks. +func ListSnapshot() (RecipeSnapshot, error) { + locks, err := lockRecipeScopes(false) + if err != nil { + return RecipeSnapshot{}, err + } + snapshot, readErr := snapshotLocked() + unlockErr := unlockRecipeScopes(locks) + if readErr != nil { + return RecipeSnapshot{}, readErr + } + return snapshot, unlockErr +} + +// RepairSnapshot validates and repairs the project cache, then materializes a +// coherent view while retaining exclusive locks for the full read-may-repair. +func RepairSnapshot() (RecipeSnapshot, error) { + locks, err := lockRecipeScopes(true) + if err != nil { + return RecipeSnapshot{}, err + } + project, err := ScopeFor(false) + if err != nil { + _ = unlockRecipeScopes(locks) + return RecipeSnapshot{}, err + } + if project.Name == "project" { + if err := repairProjectLocked(project); err != nil { + _ = unlockRecipeScopes(locks) + return RecipeSnapshot{}, err + } + } + snapshot, readErr := snapshotLocked() + unlockErr := unlockRecipeScopes(locks) + if readErr != nil { + return RecipeSnapshot{}, readErr + } + return snapshot, unlockErr +} diff --git a/internal/recipes/test_helpers_test.go b/internal/recipes/test_helpers_test.go new file mode 100644 index 00000000..773bb80a --- /dev/null +++ b/internal/recipes/test_helpers_test.go @@ -0,0 +1,26 @@ +package recipes + +import ( + "os" + "path/filepath" + "testing" +) + +func writeFile(t *testing.T, path, body string) { + t.Helper() + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(path, []byte(body), 0o644); err != nil { + t.Fatal(err) + } +} + +func readFile(t *testing.T, path string) string { + t.Helper() + b, err := os.ReadFile(path) + if err != nil { + t.Fatal(err) + } + return string(b) +} diff --git a/internal/recipes/validate.go b/internal/recipes/validate.go new file mode 100644 index 00000000..ecf5c0d8 --- /dev/null +++ b/internal/recipes/validate.go @@ -0,0 +1,144 @@ +package recipes + +import ( + "errors" + "fmt" + "io/fs" + "os" + "path/filepath" + "strings" +) + +// ErrInvalidTree is the sentinel for a cloned recipe tree the user cannot fix +// by retrying: a bad name, a missing manifest, a manifest that names the +// wrong recipe, or a script/symlink path that escapes the tree. wire.MapError +// routes it to invalid_spec, the CLI-layer code for "the request itself is +// malformed" (as opposed to a transient or environmental failure). +var ErrInvalidTree = errors.New("invalid recipe tree") + +// ValidateTree checks a cloned recipe before it becomes an active cache +// entry. It validates the manifest and every script path the manifest names. +func ValidateTree(dir, name string) error { + if err := validateRecipeName(name); err != nil { + return fmt.Errorf("%w: %v", ErrInvalidTree, err) + } + root, err := filepath.EvalSymlinks(dir) + if err != nil { + return err + } + root, err = filepath.Abs(root) + if err != nil { + return err + } + manifestPath := filepath.Join(root, "recipe.toml") + if _, err := os.Stat(manifestPath); err != nil { + if os.IsNotExist(err) { + return fmt.Errorf("%w: %s: no recipe.toml at the repository root", ErrInvalidTree, name) + } + return err + } + m, err := ParseManifest(manifestPath) + if err != nil { + return fmt.Errorf("%w: %v", ErrInvalidTree, err) + } + if m.Name != name { + return fmt.Errorf("%w: %s: recipe.toml is named %q", ErrInvalidTree, name, m.Name) + } + if err := validateScriptPath(root, m.Script); err != nil { + return fmt.Errorf("%w: %v", ErrInvalidTree, err) + } + for osName, script := range m.Scripts { + if err := validateScriptPath(root, script); err != nil { + return fmt.Errorf("%w: %s script %q: %v", ErrInvalidTree, osName, script, err) + } + } + if err := validateTreeSymlinks(root); err != nil { + return fmt.Errorf("%w: %v", ErrInvalidTree, err) + } + return nil +} + +func validateScriptPath(root, script string) error { + path := filepath.Join(root, filepath.FromSlash(script)) + if !within(root, path) { + return fmt.Errorf("script %q points outside the recipe", script) + } + resolved, err := filepath.EvalSymlinks(path) + if err != nil { + return fmt.Errorf("script %q: %w", script, err) + } + if !within(root, resolved) { + return fmt.Errorf("script %q points outside the recipe", script) + } + info, err := os.Stat(path) + if err != nil { + return fmt.Errorf("script %q: %w", script, err) + } + if !info.Mode().IsRegular() { + return fmt.Errorf("script %q is not a regular file", script) + } + return nil +} + +func validateTreeSymlinks(root string) error { + return filepath.WalkDir(root, func(path string, entry fs.DirEntry, walkErr error) error { + if walkErr != nil { + return walkErr + } + if entry.Type()&os.ModeSymlink == 0 { + return nil + } + resolved, err := filepath.EvalSymlinks(path) + if err != nil { + return fmt.Errorf("%s: %w", filepath.Base(path), err) + } + if !within(root, resolved) { + return fmt.Errorf("%s: symlink points outside the recipe", filepath.Base(path)) + } + return nil + }) +} + +func within(root, path string) bool { + rel, err := filepath.Rel(root, path) + return err == nil && rel != ".." && !strings.HasPrefix(rel, ".."+string(filepath.Separator)) +} + +// CheckCollision rejects local, bundled, and same-scope remote recipes. A +// remote recipe in another scope may shadow the lower-priority entry. +func CheckCollision(name, scope string) error { + if err := validateRecipeName(name); err != nil { + return err + } + if scope == "global" || scope == "project" { + target, err := ScopeFor(scope == "global") + if err != nil { + return err + } + if scope == "global" || target.Name == "project" { + lock, err := target.Lock() + if err != nil { + return err + } + if _, ok := lock.Recipes[name]; ok { + return fmt.Errorf("%q is already a %s remote recipe; pick another name or use --force", name, scope) + } + } + } + // Add already holds the target scope's exclusive lock. Resolve the + // collision from that locked snapshot instead of nesting another flock. + _, have, found, err := resolvePath(name) + if err != nil { + return err + } + if !found || have == "" { + return nil + } + if have == "local" || have == "bundled" { + return fmt.Errorf("%q is a %s recipe; pick another name or use --force", name, have) + } + if have == scope { + return fmt.Errorf("%q is already a %s remote recipe; pick another name or use --force", name, scope) + } + return nil +} diff --git a/internal/recipes/validate_test.go b/internal/recipes/validate_test.go new file mode 100644 index 00000000..88900dcc --- /dev/null +++ b/internal/recipes/validate_test.go @@ -0,0 +1,196 @@ +package recipes + +import ( + "os" + "path/filepath" + "strings" + "testing" +) + +func TestValidateTreeAcceptsSchema3ScriptsAndResolvedSymlinkChain(t *testing.T) { + dir := t.TempDir() + writeFile(t, filepath.Join(dir, "recipe.toml"), `schema = 3 +name = "demo" +script = "install.sh" + +[scripts] +alpine = "scripts/alpine.sh" + +[params.user] +type = "string" +default = "dev" +`) + writeFile(t, filepath.Join(dir, "install.sh"), "#!/bin/sh\n") + writeFile(t, filepath.Join(dir, "common.sh"), "#!/bin/sh\necho common\n") + if err := os.MkdirAll(filepath.Join(dir, "links"), 0o755); err != nil { + t.Fatal(err) + } + if err := os.MkdirAll(filepath.Join(dir, "scripts"), 0o755); err != nil { + t.Fatal(err) + } + if err := os.Symlink("../common.sh", filepath.Join(dir, "links", "first.sh")); err != nil { + t.Fatal(err) + } + if err := os.Symlink("../links/first.sh", filepath.Join(dir, "scripts", "alpine.sh")); err != nil { + t.Fatal(err) + } + + if err := ValidateTree(dir, "demo"); err != nil { + t.Fatalf("ValidateTree() = %v, want nil", err) + } +} + +func TestValidateTreeRejectsASymlinkChainThatEscapes(t *testing.T) { + dir := t.TempDir() + writeFile(t, filepath.Join(dir, "recipe.toml"), `schema = 3 +name = "demo" +script = "install.sh" + +[scripts] +alpine = "scripts/alpine.sh" +`) + writeFile(t, filepath.Join(dir, "install.sh"), "#!/bin/sh\n") + if err := os.MkdirAll(filepath.Join(dir, "links", "nested"), 0o755); err != nil { + t.Fatal(err) + } + if err := os.MkdirAll(filepath.Join(dir, "scripts"), 0o755); err != nil { + t.Fatal(err) + } + if err := os.Symlink("/etc/passwd", filepath.Join(dir, "links", "nested", "outside.sh")); err != nil { + t.Fatal(err) + } + if err := os.Symlink("../links/nested/outside.sh", filepath.Join(dir, "scripts", "alpine.sh")); err != nil { + t.Fatal(err) + } + + err := ValidateTree(dir, "demo") + if err == nil || !strings.Contains(err.Error(), "points outside the recipe") { + t.Fatalf("ValidateTree() = %v, want an outside-tree error", err) + } +} + +func TestValidateTreeRequiresAUniqueStrictRootManifest(t *testing.T) { + tests := []struct { + name string + body string + want string + }{ + {name: "missing", want: "no recipe.toml at the repository root"}, + {name: "unknown field", body: "name = \"demo\"\nscript = \"install.sh\"\nunexpected = true\n", want: "unexpected"}, + {name: "missing OS override", body: "schema = 3\nname = \"demo\"\nscript = \"install.sh\"\n[scripts]\nalpine = \"missing.sh\"\n", want: "missing.sh"}, + } + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + dir := t.TempDir() + if tc.body != "" { + writeFile(t, filepath.Join(dir, "recipe.toml"), tc.body) + writeFile(t, filepath.Join(dir, "install.sh"), "#!/bin/sh\n") + } + err := ValidateTree(dir, "demo") + if err == nil || !strings.Contains(err.Error(), tc.want) { + t.Fatalf("ValidateTree() = %v, want %q", err, tc.want) + } + }) + } +} + +func TestValidateTreeRejectsANameMismatch(t *testing.T) { + dir := t.TempDir() + writeFile(t, filepath.Join(dir, "recipe.toml"), "name = \"other\"\nscript = \"install.sh\"\n") + writeFile(t, filepath.Join(dir, "install.sh"), "#!/bin/sh\n") + + err := ValidateTree(dir, "demo") + if err == nil || !strings.Contains(err.Error(), `recipe.toml is named "other"`) { + t.Fatalf("ValidateTree() = %v, want name mismatch", err) + } +} + +func TestCheckCollisionDistinguishesSameScopeAndShadowing(t *testing.T) { + home := t.TempDir() + t.Setenv("STOAT_HOME", home) + wd := t.TempDir() + t.Chdir(wd) + + globalCache := filepath.Join(home, "recipes", "demo") + writeFile(t, filepath.Join(globalCache, "recipe.toml"), "name = \"demo\"\nscript = \"install.sh\"\n") + writeFile(t, filepath.Join(globalCache, "install.sh"), "#!/bin/sh\n") + if err := SaveLock(filepath.Join(home, "stoat.lock"), Lock{Recipes: map[string]LockEntry{ + "demo": {Source: "local", Ref: "main", Commit: strings.Repeat("a", 40)}, + }}); err != nil { + t.Fatal(err) + } + + if err := CheckCollision("demo", "global"); err == nil || !strings.Contains(err.Error(), "demo") { + t.Fatalf("same-scope collision = %v, want a collision", err) + } + if err := CheckCollision("demo", "project"); err != nil { + t.Fatalf("project shadowing global = %v, want nil", err) + } + + local := filepath.Join(home, "recipes", "handmade") + writeFile(t, filepath.Join(local, "recipe.toml"), "name = \"handmade\"\nscript = \"install.sh\"\n") + writeFile(t, filepath.Join(local, "install.sh"), "#!/bin/sh\n") + if err := CheckCollision("handmade", "global"); err == nil || !strings.Contains(err.Error(), "local") { + t.Fatalf("local collision = %v, want a local collision", err) + } + + if err := Install(); err != nil { + t.Fatal(err) + } + if err := CheckCollision("docker", "global"); err == nil || !strings.Contains(err.Error(), "bundled") { + t.Fatalf("bundled collision = %v, want a bundled collision", err) + } + +} + +func TestCheckCollisionUsesTargetLockOwnershipWithoutACache(t *testing.T) { + home := remoteRoot(t) + global, err := ScopeFor(true) + if err != nil { + t.Fatal(err) + } + if err := SaveLock(global.LockPath, Lock{Recipes: map[string]LockEntry{ + "demo": {Source: "old", Ref: "main", Commit: strings.Repeat("d", 40)}, + }}); err != nil { + t.Fatal(err) + } + lockBefore, err := os.ReadFile(global.LockPath) + if err != nil { + t.Fatal(err) + } + src := namedRecipeRepo(t, "demo", "demo") + if err := CheckCollision("demo", "global"); err == nil { + t.Fatal("lock-only same-scope collision was accepted") + } + if _, err := Add(global, src, false); err == nil { + t.Fatal("Add() replaced a lock-only same-scope recipe without force") + } + lockAfter, _ := os.ReadFile(global.LockPath) + if string(lockAfter) != string(lockBefore) { + t.Fatal("same-scope collision changed the target lock") + } + if _, err := os.Stat(filepath.Join(home, "recipes", "demo")); !os.IsNotExist(err) { + t.Fatalf("same-scope collision created/replaced cache: %v", err) + } + + project := t.TempDir() + t.Chdir(project) + writeFile(t, filepath.Join(project, "stoat.toml"), "[recipes]\n") + projectScope, err := ScopeFor(false) + if err != nil { + t.Fatal(err) + } + if err := CheckCollision("demo", "project"); err != nil { + t.Fatalf("project-over-global shadow = %v, want nil", err) + } + if _, err := Add(global, src, false); err == nil { + t.Fatal("global Add() replaced its lock while a project shadow was active") + } + if _, err := os.Stat(projectScope.CachePath); !os.IsNotExist(err) { + t.Fatalf("global collision unexpectedly created project cache: %v", err) + } + lockAfterShadow, _ := os.ReadFile(global.LockPath) + if string(lockAfterShadow) != string(lockBefore) { + t.Fatal("global target lock changed under project shadow") + } +} diff --git a/internal/testutil/gitrepo.go b/internal/testutil/gitrepo.go new file mode 100644 index 00000000..fef2bcde --- /dev/null +++ b/internal/testutil/gitrepo.go @@ -0,0 +1,74 @@ +package testutil + +import ( + "os" + "os/exec" + "path/filepath" + "strings" + "testing" +) + +// GitRepo creates a bare repository with one commit on branch "main" and +// returns its path. A caller passes that path to git as a URL, so no test +// needs the network. +func GitRepo(t *testing.T, files map[string]string) string { + t.Helper() + bare := filepath.Join(t.TempDir(), "src.git") + runGit(t, "", "git", "init", "--bare", "-b", "main", bare) + GitCommit(t, bare, files, "") + return bare +} + +// GitCommit clones bare, writes files, commits, optionally tags, and pushes. +// It returns the new commit sha. +func GitCommit(t *testing.T, bare string, files map[string]string, tag string) string { + t.Helper() + work := filepath.Join(t.TempDir(), "work") + runGit(t, "", "git", "clone", "-q", bare, work) + runGit(t, work, "git", "config", "user.email", "test@example.com") + runGit(t, work, "git", "config", "user.name", "test") + for name, body := range files { + WriteFile(t, filepath.Join(work, name), body) + } + runGit(t, work, "git", "add", "-A") + runGit(t, work, "git", "commit", "-q", "-m", "commit") + if tag != "" { + runGit(t, work, "git", "tag", tag) + runGit(t, work, "git", "push", "-q", "origin", tag) + } + runGit(t, work, "git", "push", "-q", "origin", "HEAD:main") + out, err := exec.Command("git", "-C", work, "rev-parse", "HEAD").Output() + if err != nil { + t.Fatal(err) + } + return strings.TrimSpace(string(out)) +} + +// GitClone creates a work tree from bare for tests that exercise a work-tree +// operation without making the operation under test perform the clone. +func GitClone(t *testing.T, bare string) string { + t.Helper() + work := filepath.Join(t.TempDir(), "work") + runGit(t, "", "git", "clone", "-q", bare, work) + return work +} + +// WriteFile writes body to path, creating parent directories. +func WriteFile(t *testing.T, path, body string) { + t.Helper() + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(path, []byte(body), 0o644); err != nil { + t.Fatal(err) + } +} + +func runGit(t *testing.T, dir, name string, args ...string) { + t.Helper() + cmd := exec.Command(name, args...) + cmd.Dir = dir + if out, err := cmd.CombinedOutput(); err != nil { + t.Fatalf("%s %v: %v\n%s", name, args, err, out) + } +} diff --git a/internal/tui/app.go b/internal/tui/app.go index 0b5c6988..d7e1d6bb 100644 --- a/internal/tui/app.go +++ b/internal/tui/app.go @@ -168,7 +168,11 @@ func preflightReport(checks []core.HostCheck) string { if c.OK { continue } - line := c.Name + ": " + c.Detail + label := c.Name + if c.Optional { + label += " (optional)" + } + line := label + ": " + c.Detail if len(c.Fix) > 0 { line += "\n fix: " + strings.Join(c.Fix, " && ") } diff --git a/internal/tui/app_test.go b/internal/tui/app_test.go new file mode 100644 index 00000000..a1b479fe --- /dev/null +++ b/internal/tui/app_test.go @@ -0,0 +1,32 @@ +package tui + +import ( + "strings" + "testing" + + "github.com/novusedge/stoat/internal/core" +) + +func TestTUIViewIncludesOptionalHostFailureAndFix(t *testing.T) { + checks := []core.HostCheck{ + {Name: "git", Detail: "not found", Optional: true, Fix: []string{"install git"}}, + {Name: "qemu-img", Detail: "not found", Fix: []string{"install qemu-img"}}, + } + // preflightReport is only the model setup used by Run; the assertion is on + // the caller-visible rendered View, where optional repair guidance must not + // disappear. + m := model{ + screen: screenList, + width: 80, + height: 24, + list: newVMList(), + preflight: preflightReport(checks), + } + got := m.View().Content + if !strings.Contains(got, "git") || !strings.Contains(got, "install git") { + t.Fatalf("TUI view omitted optional Git repair guidance: %q", got) + } + if !strings.Contains(got, "qemu-img") || !strings.Contains(got, "install qemu-img") { + t.Fatalf("required host failure missing from TUI view: %q", got) + } +} diff --git a/mcp/stoat_mcp/client.py b/mcp/stoat_mcp/client.py index 9bbcaf9b..0b6e2efc 100644 --- a/mcp/stoat_mcp/client.py +++ b/mcp/stoat_mcp/client.py @@ -19,8 +19,11 @@ import json import os +import selectors +import signal import shutil import subprocess +import time from collections.abc import Callable, Iterator, Sequence from typing import Any @@ -33,7 +36,7 @@ # Recipe lost label, target_os and shared. There is no v1 path on either side; # a server built for v1 refuses to start against a v2 binary and vice versa, # which is the entire point of checking this at startup. -EXPECTED_CONTRACT = 2 +EXPECTED_CONTRACT = 3 # Non-terminal event types we understand. Anything else is skipped per rule 3. EVENT_PROGRESS = "progress" @@ -102,16 +105,27 @@ def run( stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, stdin=subprocess.DEVNULL, - text=True, - bufsize=1, + text=False, + bufsize=0, + start_new_session=(os.name == "posix"), ) + effective_timeout = timeout if timeout is not None else self.default_timeout + deadline = None if effective_timeout is None else time.monotonic() + effective_timeout + selector: selectors.BaseSelector | None = None + returncode: int | None = None try: assert proc.stdout is not None - for line in proc.stdout: - stdout_lines.append(line) - obj = _decode(line) + selector = selectors.DefaultSelector() + selector.register(proc.stdout, selectors.EVENT_READ) + buffer = bytearray() + + def consume(line: bytes) -> None: + nonlocal result + decoded = line.decode("utf-8", errors="replace") + stdout_lines.append(decoded) + obj = _decode(decoded) if obj is None: - continue + return kind = obj.get("type") if kind == "result": # Keep the LAST one. The contract guarantees exactly one, @@ -119,15 +133,43 @@ def run( # violation degrades to "used the final answer" instead of # "silently used a stale one". result = obj - continue + return if on_event is not None and kind in (EVENT_PROGRESS, EVENT_STAGE, EVENT_LOG): on_event(kind, obj.get("data") or {}) # Any other type is skipped: rule 3. - returncode = proc.wait(timeout=timeout if timeout is not None else self.default_timeout) + + while True: + remaining = None if deadline is None else deadline - time.monotonic() + if remaining is not None and remaining <= 0: + raise subprocess.TimeoutExpired(proc.args, effective_timeout) + if not selector.select(remaining): + raise subprocess.TimeoutExpired(proc.args, effective_timeout) + chunk = os.read(proc.stdout.fileno(), 4096) + if not chunk: + if buffer: + consume(bytes(buffer)) + break + buffer.extend(chunk) + while True: + try: + end = buffer.index(10) + except ValueError: + break + consume(bytes(buffer[: end + 1])) + del buffer[: end + 1] + + remaining = None if deadline is None else max(0.0, deadline - time.monotonic()) + returncode = proc.wait(timeout=remaining) + except subprocess.TimeoutExpired: + _terminate_owned_process_group(proc) + raise finally: if proc.poll() is None: - proc.kill() - proc.wait() + _terminate_owned_process_group(proc) + if selector is not None: + selector.close() + if proc.stdout is not None: + proc.stdout.close() if result is None: raise StoatCrashed(returncode, "".join(stdout_lines)) @@ -161,6 +203,31 @@ def stream(self, *args: str, timeout: float | None = None) -> Iterator[tuple[str yield ("result", data) +def _terminate_owned_process_group(proc: subprocess.Popen[bytes]) -> None: + """Terminate this invocation and descendants without touching other jobs.""" + if os.name == "posix": + try: + os.killpg(proc.pid, signal.SIGTERM) + except ProcessLookupError: + pass + elif proc.poll() is None: + proc.terminate() + try: + proc.wait(timeout=1.0) + except subprocess.TimeoutExpired: + pass + if os.name == "posix": + try: + # The leader can exit after TERM while a descendant ignores it. + # Escalate the owned group even when proc.wait already returned. + os.killpg(proc.pid, signal.SIGKILL) + except ProcessLookupError: + pass + elif proc.poll() is None: + proc.kill() + proc.wait() + + def _decode(line: str) -> dict[str, Any] | None: """Parse one stdout line, tolerating anything that is not a JSON object. diff --git a/mcp/stoat_mcp/guards.py b/mcp/stoat_mcp/guards.py index cc32ad4b..8a0e7709 100644 --- a/mcp/stoat_mcp/guards.py +++ b/mcp/stoat_mcp/guards.py @@ -31,6 +31,9 @@ # BYO path is an arbitrary host file read, booted as a disk. _IMAGE_ID = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]*$") +_RECIPE_NAME = re.compile(r"^[a-z][a-z0-9-]*$") +_RECIPE_REF = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._/-]*$") + def check_vm_name(name: str) -> str: """Reject anything that is not a plain VM name. @@ -79,6 +82,40 @@ def check_image_id(image: str) -> str: return image +def check_index_name(ref: str) -> str: + """Accept an index recipe name with an optional safe git ref. + + The recipe name is always resolved through the curated index. A ref may + contain slashes for branches such as ``feature/topic`` but cannot contain + URL, option, or path-traversal syntax. + """ + if not isinstance(ref, str) or not ref or ref != ref.strip(): + raise GuardRejection("recipe index name is required") + if ref.startswith("-") or "\\" in ref or "\x00" in ref: + raise GuardRejection(f"recipe index name {ref!r} is not safe") + if ref.count("@") > 1: + raise GuardRejection(f"recipe index name {ref!r} has more than one ref") + name, separator, branch = ref.partition("@") + if not _RECIPE_NAME.fullmatch(name): + raise GuardRejection( + f"recipe index name {ref!r} must start with a letter and contain only letters, digits and dashes" + ) + if not separator: + return ref + if not _RECIPE_REF.fullmatch(branch) or branch.startswith("/") or branch.endswith("/"): + raise GuardRejection(f"recipe ref {branch!r} is not safe") + parts = branch.split("/") + if ".." in branch or any( + part in ("", ".", "..") + or part.startswith(".") + or part.endswith(".") + or part.endswith(".lock") + for part in parts + ): + raise GuardRejection(f"recipe ref {branch!r} is not safe") + return ref + + def shared_dir(vm: str, data_root: str | os.PathLike[str] | None = None) -> Path: """The one host directory an agent may read or write for this VM. diff --git a/mcp/stoat_mcp/server.py b/mcp/stoat_mcp/server.py index a7b320ca..48638067 100644 --- a/mcp/stoat_mcp/server.py +++ b/mcp/stoat_mcp/server.py @@ -34,6 +34,7 @@ from fastmcp import FastMCP from fastmcp.exceptions import ToolError +from . import guards from .client import Client, argv_for_bool from .errors import ContractMismatch, GuardRejection, StoatCrashed, StoatError from .guards import ( @@ -218,6 +219,21 @@ def list_recipes(os: str | None = None, backend: str | None = None) -> dict[str, return get_client().run(*argv) +@mcp.tool( + name="search_recipes", + description="Search the curated recipe index by name or description. Read-only.", + annotations={"readOnlyHint": True, "destructiveHint": False}, +) +@_guarded("search_recipes") +def search_recipes(term: str = "") -> dict[str, Any]: + argv = ["recipe", "search"] + if term.startswith("-"): + argv.extend(("--", term)) + elif term: + argv.append(term) + return get_client().run(*argv) + + @mcp.tool( name="check_recipes", description=( @@ -441,6 +457,53 @@ def update( return get_client().run(*argv) +@mcp.tool( + name="add_recipe", + description="Install a recipe from the curated index and pin its commit.", + annotations={"readOnlyHint": False, "destructiveHint": False}, +) +@_guarded("add_recipe") +def add_recipe(name: str, ref: str | None = None) -> dict[str, Any]: + if ref is None: + spec = guards.check_index_name(name) + else: + base = guards.check_index_name(name) + if "@" in base: + raise GuardRejection("add_recipe name and ref must be separate") + spec = guards.check_index_name(f"{base}@{ref}") + return get_client().run("recipe", "add", spec, "-y") + + +@mcp.tool( + name="update_recipe", + description="Fetch a remote recipe's ref again and repin it, or update every remote recipe.", + annotations={"readOnlyHint": False, "destructiveHint": False}, +) +@_guarded("update_recipe") +def update_recipe(name: str | None = None) -> dict[str, Any]: + argv = ["recipe", "update"] + if name is not None: + argv.append(_plain_recipe_name(name, "update_recipe")) + return get_client().run(*argv) + + +@mcp.tool( + name="remove_recipe", + description="Remove a remote recipe, its lock entry and its cache directory.", + annotations={"readOnlyHint": False, "destructiveHint": True}, +) +@_guarded("remove_recipe") +def remove_recipe(name: str) -> dict[str, Any]: + return get_client().run("recipe", "rm", _plain_recipe_name(name, "remove_recipe"), "-y") + + +def _plain_recipe_name(name: str, tool: str) -> str: + checked = guards.check_index_name(name) + if "@" in checked: + raise GuardRejection(f"{tool} takes a plain recipe name") + return checked + + @mcp.tool( name="clone", description=( diff --git a/mcp/tests/test_client.py b/mcp/tests/test_client.py index 27873b21..f694232a 100644 --- a/mcp/tests/test_client.py +++ b/mcp/tests/test_client.py @@ -11,8 +11,11 @@ import os import stat +import subprocess import sys import textwrap +import threading +import time import pytest @@ -146,6 +149,128 @@ def test_missing_data_field_is_an_empty_dict(client): assert data == {} +def test_timeout_covers_open_stdout_and_cleans_owned_descendants(tmp_path): + pid_path = tmp_path / "child.pid" + parent_pid_path = tmp_path / "parent.pid" + child_code = textwrap.dedent( + f""" + import signal + import time + # The group leader exits on TERM; this owned descendant requires KILL. + signal.signal(signal.SIGTERM, signal.SIG_IGN) + time.sleep(2.0) + """ + ) + fake = tmp_path / "fake-stoat-timeout" + fake.write_text( + textwrap.dedent( + f"""\ + #!/usr/bin/env python3 + import json + import os + import subprocess + import sys + import time + + args = sys.argv[1:] + cmd = args[1] if len(args) > 1 else args[0] + if cmd != "hold-open": + raise SystemExit(2) + with open({str(parent_pid_path)!r}, "w", encoding="ascii") as pid_file: + pid_file.write(str(os.getpid())) + child = subprocess.Popen([sys.executable, "-c", {child_code!r}]) + with open({str(pid_path)!r}, "w", encoding="ascii") as pid_file: + pid_file.write(str(child.pid)) + print(json.dumps({{"v": 3, "type": "progress", "cmd": cmd, "data": {{"stage": "started"}}}}), flush=True) + time.sleep(1.0) + """ + ) + ) + fake.chmod(fake.stat().st_mode | stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH) + + seen: list[tuple[str, dict]] = [] + event_seen = threading.Event() + outcome: dict[str, object] = {} + + def on_event(kind: str, payload: dict) -> None: + seen.append((kind, payload)) + event_seen.set() + + def invoke() -> None: + started = time.monotonic() + try: + Client(binary=str(fake), default_timeout=10.0).run( + "hold-open", + timeout=0.25, + on_event=on_event, + ) + except BaseException as exc: # capture the worker result for the assertion below + outcome["error"] = exc + finally: + outcome["elapsed"] = time.monotonic() - started + + worker = threading.Thread(target=invoke, daemon=True) + worker.start() + + def process_state(pid: int) -> str | None: + try: + with open(f"/proc/{pid}/stat", encoding="ascii") as stat_file: + fields = stat_file.read().split() + except FileNotFoundError: + return None + return fields[2] if len(fields) > 2 else None + + def cleanup_owned_processes() -> None: + pids: list[int] = [] + for path in (parent_pid_path, pid_path): + if path.exists(): + try: + pids.append(int(path.read_text(encoding="ascii"))) + except ValueError: + pass + for pid in pids: + if process_state(pid) not in (None, "Z"): + os.kill(pid, 9) + deadline = time.monotonic() + 2.0 + while time.monotonic() < deadline and any(process_state(pid) not in (None, "Z") for pid in pids): + time.sleep(0.01) + worker.join(timeout=2.0) + + try: + assert event_seen.wait(timeout=2.0), "fixture did not reach its startup handshake" + worker.join(timeout=3.0) + assert not worker.is_alive(), "Client.run worker did not return within its bounded fixture lifetime" + assert isinstance(outcome.get("error"), subprocess.TimeoutExpired) + assert float(outcome["elapsed"]) < 1.5 + assert seen == [("progress", {"stage": "started"})] + + child_pid = int(pid_path.read_text(encoding="ascii")) + deadline = time.monotonic() + 1.0 + while time.monotonic() < deadline and process_state(child_pid) not in (None, "Z"): + time.sleep(0.01) + if process_state(child_pid) not in (None, "Z"): + pytest.fail(f"owned child process {child_pid} survived Client.run timeout") + finally: + cleanup_owned_processes() + assert not worker.is_alive(), "owned subprocess cleanup left Client.run blocked" + + +def test_check_contract_accepts_the_remote_recipe_v3_handshake(tmp_path): + path = tmp_path / "fake-stoat" + path.write_text( + textwrap.dedent( + """\ + #!/usr/bin/env python3 + print('{"v":3,"type":"result","cmd":"version","ok":true,"data":{"contract":3,"version":"x"}}') + """ + ) + ) + path.chmod(path.stat().st_mode | stat.S_IEXEC) + + c = Client(binary=str(path), default_timeout=10.0) + assert c.check_contract() == 3 + + def test_check_contract_raises_on_mismatch(tmp_path): path = tmp_path / "fake-stoat" path.write_text( diff --git a/mcp/tests/test_guards.py b/mcp/tests/test_guards.py index 6297eab6..173ee344 100644 --- a/mcp/tests/test_guards.py +++ b/mcp/tests/test_guards.py @@ -373,3 +373,40 @@ def test_check_flag_free_rejects_a_short_flag_and_empty_values(): def test_check_flag_free_passes_ordinary_values(): assert check_flag_free(["8080:80", "2222:22"], "pairs") == ["8080:80", "2222:22"] assert check_flag_free([], "pairs") == [] + + +# --------------------------------------------------------------------------- +# recipe index names +# --------------------------------------------------------------------------- + + +@pytest.mark.parametrize( + "ref", ["tailscale", "tailscale@v1.2", "x-y@main", "tailscale@feature/topic"] +) +def test_check_index_name_accepts_name_and_optional_ref(ref: str) -> None: + from stoat_mcp import guards + + assert guards.check_index_name(ref) == ref + + +@pytest.mark.parametrize( + "ref", + [ + "https://github.com/x/stoat-tailscale", + "git@github.com:x/r.git", + "../../etc/passwd", + "a/b", + "-y", + "tailscale@../x", + "tailscale@feature..topic", + "tailscale@feature/.hidden", + "tailscale@feature/topic.lock", + "tailscale@", + "", + ], +) +def test_check_index_name_rejects_urls_paths_options_and_bad_refs(ref: str) -> None: + from stoat_mcp import guards + + with pytest.raises(GuardRejection): + guards.check_index_name(ref) diff --git a/mcp/tests/test_server.py b/mcp/tests/test_server.py index 52b6e4e9..17361b0b 100644 --- a/mcp/tests/test_server.py +++ b/mcp/tests/test_server.py @@ -24,18 +24,22 @@ "list_recipes": {"readOnlyHint": True, "destructiveHint": False}, "check_recipes": {"readOnlyHint": True, "destructiveHint": False}, "logs": {"readOnlyHint": True, "destructiveHint": False}, + "search_recipes": {"readOnlyHint": True, "destructiveHint": False}, "doctor": {"readOnlyHint": True, "destructiveHint": False}, "plan_recipes": {"readOnlyHint": True, "destructiveHint": False}, "create": {"readOnlyHint": False, "destructiveHint": False}, "start": {"readOnlyHint": False, "destructiveHint": False}, "stop": {"readOnlyHint": False, "destructiveHint": False}, "update": {"readOnlyHint": False, "destructiveHint": False}, + "add_recipe": {"readOnlyHint": False, "destructiveHint": False}, + "update_recipe": {"readOnlyHint": False, "destructiveHint": False}, "clone": {"readOnlyHint": False, "destructiveHint": False}, "snapshot": {"readOnlyHint": False, "destructiveHint": False}, "forward": {"readOnlyHint": False, "destructiveHint": False}, "wait": {"readOnlyHint": False, "destructiveHint": False}, "destroy": {"readOnlyHint": False, "destructiveHint": True}, "prune": {"readOnlyHint": False, "destructiveHint": True}, + "remove_recipe": {"readOnlyHint": False, "destructiveHint": True}, "restore": {"readOnlyHint": False, "destructiveHint": True}, # A recipe body is arbitrary guest code, so apply_recipes carries exec's # hints and exec's allow_exec check. @@ -187,3 +191,45 @@ def test_forward_refuses_a_pair_that_kong_reads_as_a_flag(fake_client): with pytest.raises(ToolError): server.forward("work", pairs=["--clear"]) assert fake_client.calls == [] + + +@pytest.mark.parametrize("term", ["-tail", "--json", "--refresh", "--"]) +def test_search_recipes_preserves_a_leading_dash_as_data(fake_client, term): + server.search_recipes(term) + assert fake_client.calls == [("recipe", "search", "--", term)] + + +def test_add_recipe_accepts_a_slash_containing_ref_and_uses_variadic_argv(fake_client): + server.add_recipe("tailscale", ref="feature/topic") + assert fake_client.calls == [("recipe", "add", "tailscale@feature/topic", "-y")] + + +@pytest.mark.parametrize( + "call", + [ + lambda: server.add_recipe("https://github.com/x/stoat-tailscale"), + lambda: server.add_recipe("tailscale@../escape"), + lambda: server.add_recipe("tailscale@feature..topic"), + lambda: server.add_recipe("tailscale@feature/.hidden"), + lambda: server.add_recipe("tailscale@feature/topic.lock"), + lambda: server.add_recipe("-y"), + lambda: server.update_recipe("tailscale@v1.2"), + lambda: server.remove_recipe("../tailscale"), + lambda: server.remove_recipe("-y"), + ], +) +def test_recipe_tools_refuse_unsafe_names_before_cli(call, fake_client): + with pytest.raises(ToolError): + call() + assert fake_client.calls == [] + + +def test_update_and_remove_recipe_use_plain_names_and_remove_has_no_force(fake_client): + server.update_recipe("tailscale") + server.update_recipe() + server.remove_recipe("tailscale") + assert fake_client.calls == [ + ("recipe", "update", "tailscale"), + ("recipe", "update"), + ("recipe", "rm", "tailscale", "-y"), + ]